CVG-Net: novel transfer learning based deep features for diagnosis of brain tumors using MRI scans

Machine learning models analysis

The study Stadlbauer et al. (2022) investigates the potential of employing multiclass machine learning (ML) algorithms on an extensive array of radiomic features derived from advanced MRI (advMRI) and physiological MRI (phyMRI) data. This innovative methodology, referred to as radiophysiomics, aims to achieve precise classification of contrast-enhancing brain tumors. The study leveraged a substantial MRI database, encompassing more than 1700 MR examinations of brain tumor patients, for the purpose of model development. The findings revealed that the application of adaptive boosting and random forest algorithms in conjunction with both advMRI and phyMRI data surpassed human reading in various performance metrics, including accuracy (0.875 vs. 0.850), precision (0.862 vs. 0.798), and F-score (0.774 vs. 0.740). Nevertheless, it was observed that radiologists demonstrated higher sensitivity (0.767 vs. 0.750) and specificity (0.925 vs. 0.902) compared to the machine learning models.

The study Latif et al. (2022) introduces an innovative method for categorizing glioma tumors, utilizing features based on deep learning in conjunction with a support vector machine (SVM) classifier. Through the application of a deep convolutional neural network for feature extraction from MR images sourced from the BraTS dataset, the study achieved noteworthy levels of accuracy in classification. Specifically, an accuracy of 96.19% was observed for the HGG Glioma type using the FLAIR modality, and a commendable accuracy of 95.46% was obtained for the LGG glioma tumor type when employing the T2 modality. Notably, this method encompassed the classification of four distinct glioma classes, including edema, necrosis, enhancing, and non-enhancing. This pioneering approach shows promising potential in the domain of glioma tumor classification.

Deep neural networks models analysis

This study Archana, Karthigha & Suresh Lavanya (2023) introduces a comparative analysis of several optimizers employed in CNNs for the purpose of brain tumor detection. Within deep learning (DL), artificial neural networks (ANNs) play a crucial role in discerning brain tumors. The medical image dataset is employed for applied methods training and evaluation. Among the various approaches investigated, the application of the AlexNet architecture in conjunction with the stochastic gradient descent (SGD) optimizer demonstrated superior performance. This proposed approach achieved a diagnostic accuracy of 0.80.

This article Sharif et al. (2020) introduces a novel approach to actively employ deep learning for the segmentation and recognition of brain tumors in MRI images. The authors utilize the pre-trained Inception V3 CNN model for robust deep feature extraction during the classification phase. These features are integrated with the dominant rotated local binary pattern (DRLBP) to enhance texture analysis. Subsequently, a particle swarm optimization (PSO) technique is employed to fine-tune the concatenated feature vector for classification using a softmax classifier. The study is conducted in two phases. In the initial phase, the segmentation approach (SbDL) is validated on BRATS2017 and BRATS2018 datasets, demonstrating promising Dice scores. For instance, the BRATS2017 dataset achieves Dice scores of 83.73% for the core tumor, 93.7% for the whole tumor, and 79.94% for the enhanced tumor. Moreover, the BRATS2018 dataset attains a Dice score of 88.34%. In the subsequent phase, the classification strategy is applied to BRATS2013, 2014, 2017, and 2018 datasets, yielding an average accuracy of over 92%.

The study Kibriya et al. (2022) is focused on developing an automated method for precise brain cancer identification through tumor diagnosis. Previous systems have struggled with issues related to subpar accuracy and high false-positive rates. To address this, the study introduces a pioneering 13-layer CNN architecture specifically designed for classifying brain tumors from MRI scans. The effectiveness of the proposed model was assessed using a standardized dataset comprising 3064 MRI images encompassing three distinct categories of brain cancer: glioma, pituitary, and meningioma. The results demonstrated an average accuracy of 97%.

The study Fabelo et al. (2019) introduces a novel framework centered on deep learning techniques, tailored for the analysis of hyperspectral images depicting live human brain tissue. Through rigorous evaluation using a diverse database of 26 in vivo hyperspectral cubes from 16 individual patients, encompassing a total of 258,810 annotated pixels, the framework proves adept at generating a thematic representation that delineates the brain’s parenchymal area and accurately identifies the tumor’s position. This information serves as a crucial aid to surgeons, significantly enhancing the precision of tumor resection procedures. Notably, the deep learning pipeline 2D convolutional neural network exhibits an impressive overall accuracy of 80% for multiclass classification, which is low compared to baseline.

Hybrid models analysis

The author of this Sudharson et al. (2022) study proposed a hybrid deep learning-based neural system for the detection of brain tumors. The study introduced fundamental concepts of image separation to address the challenges associated with partitioning brain MRI images. Various MRI pre-processing techniques were then elaborated, encompassing tasks such as image registration, bias field correction, and non-brain tissue removal. Additionally, the author presented a hybrid CNN-based partitioning approach, which proved advantageous for delineating brainstem tumors in MRI scans when augmented with prior knowledge. The proposed approach demonstrated an accuracy score of 0.93.

The study Yazdan et al. (2022) introduces an innovative approach to diagnosing brain tumors. It employs a dual-pronged strategy, utilizing a multi-scale CNN (MSCNN) architecture to develop a robust classification model, while also addressing the impact of Rician noise on MSCNN performance. This model enables the multi-classification of MRIs, distinguishing between glioma, meningioma, pituitary, and non-tumor cases. Additionally, a denoising process is implemented using a fuzzy similarity-based non-local means (FSNLM) filter to enhance classification results. The study utilizes an openly accessible MRI dataset comprising 3,264 MRIs. Experimental results demonstrate that the proposed MSCNN model outperforms both AlexNet and ResNet, exhibiting superior accuracy and efficiency while incurring lower computational expenses. Specifically, the MCNN2 iteration achieves an impressive accuracy and F1-score of 91.2% and 91%, respectively.

The analyzed works with their research limitations are summarized in Table 1.

Multi class brain tumors MRI image data

This research used multiclass MRI scan images (Hashemi, 2023) related to brain tumors to conduct experiments. The dataset comprises 21,672 MRI scan images. Figure 2 illustrates the distribution of multi-class target labels. This analysis reveals that the ‘Normal’ target class (0) contains 3,066 samples, the ‘Glioma_tumor’ target class (1) contains 6,307 samples ‘Meningioma_tumor’ target class (2) contains 6,391 samples, and the ‘Pituitary_tumor’ target class (3) contains 5,908 samples. Additionally, we conducted a sample image analysis, as shown in Fig. 3.

We utilized the state-of-the-art dataset: Multi-class Brain Tumour MRI Image Data, comprising 21,672 MRI scan images. We chose this dataset because it aligns with the problem addressed in our research study, specifically, brain tumor detection. Moreover, this dataset encompasses a substantial number of images, enhancing the generalizability of our research findings.

Novel transfer learning approach

This section comprehensively describes our novel proposed CVG-Net approach for transfer feature engineering. The step-by-step architectural workflow of the proposed approach is illustrated in Fig. 4. The brain tumor MRI image data is initially fed into the VGG16 approach, which extracts rich-level spatial features based on transfer learning. Similarly, the brain tumor MRI image data is input into the 2DCNN approach, which also extracts rich-level spatial features using transfer learning. The spatial features obtained from both approaches are then combined to create a new feature set. This newly generated feature set is subsequently employed in building the machine learning models applied in this study and for further results evaluations.

The Algorithm 1  shows the step-by-step workflow of the proposed transfer learning feature engineering.

 
_________________________________________________________________________________________________ 
 
Algorithm 1 CVG-Net Algorithm 
Input:  Multi-class MRI scans related to brain tumors. 
Output:  New Transfer learning-based deep features. 
initiate; 
1-  Fcnn  ←− CNNprediction(MiS)   // MiS ϵ MRI imagesset,    here MiS is 
original image data and Fcnn is the deep spatial feature set extracted. 
1-  Fvgg16  ←− V GG16prediction(MiS)  // MiS ϵ MRI imagesset,    here MiS 
is original image data and Fvgg16 is the deep spatial feature set extracted. 
3-  FSpat  ←− Fcnn + Fvgg16   // here FSpat is final hybrid deep spatial features 
set used for osteoarthritis detection. 
end; 
__________________________________________________________________________________________________    

SMOTE based data balancing

SMOTE has emerged as a pivotal tool in addressing class imbalance issues within medical image analysis, particularly in diagnosing brain tumors using multi-class MRI image datasets (Li et al., 2023b). Class imbalance, where certain classes are underrepresented relative to others, can significantly hinder the performance of machine learning models, as they tend to be biased towards the majority class. In medical imaging, this can have critical implications for patient care. SMOTE operates by generating synthetic samples in the feature space of the minority class, thereby alleviating the imbalance. When applied to multi-class MRI datasets for brain tumor diagnosis, SMOTE ensures that the model is exposed to a more representative data distribution, effectively discriminating between various tumor types. This methodology has demonstrated promising results, contributing to more accurate and reliable diagnoses, paramount in clinical decision-making processes.

Applied neural network methods

The application of artificial intelligence (AI) based machine learning, deep learning, and transfer learning methods has emerged as a transformative approach in medical imaging (Rehman et al., 2023; Qadri et al., 2023; Raza et al., 2022b), particularly in diagnosing brain tumors using brain MRI scans. These advanced techniques have revolutionized the way medical professionals analyze and interpret complex neuroimaging data. Machine learning algorithms can efficiently extract intricate patterns and features from large datasets of MRI scans, enabling them to differentiate between normal brain tissue and pathological regions associated with brain tumors.

Deep learning, in particular, has been instrumental in automating the tumor detection, segmentation, and classification process, as it can automatically learn hierarchical representations from raw MRI data, making it highly effective in identifying subtle abnormalities (Abunadi & Kumar, 2021; Inbarani H & Azar, 2020; Azar et al., 2023). Transfer learning techniques leverage pre-trained models on large datasets and have shown promise in enhancing diagnostic accuracy by transferring knowledge learned from other domains or medical imaging tasks to improve the performance of brain tumor diagnosis. Integrating AI-driven methods in brain tumor diagnosis accelerates the detection process and aids healthcare providers in making more accurate and timely decisions.

In this research, we employed several advanced machine learning and deep learning-based approaches for performance evaluations. We utilized two neural network approaches, 2D-CNN and VGG16, for both classification and transferring feature mechanisms from MRI images. The machine learning models we employed include the k-neighbors classifier, logistic regression, and random forest. We have provided a detailed description of the working mechanisms of each applied neural network in Table 2 and layer architecture in Table 3.

Hyperparameter setting

Hyperparameter tuning is a crucial aspect of optimizing neural network models in machine learning. These hyperparameters are configurations external to the model and can significantly impact its performance. In our research, we focus on systematically exploring and optimizing these hyperparameters to enhance the overall efficiency and effectiveness of neural networks. We employ a recursive process of training and evaluations to determine the best-fit hyperparameters. The selected hyperparameters for the neural network applied to diagnose brain tumors are presented in Table 4.

Experimental setup

Our proposed experimental study setup includes the use of the Google Colab platform to evaluate research results. The platform offers GPU backend support, providing 90 GB of disk space and 13 GB of RAM. We have employed the Python 3.0 programming language to build the neural network used in our research. We utilized the Sklearn, Tensorflow, and Keras modules for evaluating the results. During performance evaluations, true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) are used as basic parameters. The performance metrics used for performance evaluation are examined as follows:

• Accuracy: in diagnosing brain tumors using brain MRI scans refers to measuring how effectively the diagnostic system or method correctly identifies and classifies the presence or absence of brain tumors in the MRI images. (1)${\displaystyle Accuracy=\frac{TP+TN}{TP+TN+FP+FN}.}$

• Precision: in diagnosing brain tumors using brain MRI scans refers to measuring how accurately the diagnostic test identifies TP cases among all the positive cases it detects. (2)${\displaystyle \text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}.}$

• Recall: in diagnosing brain tumors using brain MRI scans, is a fundamental performance metric that measures the ability of a diagnostic model or process to correctly identify and classify TP cases of brain tumors among all the actual cases of brain tumors present in the MRI scans. (3)${\displaystyle \text{Recall}=\frac{TP}{TP+FN}.}$

• F1: in diagnosing brain tumors using brain MRI scans, is a measure that combines precision and recall to assess the accuracy of the classification model. (4)${\displaystyle F1Score=\frac{2\cdot Precision\cdot Recall}{Precision+Recall}.}$

• Runtime computations: is a neural network method’s brain tumor diagnosis time by inputting MRI images.

• Standard deviations (SD): refers to a statistical measure of the variability of the evaluation metrics obtained from multiple folds of the dataset during the cross-validation process. (5)${\displaystyle \sigma =\sqrt{\frac{1}{N}\sum _{i=1}^N{\left(x_i-\mu \right)}^2}.}$

Performance analysis with classical approaches

The performance analysis of the applied classical deep learning-based approaches, 2DCNN and VGG16, is conducted in this section. MRI brain scan images are fed into the 2DCNN and VGG16 methods, and the results are then evaluated.

Results of CNN model

The performance results of the applied CNN model during training are illustrated in Fig. 5. We conducted a time series analysis over 20 epochs of the CNN model. The evaluation is based on loss scores, accuracy, precision, and recall. The analysis reveals that the training loss is very high in the first two epochs. Subsequently, there is a notable reduction in loss, while the validation loss gradually increases. Similarly, the training accuracy starts off low in the initial two epochs, but then exhibits a steady increase. The validation accuracy scores fluctuate around 0.80. The precision and recall metrics exhibit a similar pattern to the accuracy performance scores throughout training. In conclusion, this analysis indicates that the CNN model achieved acceptable scores for brain tumor diagnosis during training. Nevertheless, there is still a need for performance enhancement.

The performance analysis of the applied CNN model for unseen testing data is described in Table 5. The analysis shows that the CNN model achieved an acceptable score of 0.76 during brain tumor diagnosis. However, the loss scores are high for the testing data. The class-wise performance scores also demonstrate moderate results. This analysis concludes that classical approaches have not proven fruitful in this case; there is a need for advanced mechanisms for performance enhancement.

Table 5:

The results analysis of applied 2D-CNN neural network.

Accuracy	Loss	Target class	Precision	Recall	F1
0.76	3.64	Normal	0.72	0.76	0.74
		Glioma_tumor	0.70	0.68	0.69
		Meningioma_tumor	0.78	0.75	0.77
		Pituitary_tumor	0.87	0.87	0.87
		Average	0.77	0.76	0.77

DOI: 10.7717/peerjcs.2008/table-5

Results of VGG16 model

The performance results of the applied pre-trained VGG16 model during training are illustrated in Fig. 6. We conducted a time series analysis over 20 epochs of the VGG16 model. The analysis reveals that the training loss is very high in the first five epochs. Subsequently, there is a notable reduction in loss, while the validation loss gradually increases. Similarly, the training accuracy starts off low in the initial two epochs, but then exhibits a steady increase. The validation accuracy scores fluctuate around 0.92. The precision and recall metrics exhibit a similar pattern to the accuracy performance scores throughout training. In conclusion, this analysis indicates that the VGG16 model achieved good scores for brain tumor diagnosis during training. Nevertheless, there is still a need for performance enhancement.

The performance analysis of the applied VGG-16 model on unseen testing data is outlined in Table 6. The analysis reveals that the VGG-16 model achieved a commendable score of 0.90 in brain tumor diagnosis. Nevertheless, the loss scores were notably high for the testing data. The class-wise performance scores also demonstrate positive results, with 0.91 precision and F1-scores. This analysis leads to the conclusion that classical approaches yielded a moderately satisfactory performance. However, there is a pressing need for advanced mechanisms to enhance performance.

Table 6:

The results analysis of applied VGG-16 neural network.

Accuracy	Loss	Target class	Precision	Recall	F1
0.90	2.66	Normal	0.99	0.89	0.94
		Glioma_tumor	0.91	0.85	0.88
		Meningioma_tumor	0.83	0.93	0.87
		Pituitary_tumor	0.96	0.95	0.95
		Average	0.91	0.90	0.91

DOI: 10.7717/peerjcs.2008/table-6

Performance analysis with Novel approach

The performance analysis of the applied advanced machine learning methods with the novel proposed CVG-Net feature engineering is presented in this section. The proposed approach extracts novel transfer features by inputting MRI brain scan images into the 2DCNN and VGG16 methods, creating a hybrid feature set. The newly created features are then input into machine learning models, and the results are subsequently evaluated.

Results with unbalanced features

In this section, the novel extracted features that are unbalanced are inputted into applied machine learning methods for performance evaluations. The performance results of machine learning methods with unbalanced features are described in Table 7. The analysis demonstrates that applied machine learning models achieved high-performance scores using the novel extracted transfer features. The LR and RF achieved accuracy performance scores of 0.81 and 0.85, respectively. The applied KCN approach achieved a good score of 0.94. This analysis concludes that with unbalanced novel features, good performance is achieved. However, the performance can be further improved by applying advanced data-balancing approaches.

Results with balanced features

After evaluating the performance scores with unbalanced feature data, we applied the famous data balancing Synthetic Minority Oversampling Technique (SMOTE) to the extracted feature data. The evaluation results of the applied machine learning approach with a balanced dataset are demonstrated in Table 8. The analysis reveals that balanced features help improve performance scores for the diagnosis of brain tumors. The proposed KNC method outperformed other applied methods with a high accuracy of 0.95 in comparison. This analysis concludes that SMOTE-based dataset balancing helps us improve performance scores.

The confusion matrix allows visualization of the performance of applied machine learning methods for the diagnosis of brain tumors. The overall performance validation, based on the confusion matrix, is illustrated in Fig. 7. The analysis shows that the applied LR and RF have high wrong predictions for target classes 2 and 3. The minimum error rate, indicating the fewest wrong predictions, is achieved by the proposed K-nearest neighbors classification (KNC) approach. This analysis validates the high-performance scores of the proposed KNC method for the diagnosis of brain tumors using MRI scans.

Validating performance with k-fold mechanism

We have validated the performance of applied machine-learning approaches in this section using both unbalanced and balanced feature data. The results of k-fold cross-validations are demonstrated in Table 9. To validate accuracy performance results, we used a fold split of 10 and determined the outcomes. The analysis shows that with unbalanced features, LR and RF achieved acceptable validation accuracy scores in the range of 0.81 to 0.84. The proposed approach achieved a commendable score of 0.94. When we evaluated results with the balanced dataset, performance scores increased for each applied approach. The analysis concludes that the proposed KNC approach achieved the highest k-fold accuracy scores of 0.96 for the diagnosis of brain MRI scans.

Computational complexity analysis

In this section, we present a computational complexity analysis of applied machine learning methods. The runtime computations, measured in seconds, of these applied machine learning methods, are determined during each model-building process using a newly created features dataset. The computation results are reported in Table 10. Our analysis demonstrates that the proposed KNC method achieved the minimum runtime computations compared to other applied methods. The proposed KNC method takes only 0.0281 s to diagnose brain tumors.

Comparison with state-of-the-art studies

For a fair performance comparison, we have included state-of-the-art studies in brain tumor diagnosis in this analysis. The performance comparison of our proposed approach is illustrated in Table 11. The studies considered for performance comparison are from the years 2020, 2022, and 2023. The analysis reveals that our proposed approach outperformed state-of-the-art studies with high-performance scores of 0.96 for the diagnosis of brain tumors using MRI brain scans.

Discussions

In this research, we have introduced a novel neural network method, CVG-Net, for feature engineering and the timely diagnosis of brain tumors. This proposed method combines 2DCNN and VGG16 to extract spatial features from MRI images. Our experiments utilized a multi-class MRI image dataset comprising 21,672 images related to glioma tumors, meningioma tumors, and pituitary tumors. Our commitment to privacy and confidentiality underscores the implementation of robust data anonymization techniques, thereby mitigating the risk of any inadvertent disclosure of sensitive personal details. This explicit ethical discourse is pivotal not only in upholding the integrity of our research but also in fostering trust within the scientific community and, more importantly, among the individuals whose health data form the foundation of our study.

Study limitations

In this research study, we have introduced a novel CVG-Net for the prompt diagnosis of brain tumors. Nevertheless, our proposed research does have some limitations. The accuracy score of our approach, currently at 96%, could be further improved by minimizing the loss error rates. To address potential overfitting issues or biases in the machine learning models, we have employed the k-fold cross-validation mechanism, which validates the performance accuracy scores. Moreover, the deep learning-based neural network models we utilized are computationally expensive and can be optimized by reducing their layered architectures.

Future work

In the future, we intend to design a graphical user interface app for medical specialists to facilitate the effective diagnosis of brain tumors using MRI scans. Also we plan to enhance the performance by implementing transfer learning-based advanced neural networks. The designed app will utilize the proposed approach in the backend for diagnosis.