Joint classification and regression with deep multi task learning model using conventional based patch extraction for brain disease diagnosis

Motivation of this research

• Brain imaging data is multidimensional and complex. The hybrid classification and regression method optimize the use of available data, potentially important to enhanced diagnostic insights and more robust models.

• Patch-based extraction approaches can concentrate on crucial areas of brain imaging, improving the feature extraction process. DMTCNN can successfully acquire and utilize these properties to enhance diagnostic performance.

• Effective therapy depends on a timely and precise diagnosis of brain diseases. With the potential to predict sickness progression and identify early signs, the DMTCNN-based approach enables prompt and suitable therapies.

• The regression portion of DMTCNN can provide unique insights into illness severity and progression, allowing for customized treatment approaches for individual patients.

The main contribution of this research

• To progress the detection of brain diseases, quantitative disease-related measures like tumor volume or disease severity are predicted by the proposed DMTCNN model.

• Compared to single-task models, the proposed model that addresses both classification and regression tasks captures complex patterns, leading to more accurate and informative diagnoses.

• Targets important regions associated with brain illnesses by minimizing computing overhead and increasing efficiency through the extraction of smaller, targeted patches from brain imaging data.

• Faster diagnosis times are achieved by the integration of multi-task learning and patch extraction, which is crucial in clinical situations where timely decision-making is essential.

• The proposed model learns many tasks concurrently, such as categorizing different brain diseases or anomalies, by extracting features from image patches using CNNs.

• Finally, the suggested DMTCNN model performs better at accurately recognizing brain illnesses than the most advanced methods.

The remainder of our study expands on this structure: The literature evaluation is covered in “Literature Survey”, and the proposed research is summarized in “Proposed System”. The datasets, evaluation standards, and experimental findings are shown in “Result and Discussion”. Finally, “Conclusion” provides a summary of this work’s conclusions.

Limitations of existing system

• Large amount of annotated data is needed to train medical DL models for classification and regression. The model’s robustness and accuracy are limited by the difficulty of obtaining labeled brain imaging data with diagnostic and severity labels.

• Despite advances in execution time, MTL model training still demands a lot of computer power, especially with big, high-resolution brain imaging datasets. This may limit accessibility in resource-constrained or smaller clinical environment.

• Some diagnostic procedures, such as biopsies, are intrusive and can cause difficulties and discomfort for patients.

• Some diagnostic approaches, particularly those that require laboratory analysis or complex imaging, can be time-consuming, delaying diagnosis and treatment.

• Many systems are not equipped to detect brain disorders at the earliest stages, resulting in inadequate patient outcomes.

Dataset

We assessed the proposed methodology in this work using two distinct brain MRI datasets. Our primary dataset, the "Figshare brain tumor dataset," was located at (Cheng, 2017). It is well known for having one of the biggest collections and for being an essential tool in the diagnosis of brain cancers. This collection includes 3,064 genuine brain MRI samples from 233 people. Of these, the meningioma class includes 708 samples, the pituitary class includes 930 examples, and the gliomas class includes 1,426 samples. Every image has the same dimensions of 512 × 512 pixels. In contrast, the Brain MRI Dataset was reduced in size and distributed in Kaggle (2020), Wang et al. (2024). There are 253 MRI samples in all, with a pixel size of 845 by 845. This dataset includes 155 MRI samples with tumors. It is important to remember that the MRI examples show a variety of structural complexity, devices employed, noise levels, bias field effects, and other features. The T1-weighted MRI image collection was chosen because of its contrast-enhanced quality. As a result, it simplifies distinguishing the tumor-affected areas, making them preferred for treatment planning. The proposed DMTCNN approach requires brain MRI scans to diagnose neurological disorders. Weighted according to T1, T2, FLAIR, and Diffusion These photos include imaging MRI sequences. Each sequence displays different brain tissue features, helping identify tumors, atrophy, and lesions. T1 scans provide anatomical details, but T2- and FLAIR images show aberrant brain fluid accumulations, which are frequent in multiple sclerosis and strokes. The deep learning model can identify disease-relevant features for classification (e.g., detecting Alzheimer’s or tumors) and regression (e.g., predicting disease severity or progression) by extracting patches from these MRI types, improving diagnostic accuracy and brain health assessment.

Pre-processing

The Canny edge detector is a pre-processing approach for detecting edges in images. It is particularly successful in distinguishing significant edges from noise by employing a number of stages such as Gaussian smoothing, gradient calculation, non-maximum suppression, and hysteresis thresholding.

In order to assist in the diagnosis of brain disorders, the Canny edge detector advances the edges of brain pictures, such as MRI or CT scans. This improvement aids in detecting structural anomalies or lesions that may designate diseases such as tumors, hemorrhages, or neurodegenerative disorders. The detector is well-defined by the steps below:

1. Gaussian smoothing: Smooths the image to reduce noise and annoying details.

(1) $$Smoothed\mathrm{\_}Image(x,y)=(f\ast G)(x,y)$$where f is the original image and G is the Gaussian kernel.

2. Gradient calculation: Computes the gradient magnitude and orientation to find potential edges.

(2) $$Gradient\mathrm{\_}Magnitude(x,y)=\sqrt{G_x{(x,y)}^2+G_y{(x,y)}^2}$$

(3) $$Graident\mathrm{\_}Orientation(x,y)={\tan }^{-1}\left({\displaystyle \frac{G_y(x,y)}{G_x(x,y)}}\right)$$where $G_x$ and $G_y$ are the derivatives of f in the x and y directions.

3. Non-maximum suppression: Thins down the edges to single pixel width to accurately locate edge points.

(4) $$Edge\mathrm{\_}Strength(x,y)=\{\begin{array}{cc}Gradient\mathrm{\_}Magnitude(x,y)\hfill & ifthepixelisalocalmaxim\hfill \\ 0\hfill & otherwise\hfill \end{array}$$

4. Hysteresis thresholding: Decides which edge pixels are part of the actual edges by applying two thresholds.

(5) $$Edge\mathrm{\_}Map(x,y)=\{\begin{array}{cc}Strong\mathrm{\_}Edge\hfill & ifEdge\mathrm{\_}Strength(x,y)\ge High\mathrm{\_}Threshold\hfill \\ Weak\mathrm{\_}Edge\hfill & ifLow\mathrm{\_}Threshold\le Edge\mathrm{\_}Strength(x,y)<1\hfill \\ No\mathrm{\_}Edge\hfill & Otherwise\hfill \end{array}$$

The first step in the multi-stage process used by the Canny edge detector is Gaussian smoothing, which smooths and reduces noise in the image to increase edge detection accuracy. The gradient of the picture is then computed using Sobel operators to ascertain the direction and size of intensity variations. The identified edges are then thinned via non-maximum suppression, leaving only the local maxima, aiding in the creation of a more lucid edge representation. To categorize pixels into strong edges, weak edges, and non-edges, the algorithm uses double thresholding, making sure that only the most important edges are kept. Finally, weak edges are eliminated and continuity is ensured by edge tracking using hysteresis, which links weak edges to strong ones. The Canny edge detector is an essential tool for a wide range of applications, including object recognition, image segmentation, and feature extraction in robotics, autonomous systems, and medical imaging. Its comprehensive approach reduces the likelihood of false edge detection and yields well-defined, thin edges. Because of its precision and robustness, it has become a typical technique for edge detection tasks.

Deep multi-task convolutional neural network

However, most practical applications do not enable task grouping ahead of time. The feature-based grouping approach is limited to measuring task correlation. When several tasks need to be put together, the situation becomes more difficult. We need a more universal approach. Additionally, in the context of multi-task learning, collaborative feature acquisition is more important than the degree of task sharing. Regarding the output, the objective of supervised learning is to extract and represent the pertinent data from the input variable. A model has a limited capacity to communicate information. Thus, if the jointly learned tasks are highly diverse, the model might become overfitted. As a result, the extent to which activities share information should be variable, particularly in situations where the relationships between the grouped tasks are not entirely evident.

We suggest using the deep multi-task CNN model to address the challenges presented by multi-task learning issues. The primary dissimilarity amongst this approach and the traditional CNN model is the number of deep connections between subnets of various tasks. Within a work group, the lower layers share supervisory signals from the higher layers. Gradient descent modifies the effect of a guiding signal in a specific lower layer to facilitate the learning of task transfer connections (TTC). This method automatically categorizes tasks into nuanced categories during the training phase. As a result, task linkages are no longer evaluated in a simple binary framework of related or unconnected; rather, they are measured based on their level of significance (Rohani, Farsi & Mohamadzadeh, 2023). The suggested model degenerates into numerous single-task learning models when the TTCs within a similar subnet ${\alpha }^{pq}\left(p=q\right)$ are 1 and those across tasks are 0. The proposed model becomes an average multi-task CNN model when the TTC factors ${\alpha }^{pq}\left(p,q\in taskset\right)$ are the same.

The function $f(.)$ of weights $W$ and input $A$ represents the standard CNN model:

(6) $$f\left(A,W\right).$$

The purpose is to reduce the discrepancy between the output $B$ of $f\left(A,W\right)$ and the ground truth:

(7) $$min\left(D\right)$$where:

(8) $$D=-\left[B\ln \left(F\right)+\left(1-B\right)\ln \left(1-F\right)\right].$$

A function of $A$, W, and TTC $\alpha$ is used to express the deep multi-task CNN model in the presence of $n$ tasks:

(9) $$F\left(A_1,A_2,....,A_n,W_1,W_2,....,W_n,\alpha \right).$$

Additionally, the multi-task model aims to minimize the cost of each task

(10) $$min\left(\sum _{p=1}^n\left(D_p\right)\right)$$where:

(11) $$D_p=-\left[B_p\ln \left(F_p\right)+1\left(1-B_p\right)\ln \left(1-F_p\right)\right]$$

(12) $$F\left(A_p,W_1,W_2,...,W_n,\alpha \right).$$

As the definition of the TTC factor, $\alpha$ is projected to be between 0 and 1. Consequently, we introduce auxiliary variable $\beta$ where:

(13) $$\alpha =sigmoid\left(\beta \right),\beta \in \left(-\mathrm{\infty },+\mathrm{\infty }\right).$$

Our multi-task model differs from the traditional CNN model primarily in how weights are updated during training. This is because, in our paradigm, every subnet must consider supervisory signals from both other subnets and its own subnet. Several optimization techniques, including adaptive gradient, RMSprop, and stochastic gradient descent, have varying weight updating details. As such, we only display the generic forms. Weight updating in the traditional CNN model can be shown as follows:

(14) $$W_{t+1}=W_t+V_{t+1}.$$

Algorithm 1 is used in our framework to update the weights, providing weights for all ${\alpha }^{pq}$ components in the TTC, n positions, and $W^1,W^2,......,W^n$.

The first algorithm uses ${\beta }_t^{pq}$ as an auxiliary variable, $V_{t+1}^{W^{pq}}$ as the $W_{t+1}^p$ and BPQ update values for a particular optimization algorithm to determine the cost of task $q$, ${\alpha }_t^p$ as the corresponding $t+1$ factor of ${\beta }^{pq}$ to task $q$, which regulates the effect of the supervisory signal from task $q$ on task $p$, and $W^p$ and $W_t^p$ as the weights at iteration $t$ and $t+1$, respectively.

(15) $$V_{t+1}^{W^{pq}}={\displaystyle \frac{\mathrm{\partial }{D}_q}{\mathrm{\partial }{W}^q}.{\displaystyle \frac{\mathrm{\partial }{W}^q}{\mathrm{\partial }{W}^p}}}$$ (16) $$V_{t+1}^{{\beta }^{pq}}={\displaystyle \frac{\mathrm{\partial }{D}_q}{\mathrm{\partial }{\alpha }^{pq}}.{\displaystyle \frac{\mathrm{\partial }{\alpha }^{pq}}{\mathrm{\partial }{\beta }^{pq}}.}}$$

Particularly, supervisory signals from other subnets will be weaker in order to maintain the dominance of information from the subnet itself. Consequently, when $p=q$, ${\alpha }_t^{pq}$ is set to 1. Batch normalization is used to reduce feature value range volatility.

Our model is superior to the conventional training strategy because it can add additional tasks incrementally while maintaining the completeness of each task branch. As training progresses, the subnets of the newly assigned tasks link to the taught tasks and learn their own parameters in the process. The parameters of the associated subnets are fixed for the tasks that have already been learned. Figure 2 depicts a scenario in which two tasks have been trained and the model has been updated to include a novel task.

Advantages of proposed method

• Sharing representations across tasks is one way that multitask learning might result in more effective learning processes. Due to the network’s improved ability to generalize from the shared data, this can lower the risk of overfitting, particularly in situations with a lack of labeled data.

• The patch-based extraction technique may improve the network’s capacity to identify minute alterations linked to brain disorders by concentrating on pertinent local brain regions. This specialized strategy may result in the identification of anomalies that are more sensitive and exact.

• The computing cost can be greatly reduced by extracting patches rather than processing complete brain scans, which makes the technology more practical for large-scale or real-time applications. Additionally, it makes it possible to analyze particular brain regions in greater detail and with greater focus. Table 1 shows that description of mathematical symbols used in this research.

Experimental setup

The Python development environment utilized for the studies was the Google Colaboratory Pro (Google Colaboratory, 2019) platform, also known as Colab Pro. With this cloud service from Google, users can create and run Python programs on a hosted GPU. A ratio that can be used is 70% of the data for training, 15% for validation, and 15% for testing. The model is capable of learning features for both classification and regression tasks from the training set, which contains the majority of the data. This ensures that the DMTCNN model can generalize effectively to new data. During the training phase, the validation set is employed to evaluate the model on unobserved data in order to fine-tune hyperparameters and prevent overfitting. Lastly, the model’s performance on both classification and regression tasks in a real-world context is evaluated using the test set, which is entirely isolated from the training process. The model’s robust performance when applied to new, unseen cases is critical for reliable brain disease diagnosis, and this balanced approach ensures that it not only learns efficiently but also performs robustly. In Table 2 shows outlines typical parameter values for DMTCNN model using conventional patch extraction for brain disease diagnosis.

Comparative methods

Deep neural network (DNN) (El-Assy et al., 2024): Prototype learning was added to the supervised and unsupervised learning stages of the DNN model’s fine-tuning procedure in order to improve both intra-class similarity and inter-class differentiation in feature representations.

Deep convolutional neural network (DCNN) (Islam & Zhang, 2018): A recent approach uses a binary classifier to gain more efficiency in early-stage analysis and to correctly identify different stages of AD.

Alzheimer’s disease diagnosis using enhanced manta ray foraging optimization based deep learning (ADD-EMRFODL) (Pusparani et al., 2023): GF is used in the ADD-EMRFODL method as an initial processing step. Additionally, it generates feature vectors using the DenseNet-121 model.

Mask region-based convolution neural network mask R-CNN (He et al., 2017): Tumor delineation accuracy is greatly improved by this technique, which uses area-based segmentation to create a mask rather than relying on a threshold or border model for precise segmentation.

Performance metrics

True positives (TP) are instances in which the classifier accurately detects positive cases. True positives (TP) are situations that have been appropriately identified as positive. False positives (FPs) occur when negative cases are incorrectly labeled as positive. FP stands for false positives. True negatives (TN) are occasions in which the classifier correctly detects negative cases. The count of true negatives is referred to as TN. Positive occurrences that are wrongly labeled as negative are known as false negatives (FN). The number of false negatives is indicated by FN.

Accuracy: In contrast to the total number of samples evaluated, it assesses the proportion of correctly classified samples, both positive and negative.

(17) $$ACC=\left(TP+TN\right)/\left(TP+TN+FP+FN\right).$$

Sensitivity: It evaluates the accuracy with which positive samples are identified.

(18) $$SEN=TP/\left(TP+FN\right).$$

Specificity: With regard to the total, it evaluates the capacity to identify negative samples.

(19) $$SPE=TP/\left(TP+FP\right).$$

F-measure: The variance is computed by subtracting the total number of false positives from the total number of genuine positives.

(20) $$F-Measure=2\ast {\displaystyle \frac{\left(precision\ast recall\right)}{\left(precision+recall\right)}}$$

Comparative analysis of performance metrics

Figure 3 and Table 3 shows comparative analysis of performance metrics of the DMTCNN methods with other approaches. The graph establishes the improved specificity, sensitivity, f-measure and accuracy of the DL method. The specificity for the DNN, DCNN, ADD-EMRFODL, and Mask R-CNN models are 70.98%, 74.12%, 86.34%, and 70.12%, respectively, compared to the DMTCNN model’s specificity of 94.18%. Similar, the recall for the DNN, DCNN, ADD-EMRFODL, and Mask R-CNN models are 81.78%, 70.77%, 67.23%, and 81.24%, respectively, compared to the DMTCNN model’s recall of 93.19%. Like, the f-measure for the DNN, DCNN, ADD-EMRFODL, and Mask R-CNN models are 89.16%, 61.14%, 79.98%, and 79.18%, respectively, compared to the DMTCNN model’s f-measure of 94.18%. Finally, the accuracy for the DNN, DCNN, ADD-EMRFODL, and Mask R-CNN models are 80.98%, 79.98%, 75.56%, and 71.23%, respectively, compared to the DMTCNN model’s accuracy of 96.97%.

Table 3:

Comparative analysis of performance metrics.

Methods	Specificity	Recall	F-measure	Accuracy
DNN	70.98	81.78	89.16	80.98
DCNN	74.12	70.77	61.14	79.98
ADD-EMRFODL	86.34	67.23	79.98	75.56
Mask R-CNN	70.12	81.24	79.18	71.23
DMCTNN	94.18	93.19	94.18	96.97

DOI: 10.7717/peerj-cs.2538/table-3

RMSE analysis

Figure 4 and Table 4 show an RMSE comparison of the DMTCNN strategy to other well-known methods. The graph demonstrates that the machine learning technique performs better with a lower RMSE. For example, the DMTCNN model has an RMSE value of 21.67% for 100 data, while the DNN, DCNN, ADD-EMRFODL, and Mask R-CNN models have RMSE values of 49.18%, 40.77%, 37.18%, and 28.19%, respectively. Moreover, the DMTCNN model has demonstrated superior performance for a wide range of data sizes with low RMSE values. Similarly, for 500 data, the RMSE score for the DMTCNN is 22.76%, whereas the DNN, DCNN, ADD-EMRFODL, and Mask R-CNN models are 46.11%, 41.87%, 38.16%, and 33.19%, respectively.

Table 4:

RMSE analysis for DMTCNN method.

Number of data from dataset	DNN	DCNN	ADD-EMRFODL	Mask R-CNN	DMTCNN
100	49.18	40.77	37.18	28.19	21.67
200	47.18	42.18	34.19	31.29	24.96
300	49.87	43.78	39.98	28.98	24.19
400	50.98	45.18	39.76	32.87	25.19
500	46.11	41.87	38.16	33.19	22.76

DOI: 10.7717/peerj-cs.2538/table-4

Execution time analysis

The database execution time of the proposed DMTCNN approach is compared to known methods in Table 5 and Fig. 5. The statistics display that the proposed DMTCNN method outperformed all other approaches collectively. The suggested DMTCNN technique has taken only 3.876 ms to execute, whereas other existing methods such as DNN, DCNN, ADD-EMRFODL, and Mask R-CNN took 15.176, 12.165, 10.432, and 5,188 ms, respectively, for 100 data their execution time. Likewise, the suggested DMTCNN technique executes in 4.875 ms for 500 data, but existing techniques such as DNN, DCNN, ADD-EMRFODL, and Mask R-CNN take 17.876, 14.176, 9.721, and 9.432 ms, correspondingly. In clinical contexts, where timely and accurate diagnoses are essential for patient care, this reduced execution time is particularly advantageous. In the context of swiftly progressing diseases such as brain disease diagnosis, faster inference not only enhances patient throughput but also enables more immediate interventions. DMTCNN is a practical and scalable solution for large-scale diagnostic applications due to its efficiency.

Table 5:

Execution time analysis for DMTCNN technique.

Number of data from dataset	DNN	DCNN	ADD-EMRFODL	Mask R-CNN	DMTCNN
100	15.176	12.165	10.432	5.188	3.876
200	14.977	13.198	10.876	7.789	3.287
300	15.118	13.665	11.231	6.165	2.567
400	16.113	12.245	10.987	8.843	3.987
500	17.876	14.176	9.721	9.432	4.875

DOI: 10.7717/peerj-cs.2538/table-5

Brain disease diagnosis analysis

Figure 6 shows the brain disease diagnosis analysis. In this work, we evaluated the suggested method on two different brain MRI datasets. Our main dataset, the Figshare brain tumor dataset, is one of the most comprehensive sets of data out there and is essential for the identification of brain tumors. 3,064 real brain MRI samples from 233 people are included in this collection. Of these, the meningioma class includes 708 samples, the pituitary class includes 930 examples, and the gliomas class includes 1,426 samples.

Discussion

The improved efficacy of DMTCNN underscores the possible advantages of combining classification and regression assignments into a single, cohesive framework for diagnosing brain disorders. The network can classify more subtle patterns in the data and produce more reliable and thorough diagnostic insights by concurrently learning from many tasks. By concentrating the model’s attention on relevant brain regions, the patch extraction technique suggestively progresses diagnostic accuracy. Following research endeavors ought to concentrate on the model’s ability to scale and adapt to numerous clinical environments. To further ensure that the model’s outputs are reliable and useful, investigating DMTCNN’s interpretability may proposal useful data for clinical decision-making. The proposed DMTCNN framework, as established by experimental results, exhibits exceptional performance with segmentation analysis accuracy at 96.97%, sensitivity at 93.19%, specificity at 94.18%, f-measure at 94.18%, RMSE at 22.76%, and an execution time of 4.875 ms.

Ablation study

In the recommended model, every module is important. In this section, we inspect the framework for the proposed DMTCNN and associate it to recognized models such as DNN, DCNN, ADD-EMRFODL, and Mask R-CNN. This comparative study is carried out through a series of ablation studies using the Brain MRI Dataset to investigate the performance development and establish the motivations behind our proposed DMTCNN.

Influence of the K-fold cross validation

In DMTCNN-based brain disease diagnosis, the implementation of 10-fold cross-validation substantially improves model robustness and dependability. Ten subsets of the dataset are created as part of the statistical technique known as cross-validation. There are ten subsets: one is for training, and the other nine are for validation. The proposed DMTCNN model outperformed, obtaining 96.97% accuracy on our input data using 10-fold cross-validation. In comparison, the current DT, RF, KNN, and NB models have accuracy performances of 70.14%, 78.39%, 87.48%, and 85.26%, respectively (Table 6). Figure 7 shows the comparison of proposed method with existing model. Table 7 shows association of the proposed method to current approaches and Table 8 shows association of the suggested model with innovative accuracy analysis approaches.

Table 6:

10-fold cross validation of DMTCNN analysis.

K-folds	DMTCNN accuracy
1-Fold	0.97
2-Fold	0.96
3-Fold	0.95
4-Fold	0.96
Fold-5	0.97
Fold-6	0.98
Fold-7	0.98
Fold-8	0.97
Fold-9	0.97
Fold-10	0.97
10-Fold mean	0.96

DOI: 10.7717/peerj-cs.2538/table-6

Table 7:

Comparison of the proposed model to current approaches (Wang et al., 2024).

Methods	Evaluation methods	Accuracy (%)	F-measure(%)
CNN (Ahmed et al., 2020)	10-fold cross validation	70.14	89.48
SVM (Collij et al., 2016)	10-fold cross validation	78.39	90.34
DT (Vidushi & Shrivastava, 2019)	10-fold cross validation	87.48	91.29
RF (Vidushi & Shrivastava, 2019)	10-fold cross validation	85.26	92.34
Proposed DMTCNN	10-fold cross validation	96.97	94.18

DOI: 10.7717/peerj-cs.2538/table-7

Table 8:

Comparison of the suggested model with innovative accuracy anlysis approaches.

Authors	Classifier	Database	Accuracy (%)	F-measure (%)
Raghavendra et al. (2021)	PNN (Probabilistic Neural Network)	Brain CT	94.37%	–
Ozaltin et al. (2022)	VGG-19	Brain CT	97.06%	87.43%
Marbun & Andayani (2018)	CNN	Brain CT	90%	–
Shalikar, Ashouri & Shahraki (2014)	SVM	Brain CT	90%	–
Chin et al. (2017)	CNN	Brain CT	90%	–
Proposed model	DMTCNN	Brain CT	96.97%	94.18

DOI: 10.7717/peerj-cs.2538/table-8