Optimized MobileNetV3: a deep learning-based Parkinson’s disease classification using fused images

Novel contribution

The significant key contributions of this article are as follows,

• In the pre-processing stage, we first utilized the image contrast enhancement algorithm to enhance the difference of a given input image. Next, the mean filter was used to eliminate the noise from the input image. Then, we blended the contrast-enhanced image and filtered the image into a single image as a fused image employing the multi-scale morphological gradient method.

• For extracting the features of the preprocessed image, we presented a new Pyramid channel-based feature attention network (PCFAN) that employed a multi-stage design with attention blocks at every step.

• To classify the levels of PD, such as mild, moderate, and severe, we employed the MobileNetV3 technique with the Dwarf Mongoose Optimization algorithm to improve the classification accuracy.

• Several ablation experiments have been performed on the PPMI and NTUA datasets. The experimental results showed that the suggested network outperformed the state efficiency compared to all other methods.

Limitations

However, this study may have some limitations:

• Due to the computational difficulty of two-dimensional CNN models, training is time-consuming.

• Substantial amounts of memory in the computer are necessary as the method may crash If the memory requirement for training a method outweighs the number of shots.

• The proposed model’s generalizability could be constrained by the limited sample size of the PD dataset employed in this investigation.

To overcome these problems, a Pyramid channel-based feature attention network (PCFAN) is used to extract the relevant features to reduce computational complexity. In our proposed method, we used a large number of images for training purposes for better prediction with sufficient memory. While comparing with our proposed approach, the existing approaches yield less prediction accuracy and high detection rates. Also, it takes high computational time. A 99.34% accuracy rate is achieved by this system, offering results that are equivalent to those of the recently suggested approach. As a result, it can be concluded that the input data is significantly reduced when models are computed while maintaining the relevant data, producing excellent classification accuracy with a low computational effort.

Image preprocessing

This subsection provides a detailed explanation of the proposed preprocessing algorithm. Some contrast abnormalities have been removed using preprocessing methods to improve classification results. We have created two copies of the input images to accomplish this. Contrast adjustment was done to enhance the visualization of the needed region in the initial copy. The image’s contrast was altered using the highest and minimum values of the image pixels.

Image contrast enhancement algorithm

A novel approach has been suggested by Ying et al. (2017) to give accurate contrast enhancement, and it was utilized to build the enhancement dataset. The weight matrix for picture fusion was first built using lighting prediction methods. The algorithm then functioned as follows. The camera response model was made available, allowing for the combining of numerous exposure photos. The optimal exposure ratio was then discovered for a decent exposure of the synthetic image in areas where the source image was underexposed. Further, a weight matrix was used to combine the input image and the created image to create a superior image. Equations (1) and (2) provide the essential formulas that the algorithm used. The images were integrated as in Eq. (4) to create an image with all pixels.

(1) $$R^c=\sum _{i=1}^N{W}_i{P}_i^c$$while N indicates the quality of images, $P_i$ the i-th image of the exposure set $W_i$ represents the weight map of the image, c for the three-color channel index, and R for the enhancement’s outcome. $P_i$ is calculated from Eq. (2).

(2) $$P_i=g(P,k_i)$$g is referred to as the brightness transform function (BTF), and k_i is the exposure ratio. In our investigation, the BTF was the beta-gamma connectivity method from Eq. (3).

(3) $$g(P,K)=\beta P^{\gamma }=e^{b(1-k^a)}{P}^{(k^a)}$$

The variables a, b, and k of the camera could be used to compute the parameters β and γ. As in the initial investigation, we used a constant parameter (a = 0.3293; b = 1.158). In the conclusion of the method, Eq. (4) was used to produce the enhanced image.

(4) $$R^c=W{P}^c+(1-W)g(P^c,k)$$

Mean filtering

Using numerous picture flattening patterns for graphic convolution processing is typical for picture de-noising in the spatial domain. The foundation of mean filtering is replacing a pixel’s single grey value with the total of all the surrounding pixels’ grey values. According to the following process, an image is created for a pixel point (a, b) in a source image with f (a, b), while its surrounding area S contains M pixels:

(5) $$G(x,y)={\displaystyle \frac{1}{M}\sum _{(i,j\in S)}f(a,b)(a,b)\notin S}$$

Multi-scale morphological gradient

The efficient operator, the multi-scale morphological gradient (MSMG), extracts gradient information from a picture to show the contrast level in a pixel’s nearby areas. As a result of this, the MSMG technique is quite effective and used for edge detection and picture segmentation. MSMG has been employed in multi-focus image fusion as a focus measure. The following details are used for MSMG. A multi-scale structural element is described as

(6) $$S{E}_j=S{E}_1\oplus S{E}_2\oplus ....\oplus S{E}_N,j\in \{1,2,.....,N\}$$

SE1 is a notation for a fundamental structural component. By using the morphological gradient operators from picture f, the gradient feature G_t can be expressed.

(7) $$G_t(x,y)=f(x,y)\oplus S{E}_t-f(x,y).S{E}_t$$while the morphological operators for dilation and erosion are denoted by $\oplus$ and (.) accordingly. Here, t refers to the number of scales. By estimating the weighted total of gradients across all scales, one may derive the multi-scale structuring element from the gradient feature. The same preprocessed images are shown in Fig. 2.

(8) $$M(x,y)=\sum _{t=1}^N{w}_t.G_t(x,y)$$

Here, the weight of the gradient in the t-th scale is denoted by

(9) $$w_t={\displaystyle \frac{1}{2t+1}}$$

Pyramid channel-based feature attention network (PCFAN)

FrameWork

This study presents a PCFAN model for feature extraction by fusing channel attention and pyramid operation advantages. Three modules comprise this PCFAN: an image reconstruction module, a module for PCFA, and a module for extracting features from three scales. Three phases comprise the three-scale feature module: Two Resblocks plus a 3 × 3 convolution layer comprise the initial feature extraction stage. At this stage, feature maps include 32 more channels (or depth).

They increase the depth of the feature maps to 64 and 128 and reduce the resolution of the feature maps by half, respectively. Unlike earlier studies, which mainly used the third stage’s output features, the PCFA, made of several channel-attention blocks, provides data from all three stages’ output features. It is possible to extract spatial and channel-dimension information using the channel-attention block. Then, a network for image reconstruction with just one CNN layer is used to recover the original clean image.

Channel attention block: To make sure that the network collects additional features, the channel attention strategy is employed in this study to look at the dependencies between feature channels.

The framework of PCFA is shown in Fig. 3. Concatenation operation is performed by fc at position $(i,j)$, $[{\mathrm{v}}_1,v_2,....v_c]$ and as a result, $\mu$ is the concatenation of $v_{\mathrm{k}}(k=1\dots C)$. These processes can record how the aggregated features depend on one another through channels. This is represented as:

(10) $$\overrightarrow{f}=\sigma ({\phi }_2(\eta ({\phi }_1(\mu ))))$$

The convolution layer, sigmoid activation function, and ReLU are denoted by $\varphi$, $\eta$, and $\sigma$. The goal ${\varphi }_1$ is to decrease the input feature channels. The features are first activated by ReLU $\eta$, then with a convolution layer 2, they are expanded to their original width. The final output feature $F_{out}$ is attained by

(11) $$F_{out}=\overrightarrow{f}\otimes f$$while $\otimes$ is an element-wise product, and the original feature is represented as f.

PCFAN: This model can retrieve features from various CNN layers and merge those characteristics simultaneously to produce more valuable features. These techniques, however, typically employ an intuitive fusion process, such as addition or concatenation. In order to integrate the advantages of the feature pyramid and the channel attention mechanism, we presented the PCFA approach.

PCFA comprises two layers of upsampling, two levels of concatenation, and four channel-attention blocks. In PCFA, top-down and bottom-up pathways are both available. The suitable channel attention blocks for the bottom-up path-way are filled with features from three tiers. PCFA reconstructs layers with higher spatial resolution from the down-top pathway using their semantically richer layers. Combining information from the top-down and bottom-up routes makes it possible to express traits effectively and comprehend their relative importance at different levels. The complementary information among low-level and high-level attributes can, therefore, be fully utilized by PCFA for extraction.

Classification

In order to classify Parkinson’s disease after successfully identifying brain images in scans, we suggested utilizing MobileNetv3 deep learning. The CNN family, known as MobileNet, was created by a team of Google, Inc. researchers for the purpose of classifying images. MobileNet included several innovative ideas through its numerous iterations to decrease the number of parameters while maintaining excellent classification accuracy. Compared to several other CNN architectures of equivalent size, MobileNet performs well in terms of accuracy because of the MADDS (multiply-add operations). MobileNetV3 (Zivkovic et al., 2022), in particular, has the top-1 accuracy among the other models. Investigating the proposed approach for this categorization challenge was primarily motivated by this.

MobileNet is made of bneck blocks, a collection of construction blocks. Figure 4A shows the overall MobileNet architecture, whereas Fig. 4B shows the intricacies of a bneck block. MobileNetV1 used depth-wise convolutional operations in place of typical convolutional methods. In Fig. 4B, a residual link between the input and results tensors is shown. Then, as depicted in Fig. 4, the creators of MobileNetV3 added both the compression and expansion stages to the start and finish of every bneck block. This setup is called an “Inverted Residual Block” (IRB), as the residual connections only make limited output and input tensor connections.

The IRB idea contributed to further lowering the model’s computational expenses. In order to further reduce computations (such as ReLU), the authors used linear activations rather than non-linear activation functions after filtering the input and output tensors. The SE module was included by the authors to finalize the MobileNetV3 concept.

The SE module also features a function of the h-swish added by the authors. The description of the Swish activation function is as follows:

(13) $$h-swish(x)=x{\displaystyle \frac{\mathrm{Re}LU6(x+3)}{6}}$$

The description states that a bneck block constructs a feature map optimized using SE components and connection residuals. Our decision to use the bneck block as the fundamental model of an architecture resembling the UNet was motivated by this.

Improved dwarf mongoose optimization algorithm (IDMO)

This phase provides details on the IDMO algorithm working process.

The IDMO mode

The Improved Dwarf Mongoose Optimization algorithm (IDMO) is suggested to improve DMO exploration and exploitation. The DMO is modified in three straightforward yet efficient ways by this optimization technique. Alphas are chosen by the IDMO rather than the DMO, which chooses them solely based on their computational load. The IDMO’s alpha selects the mongoose based on physical fitness, and a new operator is presented to control the alpha’s mobility. This increases the IDMO’s exploration and exploitability.

Secondly, randomization changes the scout group motions to diversify the search and investigate previously unexplored places. The suggested approach obtains optimization in three stages they are babysitters, forage area, and abundant food source. The search agents are the individual mongooses, described as an n-by-d matrix. The modified alpha guides the group to unexplored territory during the exploration phase by following the modeled processes.

Randomization is used to change the scout group motions in order to diversify the search process and explore previously unexplored locations. Once the requirement for babysitter swap is satisfied and babysitters are switched, the exploitation is accomplished. At this stage, the obtained solution is upgraded in order to yield superior results.

Population initialization

According to Eq. (14), a matrix of potential dwarf mongooses (X) is used to stochastically initialize the IDMO population. Among the optimization issue’s upper bound (U) and lower bound (L), the population vector lies.

(14) $$X=\left[\begin{array}{cccc}{x}_{1,1}& x_{1,2}& x_{1,d-1}& x_{1,d}\\ x_{2,1}& x_{2,2}& x_{2,d-1}& x_{2,d}\\ & :& x_{i,j}& :\\ x_{{}_{n,1}}& x_{n,2}& x_{n,d-1}& x_{n,d}\end{array}\right]$$while n represents the entire amount of dwarf mongooses in a mound, Every $X_{i,j}$ in Eq. (15) reflects the location of the jth population’s dimension.

(15) $$X_{i,j}=rand\times (U-L)+L$$

Alpha group

Babysitters are subtracted from the overall amount of dwarf mongooses to get the total population dimensions for this group. The dwarf mongoose that is the fittest is chosen to be the alpha, as shown by the alpha symbol in Eq. (16).

(16) $$\alpha =min(fi{t}_1,fi{t}_2......,fi{t}_n)$$

Peep’s vocalizations are used by the female alphas to keep them integrated. As specified in Eq. (16), the IDMO moves about the problem space while searching. It initially identifies as the strongest dwarf mongoose in the family and leads the other members of the pack towards a prospective food source. In contrast to the DMO, which relies solely on the vocalization of the alpha to sway the behavior of the other dwarf mongoose, this situation deviates from it. The IDMO’s exploration and exploitability are improved by operator a. The IDMO uses the location of the alpha to establish the location of the other mongoose.

(17) $$X_{a+1}=\alpha +phi\ast rand\ast (Xa-X_u)$$

(18) $$\omega =e^{-4\ast {(C_{iter}/Ma{x}_{iter})}^2}$$

Scout group

Scouts are responsible for selecting an appropriate sleeping mound as dwarf mongooses are semi-nomadic and are reported to never return to their sleeping mounds. Due to the dwarf mongooses’ propensity for congregating around plentiful food sources, the scouts’ fitness level is considered when choosing a prospective sleeping mound. Consequently, the fittest scout is chosen. As stated in Eq. (19), the scouts are simulated.

(19) $$X_{a+1}=\alpha +phi\ast rand\ast (X_u-X_{v)/2}$$

The babysitters

Equation (20) provides the exchange standard for the babysitter. Once the requirement is met, the counter is reset to 0, and the swapped babysitters communicate with the dwarf mongooses. By doing this, they could create mongooses better suited to their environment rather than starting from scratch, as in DMO. By multiplying L by the current iteration and CF, L is reset if it reaches zero.

(20) $$X_{a+1}=(Xb+rand\ast (\alpha -(Xu+Xv)/2\ast br$$while $CF={\left(1-{\displaystyle \frac{C_{iter}}{Ma{x}_{iter}}}\right)}^{\left(2{\displaystyle \frac{C_{iter}}{Ma{x}_{iter}}}\right)}$ directing the dwarf mongooses’ overall volitional movement, X_b, X_v, and X_u are chosen at random to take the position of the babysitters, and br denotes the birthrate.

It makes the alpha selection overhead simpler, and the IDMO’s computing complexity is significantly decreased. Dwarf mongooses forage under the leadership of the alpha female, which initiates the optimization process. The nest is tended by a small group known as the babysitters. Finding sufficient food sources resembles the IDMO’s exploratory phase. The exploitation phase of the IDMO is represented by this hypothetical scenario. The search region is further investigated and exploited by looking for a sleeping mound at night.

Experimental setting

Windows 10 was used on an Intel i5 2.60 GHz processor with 16 GB of RAM. The investigations were conducted using Python, KERAS, and TensorFlow against the backdrop of the Anaconda3 environment. In this research, the PPMI and NTUA datasets were used for validation to calculate the performances of our suggested strategy. The data samples were divided into two groups, one served as the training dataset and was used to build a classifier. In the second step, the classifier was assessed using the testing dataset.

Dataset description

The Parkinson’s Progression Makers Initiative (PPMI) collection contained photographs of patients and controls accessed for the research. An international network of clinical sites was where the PPMI was conducted. One goal of the PPMI was to gather medical, biological, consumer, and imaging data in order to hasten the establishment of biomarkers of PD progression. The ultimate objective of these biomarkers was to be employed in therapeutic investigations. For this investigation, T1-weighted MR images from PPMI were chosen.

With the use of a 1.5–3 Tesla scanner, these images were produced. It takes about 20 to 30 min to complete the scan. Axial, sagittal, and coronal views were used to acquire the three-dimensional (3D) sequence of T1-weighted MR images with a slice thickness of 1.5 mm or less. Approximately 6,500 images in the dataset. From the dataset, we utilized 4,550 (70%) images for training purposes, and the remaining 1,950 (30%) images were utilized for testing purposes.

NTUA

The NTUA Parkinson dataset contains MRI, DaT scans, and clinical information from PD-affected patients. There were about 42,000 photos that could be used for research purposes. The frames per sequence and resolution of the T1, T2, and Flair MRI image samples in this dataset varied for each image. From the dataset, we utilized 29,400 (70%) images for training purposes, and the remaining 12,600 (30%) images were utilized for testing purposes.

Evaluation criteria

To evaluate the prediction capabilities of the classifiers, evaluation measures were required. The following metrics were employed to evaluate the proposed method.

Accuracy: Accuracy is the percentage of accurate forecasts among all made predictions.

(21) $$Accuracy={\displaystyle \frac{TP+TN}{TP+FP+TN+FN}}$$

Sensitivity: Sensitivity, sometimes called true positive rate (TPR) or Recall, evaluates a system’s tendency to anticipate the future positively.

(22) $$TPR={\displaystyle \frac{TP}{TP+FP}}$$

Specificity: Specificity, commonly called true negative rate (TNR), evaluates a system’s capacity to forecast negative outcomes correctly.

(23) $$TNR={\displaystyle \frac{TN}{TN+FP}}$$

Precision: Precision, called positive prediction value (PPV), assesses a system’s capacity to generate only relevant outcomes.

(24) $$Precision={\displaystyle \frac{TP}{TP+FP}}$$

F-Measure: The harmonic mean of precision and recall was computed by F-measure.

(25) $$F-Measure=2\ast {\displaystyle \frac{\mathrm{Re}call\ast Precision}{\mathrm{Re}call+precision}}$$

Evaluation of training and testing time

The batch size, learning rate, momentum, and weight decay were each 32, 0.03, 0.9, and 0.01. A 0.01 learning rate was used initially. The learning rate reached saturation in the ReLu layer. The quantity of epochs was also a vital training parameter, as there was a chance that the network might be either under or over-fitted. We ran the network through 200 epochs of training on these datasets. The suggested model’s training and testing accuracy spanned between 0.98 and 0.99. The hyperparameter configuration is shown in Table 1.

In Figs. 6 and 7, the graphs show the categorization accuracy and loss value of the IDS concerning the number of iterations. As observed in the image, the approach used in this study had a positive convergence effect. We divided the dataset into two halves for the model’s training and testing. A total of 200 training epochs of the processed training set were used to train the suggested strategy during this phase. It was configured to learn at a rate of 0.01.

It employed L1 or L2 regularization techniques to penalize large weights in the network. This helps prevent the model from fitting the noise in the training data. Dropout is another regularization technique that randomly drops a fraction of neurons during training to reduce overfitting. Consider reducing the depth and width of your MobileNet model. It can use smaller variants like MobileNetV3 or even explore custom architectures tailored to the specific task.

Differentiation of existing machine learning approaches vs proposed

In this section, we compared our proposed deep learning approach with existing machine learning approaches. The existing machine learning approaches like SVM, RVM, and Naïve Bayes were analyzed.

We found that the suggested strategy outperformed existing approaches, demonstrating 99.06% accuracy, followed by SVM, GA-ELM, RVM, Decision tree, Naive Bayes, ANN, and CNN with 92.35%, 89.22%, 90%, 92%, 93%, 94%, and 96%, respectively. The approach proposed in this study provides a better outcome when compared to other existing solutions. CNN was the second strategy that produced a greater performance. GA-ELM performed less effectively than the other strategies. A comparison of existing approaches is shown in Fig. 8 and Table 2.

Comparison of transfer learning approaches

The existing transfer learning approaches like DenseNet121, VGG16, ResNet, MobileNet, and Inception V3 were differentiated with the proposed method. The table compares existing transfer learning approaches with the proposed ones.

F-score, sensitivity, accuracy, and specificity were also used to compare the results. In Table 3, the comparison findings for the transfer learning approach are presented. Compared with the other approaches, the proposed method yielded a superior performance. Figure 9 shows the comparison of existing approaches with the proposed method.

The receiver operating characteristic (ROC) curve shows the performance of a model used for categorization across all levels. Two variables, TPR and FPR, were plotted on this curve. The ROC curve is shown in Fig. 10.

Table 4 shows the comparison of features. The essential features were extracted based on the PCFA approach. Our proposed approach extracted all the features. Compared with the previous methods, PCFA performed well.

Comparison of existing machine learning approach with proposed deep learning approach is shown in Table 5. The existing machine learning approaches like neural network, naïve Bayes, random forest, and SVM were employed to compare. While comparing with all four methods, the proposed approach exhibited greater performances over accuracy, sensitivity, and specificity.

The results of optimized MobileNet V3 and MobileNet V3 during the testing process are shown in Table 6 and Figs. 11 and 12. Before employing the optimization approach, the MobileNet V3 yielded 98.09% accuracy in the PPMI dataset and 96.53% in the NTUA dataset; the proposed approach performed slightly lower. While we used the IDMO algorithm to optimize the proposed approach, the performance improved considerably.

Training, testing accuracy, and loss comparison are shown in Table 7. The existing approaches like DenseNet, VGG 19, ResNet, and inception V3 obtained low performances compared with the proposed model. The approach proposed in this study, optimized by IDMO, showed high accuracy and a lesser loss.

A comparison of existing related works is represented in Table 8. The existing authors used various datasets, including PD, PPMI, PD audio dataset, etc., but our proposed approach employed PPMI and NTUA datasets. While compared with existing approaches, the proposed approach yields a greater predictive performance. The second most performance was presented in reference (Loh et al., 2021).