Moth-flame optimized UNet++ with self-attention for early-stage Alzheimer’s disease prediction using multimodal input

Research gap

The prediction of AD is one of the complex and critical challenges in medical research, mainly due to the diversity of the characteristics of the disease and the numerous diagnostic data modalities. Modern methods are usually based on a single-modal source of data, such as MRI or PET imaging, which provides extensive but still inadequate information about the evolution of the disease (Chen et al., 2025). Alzheimer’s pathology is better described by integrating multimodal features, including structural changes observed on MRI, metabolic activity measured by PET scans, and cognitive impairments assessed by neuropsychological tests. Although multimodal integration is evident as essential, current approaches often fail to properly integrate multiple complementary data streams, resulting in low prediction accuracy. Furthermore, typical UNet++ architectures, often used for segmentation, rely heavily on local spatial features and are inherently limited in their ability to represent the global context required to decode complex patterns in multimodal medical images. While self-attention mechanisms have recently emerged as a potential alternative to overcome these limitations, integrating them into the UNet++ architecture, which is specifically designed for Alzheimer’s disease prediction using multimodal inputs, remains relatively underexplored. In addition, there is a scarcity of efficient ensemble learning algorithms that exploit the strengths of predictions from individual modalities, thereby significantly increasing reliability and accuracy. This gap has highlighted an urgent need for a new approach that leverages expert UNet++ configurations with self-attention mechanisms to fully exploit the benefits of multimodal data and achieve better predictive performance.

$UNet+{+}_{SA}$ integrated with MFO improves Alzheimer classification, prevents overfitting, and enhances generalization. The proposed $UNet+{+}_{SA}$–MFO framework addresses challenges like capturing complex patterns like texture and spatial relationships over earlier methods that rely on pretrained models that are exposed to overfitting and a lack of adaptability in clinical practices.

Contributions

Significant contributions of this work include,

• Proposed $UNet+{+}_{SA}$–MFO framework enhances AD identification through the integration of multimodal inputs, including MRI, PET imaging, and neuropsychological assessments.

• $UNet+{+}_{SA}$captures complex patterns, such as texture and spatial relationships, from input MRI and PET images.

• MFO finetunes $UNet+{+}_{SA}$ parameters and ensures enhanced generalization and reduced computational loads.

• Optimized $UNet+{+}_{SA}$ provides a notable contribution to the field of Alzheimer’s Detection through multimodal inputs.

Organization of the article

‘Introduction’ provides an overview of significance of Alzheimer’s disease prediction and role of deep learning in the field of neurodegenerative disorders, research contributions and scope of the study, ‘Related Works’ defines a comprehensive review of recent works related to machine learning and deep learning for neurodegenerative disorders, ‘Materials and Methods’ details dataset description, also elaborates the proposed framework, ‘Results and Discussion’ demonstrates the performance of proposed system against existing multimodal methodologies and detailed discussion of performance of all methods and ‘Conclusion’ summarizes the key findings of the research and future interventions of the proposed system on various neurodisorders.

Problem statement

Alzheimer’s disease is a progressive neurodegenerative disorder that is usually identified by cognitive decline, memory impairment, and functional disabilities. Early and accurate prediction of AD is vital for timely intervention and treatment. The heterogeneous nature of Alzheimer’s pathology necessitates leveraging multimodal data sources, including MRI, PET, and neuropsychological test scores, which collectively capture structural, metabolic, and cognitive biomarkers. Despite the complementary nature of these modalities, current approaches often fail to integrate multimodal features effectively, resulting in suboptimal performance in AD prediction. Mathematically, the problem involves learning a predictive function $f$ and is presented in Eq. (1).

(1) $$f:(X_{MRI},X_{PET},X_{NP})\to y$$where $X_{MRI}\in D^{H_{MRI}X{W}_{MRI}X{C}_{MRI}}$ depicts an MRI image, $X_{PET}\in D^{H_{PET}X{W}_{PET}X{C}_{PET}}$ depicts a PET image and $T_{NP}\in D^n$ depicts neurological test scores for n samples, and $y\in$ $\left\{0,1,2\right\}$ presents 0—Alzheimer’s Disease (AD), 1—Mild Cognitive Impairment (MCI), or 2—Cognitive Normal (CN).

Dataset

This study used the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, a publicly available, curated resource for neuroimaging and clinical data. The dataset is accessible at: https://adni.loni.usc.edu/data-samples/adni-data/. Table 1 summarizes the key characteristics of the ADNI dataset, including the distribution of samples across different disease categories (Alzheimer’s Disease (AD), Mild Cognitive Impairment (MCI), and Cognitively Normal (CN)), as well as the data modalities available (MRI, PET, and neuropsychological tests).

The ADNI dataset consists of 500 samples, categorized into three clinical groups: 109 individuals with AD, 204 with MCI, and 187 with CN status. The dataset comprises data from three modalities—MRI, PET scans, and neuropsychological tests—with each sample containing values across all modalities. The age distributions vary slightly across groups, with average ages of 73.1 ± 7.6 years for the AD group, 70.3 ± 7.4 years for the MCI group, and 72.0 ± 6.8 years for the CN group. Gender distribution is relatively balanced in the MCI and CN groups, while the AD group shows a slightly higher proportion of males. The cognitive scores further differentiate the groups, with the Mini-Mental State Examination (MMSE) revealing lower cognitive performance in the AD group (18.2 ± 4.7), moderate performance in the MCI group (24.3 ± 3.6), and near-normal scores for the CN group (29.0 ± 0.8). The Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAS-Cog) scores corroborate this trend, with the AD group having the highest cognitive impairment (21.5 ± 5.9), followed by the MCI group (17.4 ± 5.2), and CN individuals (6.5 ± 2.3). The PET and neuropsychological test scores, though not detailed in this table, further contribute to the differentiation between these clinical groups. This dataset is valuable for understanding the progression of Alzheimer’s disease and for developing predictive models that integrate multimodal data for more accurate diagnosis and prognosis. Finding records with all three modalities (MRI, PET, and neuropsychological assessments) aligned for the same patient is challenging due to data privacy, acquisition costs, and protocol differences. ADNI remains the most comprehensive public dataset offering this multimodal alignment; other sources, such as OASIS, often lack PET scans or synchronized assessments, limiting direct comparability.

Computing infrastructure

All experiments were conducted on a workstation running Ubuntu 22.04 LTS (64-bit). The hardware configuration included an Intel® Core™ i7-12700K CPU @ 3.60 GHz, 32 GB RAM, and a single NVIDIA RTX 3080 GPU with 10 GB VRAM. The deep learning models were implemented in Python 3.10 using TensorFlow 2.10 and PyTorch 2.1. Additional libraries, including scikit-learn 1.3, NumPy 1.26, and OpenCV 4.8, were utilized for data processing and evaluation.

Evaluation method

The evaluation protocol included:

• Baseline Comparisons: The UNet++(SA)–MFO Framework was compared with standard single-modal models, simple and weighted voting ensembles, and existing multimodal frameworks.

• Ablation Studies: Ablation experiments were conducted to assess the effect of each modality, feature fusion, self-attention, Moth Flame Optimization, and ensemble learning.

• Comparison with and without Ensembles: The performance of the framework with and without ensemble methods was explicitly evaluated to quantify improvements in robustness and accuracy.

• To mitigate the risk of overfitting due to the relatively small sample size (n = 500) and the complexity of the UNet++_(SA)–MFO framework, several strategies were employed. First, a 5-fold cross-validation was implemented, ensuring that the model was trained and validated on different subsets of the ADNI dataset, with each fold maintaining a balanced distribution of AD, MCI, and CN classes. This approach helped assess the model’s generalization across unseen data splits. Second, regularization techniques such as dropout (rate = 0.3) were integrated into the UNet++ architecture at dense layers and attention gates to prevent over-reliance on specific features. Additionally, L2 weight regularization (lambda = 0.01) was applied to the convolutional layers to constrain model complexity. Third, data augmentation techniques, including random rotation (±10 degrees), flipping, and intensity scaling (±10%), were applied to MRI and PET images during training to artificially increase dataset variability and enhance robustness. These combined methods ensured that the model did not overfit to the training data, as evidenced by stable validation performance metrics across folds (e.g., validation accuracy within ±2% of training accuracy).

Assessment metrics

The following metrics were used for quantitative evaluation: Accuracy, Precision, Recall, F1-score, and Area Under the Receiver Operating Characteristic Curve (AUC-ROC).

• Accuracy measures the overall proportion of correct predictions.

• Precision is the proportion of true positives among all positive predictions, which is important for minimizing false positives.

• Recall (sensitivity) measures the proportion of true positives correctly identified, which is crucial for early disease detection.

• F1-score is the harmonic mean of precision and recall, providing a balanced measure, especially for imbalanced datasets.

• AUC-ROC evaluates the model’s ability to distinguish between classes across all thresholds.

These metrics collectively provide a comprehensive assessment of the model’s correctness and reliability, which is essential for potential clinical application.

UNet++(SA)–MFO framework

This work aimed to design an efficient, comprehensive system for predicting Alzheimer’s disease by integrating multimodal inputs, including MRI, PET, and neuropsychological test data. Rationale for base learners: We selected SVM (RBF), kNN, and Random Forest as base learners over the fused features because (i) they perform robustly on small-to-moderate datasets (n = 500) with high-dimensional, nonlinearly separable representations; (ii) they are relatively stable under class imbalance after calibration (e.g., Platt scaling for SVM) and provide complementary decision biases (margin-based, local-neighborhood, and bagged tree ensembles); and (iii) they offer fast, well-understood training with interpretable importance estimates (RF). While modern alternatives such as gradient boosting (XGBoost/LightGBM) and deep ensembles are powerful for tabular data, pilot tests and prior literature indicate that they require extensive hyperparameter tuning and larger validation budgets to avoid overfitting. Our weighted stacking emphasizes complementary error patterns and probability calibration, yielding consistent gains without increasing training complexity. Future work will benchmark gradient boosting and deep ensemble variants within the same fusion pipeline under an identical data split.

The proposed $UNet+{+}_{SA}$–MFO framework includes a UNet++ architecture augmented with self-attention mechanisms, optimized using the Moth Flame Optimization (MFO) algorithm, and integrated with an ensemble learning to enhance the prediction accuracy and robustness. This dataset consists of tabular MRI and PET images, and neuropsychological test results. The proposed framework introduces a UNet++ neural network integrated with attention gates at skip connections.

Attention gate

Attention mechanisms have been effectively integrated into deep learning models to improve performance across various applications, including image denoising and extracting features from complex networks. These mechanisms enable models to concentrate on the most relevant parts of data, thereby enhancing their ability to extract and preserve critical features. Specifically, self-attention mechanisms excel at capturing relationships between any two positions within the data, enabling the modeling of long-range dependencies without significantly increasing computational or storage costs (Sulaiman et al., 2024). To address the focus on locations associated with impact features, the proposed framework incorporates an Attention Gate into the UNet++ model. Figure 1 depicts the configuration of the attention gate. Attention Gate takes two inputs: (i) upsampling function from the decoder path ( $X_{i+1,0}$) which acts as a gating signal and corresponding feature map from the encoder path ( $X_{i,0}$). The gating signal guides the network to signify target regions relevant to the segmentation task by suppressing irrelevant areas. This leads to improved learning of crucial regions associated with the target region. Attention Gate improves semantic information generation via skip connections; to achieve this, a sigmoid activation function is employed to optimize the convergence of parameters of the gate and later calculates the attention coefficient α. Coefficients are later used to amplify encoder feature maps via pixel-wise multiplication, yielding the final refined output.

Let $W_e$ and $W_u$ are learnable weights, and the encoder and decoder features are

(2) $$\mathrm{E}=W_e\ast X_{i,0}$$

(3) $$\mathrm{D}=W_u\ast X_{i+1,0}.$$

Equations (2) and (3) ensure that both encoder and decoder features align in the same dimensional space; * denotes the convolution operation. Attention score α is calculated as the combination of learnable parameters like $W_{\alpha }$ and $b_{\alpha }$ encoder and decoder features like E and D, and an activation function (which scales the attention score to [0, 1]). Equation (4) defines the attention score α

(4) $$\alpha =\sigma (W_{\alpha }\ast (\mathrm{Q}+\mathrm{K})+b_{\alpha }.$$

The computed attention score α is used to scale the encoder feature map ${{\hat{X}}_{i,0}}^{\prime }$, Eq. (5) defines the scaled encoder feature map

(5) $${{\hat{X}}_{i,0}}^{\prime }=\alpha \cdot X_{i,0}$$where denotes element-wise multiplication. ${{\hat{X}}_{i,0}}^{\prime }$ is the redefined feature map which passes relevant information only to the decoder.

UNet++ augmented with self attention

UNet++ is a segmentation method based on the UNet algorithm. The location and boundaries of the objects being learned cannot be defined in a UNet, as it requires uniform information across the region to be segmented. UNet++ has been designed to address the limitations of UNet. The encoder and decoder features are connected, and UNet++ integrates features from the intermediate node via nested skip connections. This framework uses UNet++ augmented with a self-attention mechanism, which combines the strengths of UNet++ and Attention Gates. This UNet++(SA)–MFO Framework integrates an encoder-decoder architecture with an integrated Attention Gate at the skip connection. Here, data from different nodes in the integration are on the same scale. The feature maps at various levels of UNet++ contain detailed information about the same image. Node $X_{1,4}$ decoder path captures spatial information of the object, such as edges and contours. As $X_{1,4}$ A node deeper in the mesh has stronger semantic information about the image. UNet++ utilizes dense skip connections to propagate information from the encoder to the decoder. However, not all features passed through these connections are equally significant; some may be redundant or irrelevant at multiple times. This may degrade medical imaging tasks, such as Alzheimer’s disease prediction. To address the above issue, Attention Gates are incorporated into skip connections, enabling the decoder to selectively focus on the most relevant encoder features by learning attention weights. This ensures (i) reduction of irrelevant information in feature maps, (ii) enhanced localization of significant features, and (iii) enhanced overall performance in multimodal tasks. Figure 2 explains the proposed framework. The various stages of the proposed framework are as follows:

• Stage 1 (Preprocessing): Skull stripping, intensity correction and normalization, registration and alignment, smoothing and denoising, segmentation, and ROI extraction are done on MRI and PET images. Neuropsychological test data from the ADNI dataset, comprising 10 features per sample (e.g., MMSE, ADAS-Cog scores), were preprocessed to ensure compatibility with the UNet++(SA) framework for multimodal integration. First, numerical features were normalized using z-score standardization (mean = 0, standard deviation = 1) to align their scales with those of image-derived features, preventing dominance by larger test scores during feature fusion. This was computed as z = (x − μ)/σ, where x is the raw score, μ is the mean, and σ is the standard deviation across the training set for each feature. Second, missing values, which occurred in less than 5% of samples, were handled using mean imputation based on the training set’s feature-wise averages to maintain dataset integrity. Third, the normalized test data were encoded into a dense vector representation via a fully connected layer (DenseNet) with 64 units and ReLU activation, resulting in a feature vector F(Neuro) of size 128 × 128 × 10 after reshaping to match the spatial dimensions of image features for concatenation (Stage 4: Feature Fusion). This preprocessing ensured seamless integration with MRI and PET features, preserving clinical relevance in the fused feature set F′.

• Stage 2 (Feature Extraction using $\mathit{U}\mathit{N}\mathit{e}\mathit{t}+{+}_{\mathit{S}\mathit{A}}$): UNet++ is enhanced with a self-attention mechanism. The attention gates are placed in a dense skip connection that collects the current node’s path inputs and up-samples from the down layer to capture spatial context. For each modality, extracted features using $UNet+{+}_{SA}$ are (1) $F_{MRI}$ = $UNet+{+}_{SA}\left(X_{MRI}\right)$, (2) $F_{PET}$ = $UNet+{+}_{SA}\left(X_{PET}\right)$ and (3) $F_{Neuo}$ = DenseNet $\left(X_{Neuro}\right)$, $F_{MRI}$, $F_{PET}$ and $F_{Neuro}$ represents features.

• Stage 3 (Hyperparameter Optimization): Moth Flame optimization (MFO) is used to finetune $UNet+{+}_{SA}$.

MFO is a metaheuristic proposed by Mirjalili (2015) based on the swarming behavior of moths. This population-based optimization method imitates the behavior of moths drawn into a light source to find the best possible solution. It combines local search strategies and population-based algorithms to enhance local and global exploration of the solution space (Tapasvi, Gnanamanoharan & Kumar, 2024). MFO is used to identify the optimal hyperparameter set for UNet++ and ensures high accuracy, faster convergence, and better generalization. Hyperparameters, such as learning rate, number of filters (n), and dropout rate (p), are tuned using MFO. Self-attention in UNet++ enhances the model’s ability to capture both spatial relationships and global contexts. MFO can fine-tune the attention parameter mechanism, such as the number of attention heads and the attention scaling factor. The finetuned parameters help UNet++ to extract more semantic features from multimodal inputs.

• Step1: Let $M_i$ be candidate solutions to encode the hyperparameters of $UNet+{+}_{SA}$ {learning rate η, number of filters $n_f$, dropout rate p, batch size b, number of attention heads $n_{AH}$, attention scaling s} in the optimization process and $F_j$ represents the best solution found.

• Step 2: The transverse orientation behavior of moth candidates is defined as the navigation strategy of maintaining a fixed angle relative to the light source. Equation (6), defines the navigation of a moth using a logarithmic spiral. (6) $$M_i^{t+1}=P_j^t+e^{bD}\cdot \mathrm{c}\mathrm{o}\mathrm{s}\left(2\pi r\right)$$

• $where$ $M_i^{t+1}$ is the new position of the i^th moth, $P_j^t$ is jth flame, i.e., the best solution, b is a constant that controls the spiral shape, D is Euclidean distance between flame and moth, r is a random number in range [0, 1], which introduces randomness of exploration.

• Stage 4 (Feature Fusion): $F_{MRI}$, $F_{PET}$ and $F_{Neuro}$ are concatenated to create a fused feature $F^{\prime }$

  $F^{\prime }$ = concat ( $F_{MRI}$, $F_{PET}$ and $F_{Neuro}$])

• Stage 5 (Classification): Fused features are passed to classification algorithms SVM, kNN, and Random Forest.

• Stage 6 (Ensemble Method): Predictions from various models are combined using a weighted stacking ensemble. $\hat{y}={\sum }_{i=1}^N{w}_i.{\hat{y}}_i$, where $w_i$ is the weight associated with the i^th classifier.

Performance analysis

Table 2 compares the efficacy of models using multimodal inputs, including fused features from MRI, PET, and neuropsychological tests. The results are based on the ADNI dataset and include both baseline U-Net models with self-attention and those enhanced with ensemble learning. To demonstrate the robustness of the UNet++_(SA)–MFO framework, performance metrics were averaged across five independent runs, each with a different random seed for data splitting and model initialization, following a 5-fold cross-validation protocol. The reported accuracy of 91% represents the mean across these runs, with a standard deviation (SD) of ±0.8%, indicating consistent performance. Similarly, precision (89.5% ± 0.6%), recall (88.9% ± 0.7%), F1-score (89.2% ± 0.6%), and AUC-ROC (93.5% ± 0.7%) were computed as means with their respective SDs, reflecting minimal variability and high robustness across runs. These averaged results, along with low variances, confirm the framework’s stability against initialization and data split variations, enhancing confidence in its generalizability despite the small dataset size (n = 500). Detailed per-run metrics are summarized in the Supplemental Material for transparency.

Figure 3A presents the Simple Voting Ensemble as a baseline model for Alzheimer’s disease prediction, aggregating predictions from individual models trained on MRI, PET, and neuropsychological test data via majority voting. With an overall accuracy of 83.6%, this model demonstrates moderate performance. It achieves a recall of 89.3% for AD classification, indicating a high ability to correctly identify AD cases. However, the precision for AD is lower at 80.6%, suggesting a higher occurrence of false positives. The True Negative Rate (TNR) for CN individuals is 90.4%, indicating that the model effectively identifies healthy individuals, yet there is still room for improvement. Although it offers simplicity and ease of implementation, its limited ability to integrate complementary multimodal data and lack of advanced feature fusion or self-attention mechanisms restrict its overall performance, particularly in distinguishing between similar classes such as MCI and CN.

Figure 3B depicts Weighted Voting Ensemble that builds on the Simple Voting model by assigning different weights to each modality based on their individual contribution to the prediction task. This approach enhances the model’s ability to prioritize more informative modalities, such as MRI, over others when necessary. As a result, the Weighted Voting model achieves a slightly improved accuracy of 84.6% compared to the Simple Voting model. Precision for AD improves to 83.9%, and recall increases to 91.2%, reflecting the model’s better performance in both minimizing false positives and correctly identifying AD cases. The TNR also improves to 92.3% for CN classification. While the Weighted Voting Ensemble shows tangible improvements, its reliance on weighted contributions does not fully exploit the complex relationships between different data types. Furthermore, the lack of a self-attention mechanism means the model still struggles to focus on the most critical features in each modality, limiting its overall effectiveness in handling multimodal data.

Figure 3C illustrates the UNet++(SA)–MFO Framework, a full ensemble model that incorporates self-attention-enhanced U-Net (SAU-Net) architectures for each modality, followed by a fusion step to integrate features from MRI, PET, and neuropsychological tests. This model leverages the strengths of self-attention mechanisms, which allow it to focus on the most relevant features within each modality, and multimodal fusion enables it to combine complementary information from different sources. The Full Ensemble model achieves the highest accuracy of 86.8%, with precision for AD at 87.1% and recall at 93.1%. Additionally, it demonstrates the highest TNR of 95.2%, making it particularly robust in avoiding false positives for CN individuals. These improvements suggest that the Full Ensemble model is significantly more capable of accurately classifying all three categories (AD, MCI, and CN) and is well-suited for clinical applications. However, the model’s increased complexity, driven by self-attention and multimodal fusion, results in higher computational demands, potentially limiting its feasibility in resource-constrained environments. Figure 4 presents the performance of the UNet++(SA)–MFO Framework compared to existing ensembles across various performance metrics.

The Simple Voting Ensemble provides a reliable starting point for Alzheimer’s disease prediction, with a respectable accuracy of 83.6%. However, its inability to fully integrate multimodal data and its lower precision (80.6%) for AD classification indicate limitations in distinguishing between subtle cognitive impairments. The Weighted Voting Ensemble improves accuracy by assigning modality-specific weights, achieving 84.6% and enhancing both precision and recall. However, it still does not leverage the full potential of multimodal data, as it lacks sophisticated mechanisms such as self-attention, which limits its performance when classifying complex cases.

The Full Ensemble model (SAU-Net + Multimodal Fusion) outperforms both the Simple and Weighted Voting methods, achieving 86.8% accuracy and significantly improving precision (87.1%) and recall (93.1%), making it the most effective model for predicting Alzheimer’s disease. Its use of self-attention to focus on the most relevant features and the fusion of multimodal data give it a substantial advantage in capturing complex patterns. This results in superior performance, particularly in distinguishing between MCI and CN, while maintaining a high TNR (95.2%) for correctly identifying CN individuals. However, its complexity, due to the incorporation of self-attention and the fusion mechanism, results in higher computational costs and challenges to model interpretability.

Figure 5 presents the ROC performance of the various methods used in this experiment. The Proposed UNet++(SA)–MFO Framework achieves the highest AUC of 93.5%, demonstrating superior classification performance compared to all other models. It is followed by the Ensemble Model (Simple Voting), with an AUC of 92.6%, and the Multimodal CNN, with an AUC of 91.1%. Other notable models include the DenseNet (Neuro Tests) with an AUC of 90.2% and the ResNet-50 (PET) with an AUC of 89.4%. In contrast, the Baseline CNN (MRI Only) performs least effectively, with an AUC of 86.1%, underscoring the limitations of single-modality approaches compared to multimodal approaches. The curves clearly illustrate the advantage of ensemble methods and multimodal frameworks in enhancing predictive accuracy for Alzheimer’s disease. These results emphasize the efficacy of combining multiple data modalities and using advanced architectures, such as SAU-Net, to achieve robust, reliable diagnostic predictions.

The reported AUC-ROC of 93.5% for the UNet++_(SA)–MFO framework was computed using a one-vs.-rest (OvR) strategy for multi-class classification across AD, MCI, and CN categories. For each class, a binary ROC curve was generated by plotting the True Positive Rate (Recall) against the False Positive Rate (1-Specificity) at various probability thresholds, based on predictions from the weighted stacking ensemble output. The area under each ROC curve was calculated using the trapezoidal rule, and the final AUC-ROC was obtained by macro-averaging the individual AUCs across the three classes, ensuring equal weighting for each category regardless of class imbalance. This computation was performed on the test set (15% of the ADNI dataset, approximately 75 samples), following 5-fold cross-validation to ensure robust estimation. The AUC-ROC of 93.5% reflects the model’s ability to distinguish between classes across all thresholds, as visualized in Fig. 5, and was validated across multiple runs (mean AUC-ROC ± standard deviation: 93.5% ± 0.8%) to confirm consistency.

Statistical testing across five runs indicates that UNet++(SA)–MFO significantly outperforms the Simple Voting and Weighted Voting baselines on Accuracy, F1, and macro-AUC (paired tests, adjusted p-values < (insert values)), with large effect sizes (Cohen’s d ≈ [insert] for AUC).

Analysis on the impact of MFO and ensembles on the proposed framework

Table 3 demonstrates the significant improvements achieved by applying MFO to the proposed framework across all sample categories: AD, MCI, and CN. For AD samples, MFO enhances precision, recall, and F1-score by 3.1%, 2.3%, and 2.7%, respectively, reflecting its effectiveness in accurately identifying patients with Alzheimer’s disease.

Similarly, for MCI, notable improvements are observed, with a 2.6% increase in precision and a substantial 3.5% boost in accuracy, indicating better differentiation between MCI and other classes. The CN category also benefits from optimization, with a 3.2% increase in recall and a 2.9% increase in F1-score, demonstrating the optimized model’s ability to classify healthy individuals correctly. Overall, the application of MFO consistently improves precision, recall, F1-score, and accuracy across all categories, underscoring its pivotal role in enhancing the predictive reliability and robustness of the proposed multimodal framework. Figure 6 compares the proposed framework’s performance with and without ensemble methods.

Integrating ensemble methods into the proposed framework yields significant improvements in precision (+2.4%), recall (+2.4%), and F1-score (+2.5%), underscoring their role in enhancing the model’s robustness and generalization. Ensemble methods leverage the strengths of multiple predictions, reducing variance and improving overall model performance. Unlike many state-of-the-art approaches that rely on single predictions, ensemble methods provide greater stability and reliability in multimodal classification.

Ablation study

Ablation study to systematically analyze and assess the contribution of various components of a model or framework, such as data modalities, ensemble methods, and the enhanced U-Net with self-attention (SAU-Net). A comprehensive ablation study was conducted to evaluate the contribution of each component in the proposed framework for Alzheimer’s disease prediction. The study investigates the individual and collective impacts of multimodal data, the enhanced U-Net model with self-attention (SAU-Net), moth flame optimization (MFO), and the ensemble mechanism. This study helps identify the performance improvements achieved at each stage of the framework, enabling a better understanding of the model’s architecture and its individual components.

• i.

  Effect of Individual Modalities (MRI, PET, Neuropsychological Tests): Performance with single modalities (e.g., VGG-16, ResNet-50, DenseNet) is significantly lower than with multimodal and fused data frameworks. The following are key observations: (i) MRI alone achieves an AUC of 87.5% (VGG-16), (ii) PET alone achieves an AUC of 89.4% (ResNet-50), and (iii) Neuropsychological tests alone achieve an AUC of 90.2% (DenseNet). The individual modalities capture disease-specific features; however, they do not fully exploit complementary information from other modalities.

• ii.

  Effect of Multimodal Fusion: Using multimodal data without feature fusion achieves an AUC-ROC of 91.1%, demonstrating the complementary nature of MRI, PET, and neuropsychological test data. Incorporating feature fusion further enhances the model’s performance, achieving an AUC-ROC of 91.1% and demonstrating the value of combining information across modalities.

• iii.

  Impact of the Enhanced SAU-Net: Replacing baseline models with the Proposed SAU-Net (without optimization) increases AUC-ROC to 93.5%, showcasing the effectiveness of the self-attention mechanism in capturing intricate relationships across fused features.

• iv.

  Impact of MFO: Introducing MFO to optimize the SAU-Net model’s hyperparameters further enhances its performance, achieving an AUC-ROC of 93.8%. This indicates that MFO successfully finetunes the model for better classification.

• v.

  Impact of Ensemble Methods: The final ensemble approach combines the outputs of multiple predictions from the SAU-Net model with MFO. The ensemble method increases robustness and predictive power, resulting in the highest AUC-ROC of 93.5% and the best overall metrics, including an accuracy of 91%.

Comparison with existing methods

To benchmark against recent multimodal transformer-based frameworks, we compared our UNet++_(SA)–MFO with AD-Transformer (Yu et al., 2024) and FusionNet (Muksimova et al., 2025a). AD-Transformer, a unified transformer fusing sMRI, clinical, and genetic data, achieves 89.0% effective multi-class accuracy but is limited to binary tasks. FusionNet, integrating MRI/PET/CT with longitudinal analysis, reaches 90.0% accuracy. Our framework outperforms both, with 91.0% accuracy and 93.5% AUC-ROC on the expanded ADNI cohort (n = 500), due to self-attention-enhanced feature extraction and MFO optimization (Table 4).

Model interpretability and clinical validation

To enhance clinical trust and facilitate adoption of the UNet++_(SA)–MFO framework in real-world diagnostic workflows, we conducted comprehensive interpretability analyses using three complementary visualization techniques: attention heatmaps, saliency maps, and feature attribution analysis. These methods provide neurologists and radiologists with transparent, anatomically grounded explanations of model predictions.

Saliency map analysis

The saliency overlay maps from the sagittal, coronal, and axial perspectives illustrate the spatial distribution of model attention in predicting Alzheimer’s Disease using MRI data (Fig. 7). In all three orientations, the emphasized regions correspond to areas spanning the medial temporal lobe, with strong saliency localized around the hippocampus. The red intensity overlays indicate voxels that significantly affected the model’s categorization, suggesting that the network predominantly relies on hippocampal structures and adjacent cortical regions to distinguish between Alzheimer’s-affected brains and healthy controls. This spatial emphasis is uniform across perspectives, underscoring the hippocampus as a pivotal biomarker in Alzheimer’s pathogenesis, characterized by gradual atrophy. The significance of saliency in the hippocampus area enhances the model’s interpretability by illustrating its dependence on clinically pertinent neuroanatomical characteristics. In summary, these saliency overlays validate that the deep learning model identifies significant structural patterns related to Alzheimer’s development, rather than relying on irrelevant global brain characteristics.

Attention heatmaps analysis

The Fig. 8 illustrates the attention heatmaps generated by the proposed UNet++_SA–MFO model, utilized for identifying AD in structural MRI scans. Brighter yellow and orange regions signify higher attention weights, highlighting voxels that significantly influence the classification decision. The model demonstrates significant activations in the medial temporal lobe, hippocampal formations, and parietal-temporal cortical regions, which are recognized biomarkers of Alzheimer’s disease. The emphasis on hippocampal atrophy and ventricular enlargement indicates that the model has effectively correlated neurodegenerative anatomical features with disease.

Feature attribution analysis

The feature attribution analysis employing Integrated Gradients and Grad-CAM offers significant insight into the deep learning model’s interpretation of structural brain MRI data for Alzheimer’s Disease prediction (Fig. 9). The Integrated Gradients map showed detailed voxel-level significance localized within the medial temporal lobe, especially around the hippocampus. This suggests that little intensity fluctuations in these areas significantly affect the model’s choice, indicating sensitivity to hippocampus tissue integrity. The Grad-CAM image, conversely, emphasized extensive spatial activation patterns focused on the hippocampus formation and the neighboring inferior horn of the lateral ventricle. These areas reflect characteristic anatomical alterations in Alzheimer’s pathogenesis, such as hippocampal atrophy and ventricular enlargement. The attribution maps collectively indicate that the model concentrates on physically and clinically pertinent regions linked to early neurodegeneration in Alzheimer’s Disease. The alignment between model attention and recognized biomarkers enhances the interpretability and biological validity of the network’s predictions.

External validation and generalization plan

To assess generalizability, we will: (i) evaluate on OASIS-3 and AIBL using the available modalities (MRI + NP; PET when available), (ii) apply scanner/site harmonization (e.g., ComBat and intensity standardization) and domain adaptation (feature-level alignment) before inference, (iii) reproduce the ADNI split strategy and compute identical metrics, and (iv) report confidence intervals and significance testing across sites. In parallel, we will collaborate with clinical partners to curate a multi-center cohort with PET availability and standardized NP batteries. We will preregister the validation protocol, release configuration files and code for reproducibility, and explore model compression (e.g., pruning, quantization) to reduce inference latency toward deployment.