Synthetic data analysis for early detection of Alzheimer progression through machine learning algorithms

Introduction

Alzheimer’s disease (AD), is one of the most common neurodegenerative diseases that primarily affects brain tissue (Masters et al., 2015; Ashayeri et al., 2024), with older adults being the most affected. The global prevalence of dementia exceeds 55 million individuals, a figure projected to escalate to 139 million by 2050, particularly affecting low and middle-income countries. Notably, 60% of those afflicted currently reside in such nations, a proportion of it, is anticipated to surge to 71% by 2050. The onset of a new dementia case presents approximately every 3 s worldwide, yet up to three-quarters of those affected lack a formal diagnosis.

Anxiety regarding dementia is widespread, with nearly 80% of the populace expressing apprehension about its potential onset and a quarter perceiving dementia as inevitable and unpreventable. Misconceptions regarding dementia prevail, as approximately 62% of healthcare practitioners mistakenly stake it as an inherent aspect of aging. Furthermore, a significant portion of caregivers, approximately 35%, admit to concealing a family member’s dementia diagnosis.

The toll of caregiving on health is considerable, with over half of caregivers reporting a decline in their own health due to their responsibilities, despite expressing positive sentiments regarding their caregiving roles. These statistics underscore the urgent need for enhanced awareness, support and interventions to address the multifaceted challenges posed by dementia on a global scale (Long & Benoist, 2023). It is possible to detect AD through certain biomarkers present in the cerebrospinal fluid and the accumulation of Beta amyloid, as well as the use of neurological images (Masters et al., 2015).

Dementia encompasses various neurological conditions characterized by impairments in memory, cognition, behavior and emotion. Early manifestations often include memory loss, challenges in executing familiar tasks, linguistic difficulties and alterations in personality. Despite extensive research efforts, dementia remains devoid of a definitive cure, nevertheless, numerous support mechanisms exist to aid individuals affected by dementia and their caregivers. Remarkably, dementia disregards societal divisions, impacting individuals across all social, economic and ethnic strata.

As AD stands the foremost etiology of dementia, vascular disease, dementia with Lewy bodies and frontotemporal dementia represent significant contributors to this multifaceted syndrome.

By implementing machine learning (ML) and deep learning techniques, which are part of artificial intelligence, the necessary conditions have been generated to be able to process large amounts of information and promote problem solving, the improvement of diagnostic support tools and establish new lines of action based on the results of each analysis.

According to the World Health Organization (WHO), the global population aged 60 and above reached approximately one billion individuals in 2020, with projections indicating a doubling to 2.1 billion by the foreseeable future. Notably, the demographic of individuals aged 80 and older is anticipated to undergo a threefold increase during this period, reaching a staggering 426 million individuals.

The economic burden associated with dementia is poised to escalate significantly, WHO forecasts suggest that the annual costs linked to dementia will more than double from $1.3 trillion in 2019 to an estimated $2.8 trillion by 2030. This public challenge underscores the imperative need for proactive measures in healthcare policy, research and caregiving infrastructure to address the evolving needs of aging populations and mitigate the socioeconomic ramifications of dementia on a global scale (Long & Benoist, 2023).

Assessment of AD progression encompasses a multifaceted approach, leveraging specialized clinical data amalgamated from comprehensive clinical histories, neuropsychological evaluations and diverse clinical assessments like blood assays and electroencephalography. This comprehensive array of studies serves to discern risk factors and distinguish symptoms akin to dementia. Through integration into machine learning frameworks, this data enables predictive evaluation criteria, facilitating tailored action plans or additional investigations based on the patient’s current condition or accurate exclusion of the ailment. Simultaneously, clinical features can be simulated, marking a pivotal stride towards personalized medicine.

Methods

According to the stages of the processes presented in the research of Sánchez-Reyna et al. (2021) and Morgan-Benita et al. (2022), the methodology proposed for this study also consists of six stages, adapting each individual process to the experimentation carried out in this study, as shown in the Fig. 1. Firstly, in Stage 1 (Fig. 1A), a comprehensive description of the datasets utilized in the study is provided. Following this, Stage 2 (Fig. 1B) involves the creation of the dataset of interest by selecting subjects based on predefined inclusion criteria. Additionally, this stage encompasses the verification and treatment of empty fields within the dataset, as well as data transformation procedures as necessary. In Stage 3 (Fig. 1C), the dataset is divided into thirteen different groups, which are then segmented into two experimental phases. In Phase 1, ten datasets are used due to the incorporation of five cognitive states for subject diagnosis. These ten datasets are mentioned below: CN vs subjective memory concern (SMC), CN vs early mild cognitive impairment (EMCI), CN vs late mild cognitive impairment (LMCI), CN vs AD, SMC vs EMCI, SMC vs LMCI, SMC vs AD, EMCI vs LMCI, EMCI vs AD and LMCI vs AD. On the other hand, Phase 2 involves three datasets, as a reorganization of the diagnoses is carried out in accordance with the ADNI criteria, resulting in the use of three cognitive states for subject diagnosis. Phase 2, three datasets are delineated as follows: CN vs MCI, CN vs AD and MCI vs AD. Furthermore, an additional dataset is generated alongside the original one. The original dataset is termed an “Original” dataset and a “Synthetic” dataset is created from it. Both datasets are utilized for each of the classifications proposed in this stage, resulting in a total of twenty-six datasets for developing models in the classifier ensemble. Subsequently, in Stage 4 (Fig. 1D), feature selection is conducted employing recursive feature elimination (RFE) and Akaike Information Criterion (AIC) techniques to identify the most relevant neuropsychological assessments for analysis. In stage 5 (Fig. 1E), a set of classifiers is used to generate ML models to evaluate the set of neuropsychological assessments obtained from the previous stage. This stage is designed to evaluate the predictive ability of selected neuropsychological assessments to distinguish between various cognitive states. This process is performed on both Original and Synthetic datasets. Moreover, once the best set of neuropsychological assessments in the Synthetic dataset was identified, these neuropsychological assessments were taken for “Custom model” building, with the goal of generating a smaller ML model, allowing to reduce the number of neuropsychological assessments, so that the selected neuropsychological assessments can distinguish between various cognitive states. Finally, in Stage 6 (Fig. 1F), the performance of the developed models is validated using various metrics including sensitivity, specificity, precision, accuracy and F1-score. These metrics are employed to evaluate the robustness and efficacy of the models in accurately predicting disease progression and classification.

Results

The original database called ADNIMERGE has 16,431 observations and 116 features. The data analyzed passed various filters consisting of 323 patients, all of them satisfying the inclusion criteria described in Table 3. The ADNIMERGE data was treated with data imputation and applying the inclusion criteria, finally 14 resulting features were obtained which are shown in Table 4. These features were processed by RFE and AIC, obtaining diverse sets of features that were then integrated into the ML models within the ensemble (see Tables S1 and S2, are presented in the supplementary files). The comparison between the number of subjects in the original data set by degree of cognitive impairment with the number of subjects in the synthetic data set is shown in Table S3, a representative figure to this data: https://github.com/unciafidelis/RFEAIC (Morgan, 2024).

The mean and standard deviation were calculated to analyze central tendency and statistical dispersion for all datasets: CN vs SMC, CN vs EMCI, CN vs LMCI, CN vs AD, SMC vs EMCI, SMC vs LMCI, SMC vs AD, EMCI vs LMCI, EMCI vs AD, LMCI vs AD, MCI vs AD, CN vs AD and CN vs MCI, the results of these statistical calculations are presented in the supplementary files.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the CN vs SMC (phase 1) datasets, the results of these statistical calculations are presented in Table S4, the results of these statistical calculations are presented in the supplementary files.

Table 5 presents the performance metrics for different ML models used to classify CN vs SMC in Phase 1 in the ensemble. The Custom feature selection model, built using a reduced set of neuropsychological assessments selected via the AIC, consistently outperforms the other models. It achieves the highest sensitivity of 0.6563, specificity of 0.7778, precision of 0.8400, accuracy of 0.7000 and F1-score of 0.7368, indicating that it is the most robust and reliable in distinguishing between the two cognitive states. The feature set using the Original dataset, exhibits perfect sensitivity of 1.0000 but suffers from lower precision of 0.1429 and a F1-score of 0.2500. The feature set with the Synthetic dataset, shows moderate performance with sensitivity of 0.6129, specificity of 0.6842, precision of 0.7600, accuracy of 0.6400 and F1-score of 0.6786, but still falls short compared to the Custom model. The Custom model demonstrates superior balance and effectiveness in classification compared to both the Original and Synthetic models.

The feature “FAQ” was obtained through RFE in the original dataset CN vs SMC. In contrast, the features “AGE”, “ADAS11”, “ADAS13” and “TRABSCOR” were obtained in the synthetic dataset. Additionally, the “AGE” and “ADAS13” features was derived from a custom model. Subsequently, all models were tested in the ensemble to generate the metrics presented in Table 5.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the CN vs EMCI (phase 1), the results of these statistical calculations are presented in Table S5, the results of these statistical calculations are presented in the supplementary files.

Table 5 summarizes the performance metrics for the models used to classify CN vs EMCI in Phase 1 in the ensemble. The feature set utilizing the Original dataset model, shows perfect sensitivity of 1.0000, indicating it correctly identifies all positive cases. The feature set using the Synthetic dataset and Custom feature selection models, have the same sensitivity of 0.9474, demonstrating high performance in detecting positive cases, though slightly lower than the Original model. In terms of specificity, both the Synthetic and Custom models achieve equal values of 0.7576, which are slightly lower than the Original’s specificity of 0.7813. For precision, the Original’s models perform best of 0.7308, with the Synthetic and Custom, having a precision of 0.6923. The Original also leads in accuracy of 0.8627 compared to the Synthetic and Custom, which both have an accuracy of 0.8269. Finally, the Original model achieves the highest F1-score of 0.8444, reflecting its superior balance between precision and recall, while the Synthetic and Custom models both have an F1-score of 0.8000. The feature set with the Original dataset demonstrates the best performance across all metrics, with the Synthetic and Custom models showing similar results but falling short in comparison.

In the original dataset CN vs EMCI, the features “PTGENDER”, “CDRSB” and “FAQ” were obtained through RFE. Similarly, in the synthetic dataset and the custom model, the same features “PTGENDER”, “CDRSB” and “FAQ” were included. The models incorporating these features were evaluated using ensemble implementation and the resulting metrics are presented in Table 5.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the CN vs LMCI (phase 1) datasets, the results of these statistical calculations are presented in Table S6, the results of these statistical calculations are presented in the supplementary files.

Table 5 presents the performance metrics for models classifying CN vs LMCI in Phase 1 implemented in the ensemble. All three feature sets in Original and Synthetic datasets and also Custom feature selection models achieve identical results across all metrics. Each model exhibits perfect sensitivity 1.0000, indicating flawless detection of positive cases. Specificity is also uniform at 0.9615, reflecting the models’ high accuracy in identifying negative cases. Precision is slightly higher in the Synthetic and Custom models of 0.9600 compared to the Original model of 0.9565, though the difference is minimal. Accuracy is nearly the same across models, with the Synthetic and Custom models at 0.9800 and the Original model at 0.9792. The F1-score is also closely aligned, with the Synthetic and Custom models achieving 0.9796, while the Original model scores slightly lower at 0.9778. Overall, all models demonstrate exceptional performance with nearly identical outcomes, highlighting that there is no significant difference in their effectiveness for this classification task.

In the original dataset CN vs LMCI, the features “CDRSB”, “LDELTOTAL” and “FAQ” were obtained through RFE. Similarly, in the synthetic dataset and the custom model, the same features “CDRSB”, “LDELTOTAL” and “FAQ” were included. The models incorporating these features were evaluated using ensemble implementation and the resulting metrics are presented in Table 5.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the CN vs AD (phase 1) datasets, the results of these statistical calculations are presented in Table S7, the results of these statistical calculations are presented in the supplementary files.

Table 6 illustrates the performance metrics for feature sets classifying CN vs AD in Phase 1 in the ensemble. All three feature sets with the Original and Synthetic datasets and the Custom feature selection models achieve perfect scores across all metrics. Specifically, they all exhibit a sensitivity, specificity, precision, accuracy and F1-score of 1.0000. This indicates that the models flawlessly distinguish between CN and AD cases, with no false positives or false negatives and perfectly predict all instances correctly. However, the perfect scores across all metrics for all models suggest a potential issue of overfitting. Overfitting occurs when a model learns the training data too well, including the noise and outliers, leading to excellent performance on the training set but potentially poor generalization to new, unseen data. This is particularly concerning in medical diagnostics, where the model’s ability to generalize to different patient populations is crucial. The results imply that while the models perform perfectly on the provided datasets, further validation with independent datasets is essential to ensure their robustness and generalizability.

In the original dataset CN vs AD, the feature “FAQ” was extracted by RFE. Similarly, “FAQ” was also the output of RFE in the synthetic dataset, as well as in the custom model. The models incorporating this feature were evaluated using ensemble implementation and the resulting metrics are presented in Table 6.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the SMC vs EMCI (phase 1) datasets, the results of these statistical calculations are presented in Table S8, the results of these statistical calculations are presented in the Supplementary Files.

Table 6 summarizes the performance metrics for models distinguishing SMC vs EMCI in Phase 1 in the ensemble. The feature set with the Synthetic dataset and the Custom feature selection models display identical performance, with a sensitivity of 0.9130, specificity of 0.8276, precision of 0.8077, accuracy of 0.8654 and F1-score of 0.8571, indicating their strong capability in accurately identifying both positive and negative cases while maintaining a balanced trade-off between precision and recall. The feature selection with the Original dataset, though slightly underperforming in sensitivity with 0.8846 and specificity of 0.7857, demonstrates higher precision of 0.8846 and F1-score of 0.8846, suggesting a greater proportion of true positive predictions among the positives. Overall, the Synthetic and Custom models offer a slight edge in balanced classification accuracy and consistency, while the Original model excels in precision, making it potentially more reliable for correctly identifying true positives.

In the original dataset SMC vs EMCI, the features “AGE”, “CDRSB” and “TRABSCOR” were obtained through RFE. Similarly, in the synthetic dataset and the custom model, the same features “AGE”, “CDRSB” and “TRABSCOR” were included. The models incorporating these features were evaluated using ensemble implementation and the resulting metrics are presented in Table 6.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the SMC vs LMCI (phase 1) datasets, the results of these statistical calculations are presented in Table S9, the results of these statistical calculations are presented in the supplementary files.

Table 6 provides the performance metrics for models classifying SMC vs LMCI in Phase 1 in the ensemble. The feature set with the Original dataset model achieves perfect scores across all metrics, with a sensitivity, specificity, precision, accuracy and F1-score of 1.0000, indicating flawless identification of both positive and negative cases without any errors. The feature set with the Synthetic dataset model shows high, yet slightly lower, performance with a sensitivity of 0.8846, specificity of 1.0000, precision of 1.0000, accuracy of 0.9348 and F1-score of 0.9388, suggesting excellent classification but with a small number of false negatives. The Custom feature selection model, while still robust, exhibits a sensitivity of 0.8750, specificity of 0.9091, precision of 0.9130, accuracy of 0.8913 and F1-score of 0.8936. Although these metrics are slightly lower than those of the Original and Synthetic models, they demonstrate good balance between true positive and true negative predictions. Overall, while the Original model performs perfectly, indicating potential overfitting, the Synthetic model maintains strong performance with perfect specificity and the Custom model offers a slightly lower but balanced classification capability.

In the original dataset SMC vs LMCI, the features “AGE”, “CDRSB”, “ADAS11”, “ADAS13”, “RAVLT.immediate”, “LDELTOTAL” and “TRABSCOR” were obtained through RFE. In the synthetic dataset, the features “PTGENDER”, “PTMARRY”, “CDRSB”, “ADAS11”, “ADAS13”, “ADASQ4”, “MMSE” and “LDELTOTAL” were included. Lastly, in the custom model, only “ADAS13” and “LDELTOTAL” were used as features. The ensemble of these models was evaluated and the resulting metrics are presented in Table 6.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the SMC vs AD (phase 1) datasets, the results of these statistical calculations are presented in Table S10, the results of these statistical calculations are presented in the supplementary files.

Table 7 presents the performance metrics for models distinguishing SMC vs AD in Phase 1 in the ensemble. The feature set with the Original dataset model achieves perfect sensitivity of 1.0000, indicating flawless identification of all positive cases, but has lower specificity of 0.8750, precision of 0.6667, accuracy of 0.9000 and F1-score of 0.8000. This suggests a higher rate of false positives, reducing the reliability of its positive predictions. The feature set with the Synthetic dataset model, with perfect scores across all metrics (1.0000 for sensitivity, specificity, precision, accuracy and F1-score), demonstrates an ideal balance with flawless identification of both positive and negative cases, indicating potential overfitting. The Custom feature selection model also shows strong performance, achieving perfect sensitivity (1.0000) and nearly perfect scores for specificity (0.9333), precision (0.9286), accuracy (0.9643) and F1-score (0.9630). While the Custom feature selection model slightly underperforms compared to the feature set with the Synthetic dataset model, it still maintains a high level of accuracy and reliability in distinguishing between SMC and AD cases, providing a robust and well-balanced classification.

In the original dataset SMC vs AD, the features “CDRSB”, “ADAS13”, “ADASQ4”, “RAVLT.immediate”, “RAVLT.learning”, “LDELTOTAL” and “FAQ” were obtained through RFE. In the synthetic dataset, the same features “CDRSB”, “ADAS13”, “ADASQ4”, “RAVLT.immediate”, “LDELTOTAL” and “FAQ” were included. Lastly the custom model, only “ADAS13” was utilized as a feature. The ensemble model incorporating these features was evaluated and the resulting metrics are described in Table 7.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the EMCI vs LMCI (phase 1) datasets, the results of these statistical calculations are presented in Table S11, the results of these statistical calculations are presented in the supplementary files.

Table 7 shows the performance metrics for models differentiating EMCI vs LMCI in Phase 1 in the ensemble. The feature set with the Synthetic dataset model exhibits the best performance overall, with the highest sensitivity of 0.9583, indicating superior detection of positive cases and a strong specificity of 0.8929, showing effective identification of negative cases. It also has high precision of 0.8846, accuracy of 0.9231 and F1-score of 0.9200, reflecting a well-balanced model with minimal false positives and negatives. The feature set with the Original model demonstrates decent performance with a sensitivity of 0.7692, specificity of 0.8696, precision of 0.8696, accuracy of 0.8163 and F1-score of 0.8163. Although it has good specificity and precision, its lower sensitivity suggests it may miss more true positive cases compared to the Synthetic model. The Custom feature selection model, while showing reasonable performance, falls behind with a sensitivity of 0.8571, specificity of 0.7419, precision of 0.6923, accuracy of 0.7885 and F1-score of 0.7660. The Custom model’s lower specificity and precision indicate a higher rate of false positives and its overall lower metrics suggest it is less effective than the Synthetic model in this classification task. Overall, the Synthetic model demonstrates the highest performance, making it the most reliable for distinguishing between EMCI and LMCI.

In the original dataset EMCI vs LMCI, the features “AGE”, “ADAS11”, “ADAS13”, “RAVLT.immediate”, “LDELTOTAL”, “TRABSCOR” and “FAQ” were identified. In the synthetic dataset, the features “AGE”, “ADAS11”, “ADAS13”, “ADASQ4”, “MMSE”, “RAVLT.immediate”, “LDELTOTAL”, “TRABSCOR” and “FAQ” were included. Lastly, in the custom model, only “AGE”, “ADAS13” and “RAVLT.immediate” were utilized as features. The ensemble model incorporating these features was evaluated and the resulting metrics are presented in Table 7.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the EMCI vs AD (phase 1) datasets, the results of these statistical calculations are presented in Table S12, the results of these statistical calculations are presented in the supplementary files.

Table 7 presents the performance metrics for models distinguishing EMCI vs AD in Phase 1 in the ensemble. Both the feature set with the Synthetic dataset and Custom feature selection models exhibit identical, high performance with a sensitivity of 0.9583, specificity of 0.8929, precision of 0.8846, accuracy of 0.9231 and F1-score of 0.9200. These results indicate excellent detection of positive cases and strong identification of negative cases, along with a high proportion of true positive predictions among all positive predictions. The feature set with the Original dataset model, while slightly underperforming compared to the Synthetic and Custom models, still shows strong performance with a sensitivity of 0.7143, specificity of 0.9600, precision of 0.8333, accuracy of 0.9063 and F1-score of 0.7692. This suggests that while the Original model may miss more positive cases, it effectively identifies negative cases. Overall, the Synthetic and Custom models demonstrate the best and most balanced performance, making them the most reliable for distinguishing between EMCI and AD. The Original model, despite being less sensitive, maintains a strong specificity and good overall performance.

In the original dataset EMCI vs AD, the features “CDRSB”, “ADAS13”, “LDELTOTAL”, “TRABSCOR” and “FAQ” were identified through RFE. In the synthetic dataset, the features “CDRSB”, “ADAS11”, “ADAS13”, “RAVLT.immediate”, “RAVLT.learning”, “LDELTOTAL” and “TRABSCOR” were included. Additionally, in the custom dataset, only “CDRSB” and “ADAS13” were utilized as features. The ensemble model incorporating these features was evaluated and the resulting metrics are presented in Table 7.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the LMCI vs AD (phase 1) datasets, the results of these statistical calculations are presented in Table S13, the results of these statistical calculations are presented in the Supplementary Files.

Table 8 shows the performance metrics for models distinguishing LMCIvsAD in Phase 1 in the ensemble. The Custom feature selection model demonstrates the best performance with a sensitivity of 0.7778, specificity of 0.8947, precision of 0.9130, accuracy of 0.8261 and F1-score of 0.8400. This indicates a strong ability to identify both positive and negative cases accurately, with a high proportion of true positive predictions and a balanced trade-off between precision and recall. The feature set with the Synthetic dataset model, while showing a reasonable performance, has a lower sensitivity of 0.6429, specificity of 0.7222, precision of 0.7826, accuracy of 0.6739 and F1-score of 0.7059, suggesting a relatively higher rate of false negatives and positives compared to the Custom model. The feature set with the Original dataset model’s performance metrics are not available (NA) for sensitivity and F1-score, with precision also being 0.0000, indicating a potential issue in identifying positive cases. However, it has a specificity of 0.7931 and accuracy of 0.7931, indicating reasonable identification of negative cases. Overall, the Custom model exhibits the best overall performance in distinguishing between LMCI and AD, followed by the Synthetic model, while the Original model appears to lack effectiveness in positive case detection.

In the original dataset LMCI vs AD, the feature “RAVLT.learning” was identified through RFE. In the synthetic dataset, the features “AGE”, “CDRSB”, “RAVLT.immediate”, “RAVLT.learning” and “FAQ” were included. Also, the custom model, only “AGE” and “CDRSB” were utilized as features. The ensemble model incorporating these features was evaluated and the resulting metrics are presented in Table 8.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the CN vs MCI (phase 1) datasets, the results of these statistical calculations are presented in Table S14, the results of these statistical calculations are presented in the supplementary files.

Table 9 displays the performance metrics for models distinguishing CN vs MCI in Phase 2 in the ensemble. The feature set with the Synthetic dataset model exhibits the highest overall performance, with a sensitivity of 0.9574, specificity of 0.9057, precision of 0.9000, accuracy of 0.9300 and F1-score of 0.9278. These metrics indicate an excellent balance in identifying true positive and negative cases, with a high proportion of accurate positive predictions. The Custom feature selection model shows strong specificity (0.9767) and precision (0.9800), indicating effective identification of negative cases and a high rate of correct positive predictions. However, it has a slightly lower sensitivity (0.8596) and F1-score (0.9159), suggesting it may miss some positive cases despite high overall accuracy (0.9100). The feature set with the Original dataset model demonstrates good performance, with a sensitivity of 0.9130, specificity of 0.8182, precision of 0.8400, accuracy of 0.8667 and F1-score of 0.8750. While the Original model performs well, it is slightly less balanced than the Synthetic and Custom models, particularly in terms of specificity and precision. Overall, the Synthetic model shows the best balance and highest accuracy in distinguishing between CN and MCI, followed by the Custom and Original models.

In the original dataset CN vs MCI, the features “PTGENDER”, “CDRSB” and “ADAS11” were identified through RFE. In the synthetic dataset, the features “PTGENDER” and “CDRSB” were included. Moreover, in the custom model, only “CDRSB” is utilized as feature. The ensemble model incorporating these features was evaluated and the resulting metrics are presented in Table 9.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the CN vs AD (phase 2) datasets, the results of these statistical calculations are presented in Table S15, the results of these statistical calculations are presented in the supplementary files.

Table 9 presents the performance metrics for models distinguishing CN vs AD in Phase 2 in the ensemble. The feature set with the Original model achieves perfect scores across all metrics with a sensitivity, specificity, precision, accuracy and F1-score of 1.0000, indicating flawless identification of both positive and negative cases. This suggests that the Original model is highly effective in this classification task, though it may indicate potential overfitting due to its perfect scores. The feature set with the Synthetic dataset and Custom feature selection models both show strong performance, with identical metrics: a sensitivity of 0.9231, specificity of 0.9024, precision of 0.9000, accuracy of 0.9125 and F1-score of 0.9114. These models also demonstrate high effectiveness in identifying both positive and negative cases with a slight trade-off between sensitivity and specificity compared to the Original model. Overall, while the Original model performs flawlessly, the Synthetic and Custom models offer robust and reliable performance, making them valuable for distinguishing between CN and AD.

In the original dataset CN vs AD, the features “CDRSB”, “ADASQ4” and “RAVLT.learning” were identified through RFE. In the synthetic dataset, “RAVLT.immediate” and “FAQ” were included. Furthermore, in the custom model, “RAVLT.immediate” and “FAQ” were also utilized as features. The ensemble model incorporating these features was evaluated and the resulting metrics are presented in Table 9.

The mean and standard deviation were calculated to analyze the central tendency and statistical dispersion of the data, for the MCI vs AD (phase 2) datasets, the results of these statistical calculations are presented in Table S16, the results of these statistical calculations are presented in the Supplementary Files.

Table 9 presents the performance metrics for models distinguishing MCIvsAD in Phase 2 in the ensemble. The feature set with the Synthetic model exhibits the highest performance with a sensitivity of 0.9184, specificity of 0.9020, precision of 0.9000, accuracy of 0.9100 and F1-score of 0.9091. This indicates a strong ability to identify both positive and negative cases accurately, with a well-balanced trade-off between precision and recall. The Custom feature selection model also shows robust performance with a sensitivity of 0.8276, specificity of 0.9524, precision of 0.9600, accuracy of 0.8800 and F1-score of 0.8889. It demonstrates a higher precision and specificity compared to the Synthetic model but has slightly lower sensitivity and accuracy. The feature set with the Original dataset model, while performing reasonably, shows lower metrics with a sensitivity of 0.5000, specificity of 0.9231, precision of 0.3333, accuracy of 0.8929 and F1-score of 0.4000. This suggests it is less effective in detecting positive cases compared to the Synthetic and Custom models. Overall, the Synthetic model provides the best balanced performance for distinguishing between MCI and AD, followed by the Custom model, while the Original model lags behind in effectiveness.

In the original dataset MCIvsAD, the feature “FAQ” was identified through RFE. In the synthetic dataset, the features “CDRSB”, “ADAS13”, “RAVLT.immediate”, “RAVLT.learning”, “LDELTOTAL” and “FAQ” were included. Additionally, in the custom model, only “ADAS13” was utilized as a feature. The ensemble model incorporating these features was evaluated and the resulting metrics are presented in Table 9.

Discussion

The proposed methodology demonstrates the effectiveness of using ML models for feature selection and classification of subjects based on their level of cognitive impairment. The study utilized data from the ADNIMERGE dataset, encompassing various ADNI study groups. The dataset underwent a selection process, focusing on genetic indices and neuropsychological assessments, as well as nominal scaling for qualitative features and NA data filtering. Balancing techniques were applied to address data imbalances for behavioral analysis.

The initial dataset was then separated into different subsets corresponding to phases 1 and 2, aiming to establish a continuum of cognitive transitions throughout the disease stages. This approach led to the creation of thirteen datasets, each targeting specific cognitive transitions, suggesting a statistical relationship between stages. RFE was employed on each dataset to select the most significant features, followed by the creation of ensemble multivariate models for diagnosing AD or identifying the progression of cognitive impairment. This methodology enabled the establishment of more sensitive neuropsychological assessments for each disease stage.

Results analysis revealed that features such as “ADAS13” “CDRSB” and “AGE” were frequently selected in the custom models. These features are well-established in individual diagnostic assessments, with ADAS13 proposed by Mohs (1983), CDRSB proposed by Morris (1991) and AGE being a universally recognized factor (Hebert et al., 1995; Castellani, Rolston & Smith, 2010). Their recurrent selection in various models underscores their importance in classifying patients with dementia. The study employed statistical validation metrics, including sensitivity, specificity and accuracy, to validate the ML models and their predictive capabilities.

According to Table 10, Divya, Shantha Selva Kumari & Initiative (2021) employed feature selection techniques, including RFE and genetic algorithms, to create optimized ML models for classifying subjects with different cognitive states. Their models demonstrated good sensitivity and accuracy for distinguishing between CN vs. MCI and CN vs. AD groups, but showed lower performance for the MCI vs. AD subjects. These models predominantly used features extracted from medical images (MRI) along with the MMSE neuropsychological test, with the number of features exceeding eighty. Likewise, the author Helaly, Badawy & Haikal (2022) used as a source of information images of 300 patients divided into four classes AD, EMCI, LMCI and NC, each class has 75 patients with a total number of images of 21 and 816 scans, to perform the implementation of pre-trained CNN to classify the four classes defined in an automated way. The proposed model of this research achieved very promising results with respect to several related researches; however, the analysis and processing of images turns out to be a much more complex process, which involves much more time and processing power. The present study demonstrates a performance equal or superior to the works based on image analysis, highlighting the contribution in the development of more efficient ML models focused on tabular data.

In contrast, the models proposed in the present work do not rely on medical image features and achieve comparable performance to those of Divya, Shantha Selva Kumari & Initiative (2021). This highlights the potential of using neuropsychological assessments as a diagnostic tool, especially in settings where access to medical imaging studies is limited. The ability to achieve similar performance without the need for medical imaging features strengthens the feasibility and practicality of the models proposed in this work.

The models presented in the present work demonstrate comparable performance to those proposed by Battista, Salvatore & Castiglioni (2017), Vinutha et al. (2020) and Franciotti et al. (2023), with the number of features exceeding five. These studies evaluate aspects such as dataset resizing, feature selection and the performance of ML models for diagnosing or classifying subjects with different cognitive states. Vinutha et al. (2020) showed that the CDRSB feature alone achieved an accuracy above 0.90, highlighting its importance in diagnosis, between NC and MCI. On the other hand, Franciotti et al. (2023) emphasized the significance of the Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAS-Cog-13) in global cognition assessment.

However, our models utilize only neuropsychological assessments, avoiding the need for laboratory tests or biopsies for diagnosing or classifying patients between CN, MCI or AD. This approach could be advantageous for patients who are unable or unwilling to undergo invasive tests. Furthermore, by identifying a set of relevant neuropsychological assessments, our models can form a neuropsychological battery for inferring between different cognitive states, providing a viable alternative for vulnerable patients.

In phase 1, CN vs AD the results were flawless with just a single feature in both datasets. By examining the statistical mean and standard deviation of this feature in each dataset, we can conclude that this outcome is not due to overfitting, “FAQ” is sufficiently effective in distinguishing between the two cognitive states. However, it is important to note that this conclusion may be influenced by the limited data available for AD patients (only 22 patients). A similar case is observed with SMCvsLMCI (phase 1) in the Original dataset, achieving a perfect score across all metrics. An analysis of the mean, standard deviation and the features selected by RFE suggests overfitting, likely due to the number of features and the disproportionate quantity of LMCI patients compared to CN patients. When using synthetic data, the performance changed, showing lower scores with a similar number of features, consistently including “CDRSB”, “ADAS11”, “ADAS13” and “LDELTOTAL”, this behavior indicates a corrected pattern and highlights the importance of these features. In phase 1, SMCvsAD comparison in the synthetic dataset shows a perfect score across all metrics. This case, which uses five features, clearly demonstrates overfitting. The issue seems to be the number of features, as the original dataset selected only one feature. Although the mean and standard deviation do not present substantial changes, one key point stands out: the limited quantity of AD data particularly affects this case, exacerbated by the limited number of SMC patients in comparison. The nature of synthetic data is to replicate values to match the original observations, which contributes to this overfitting problem. In Phase 2, the original dataset also presents a perfect score across all metrics. In this case, as in Phase 1, this comparison does not show overfitting because “FAQ” once again proves to be the key feature in differentiating between these cognitive states. However, the perfect score is present only in the original dataset in this phase, implying that this perfect score could change with more AD observations.

Lastly, an analysis was made of each of the features and how they are present in each of the models, in order to analyze their importance within the models.

• Age of the participant: Age is a critical factor in cognitive decline and is a well-established risk factor for AD and other dementias. It was selected through RFE and synthetic datasets in several comparisons (e.g., CN vs SMC, SMC vs EMCI, EMCI vs LMCI, LMCI vs AD). Its consistent selection indicates its strong association with disease progression and its role in predicting cognitive state.

• Gender of the participant: Gender differences in the prevalence and progression of AD are documented, with women being at higher risk. PTGENDER was identified through RFE in datasets such as CN vs EMCI and CN vs MCI, highlighting its importance in differentiating cognitive states potentially due to biological and hormonal differences.

• Marital status of the participant: Marital status can influence social support levels, which impact cognitive health. Although PTMARRY was only selected in the synthetic dataset for SMC vs LMCI, it suggests a possible correlation with cognitive function, possibly due to the psychological and social benefits of being married.

• Clinical dementia rating scale sum of boxes score: CDRSB is a direct measure of dementia severity and was consistently selected across multiple datasets (e.g., CN vs EMCI, CN vs LMCI, SMC vs EMCI, SMC vs LMCI, EMCI vs AD, LMCI vs AD, CN vs AD, MCI vs AD). Its consistent inclusion underscores its importance in accurately gauging cognitive decline.

• Alzheimer’s disease assessment scale, 11-item version: ADAS11 is used to evaluate the severity of cognitive symptoms associated with AD. Its selection in datasets such as CN vs SMC, SMC vs LMCI and EMCI vs LMCI indicates its relevance in differentiating stages of cognitive impairment.

• Alzheimer’s disease assessment scale, 13-item version: ADAS13 extends the ADAS11 by including additional tasks to capture more severe impairments. It was selected in various datasets (e.g., CN vs SMC, SMC vs LMCI, SMC vs AD, EMCI vs LMCI, EMCI vs AD, MCI vs AD), highlighting its comprehensive assessment capabilities for cognitive decline.

• Alzheimer’s disease assessment scale, question 4 score: ADASQ4 specifically measures memory recall, a key area affected early in AD. It was included in synthetic and custom models for SMC vs LMCI, SMC vs AD and EMCI vs AD, indicating its specific relevance in memory assessment.

• Mini-mental state examination score: MMSE is a widely used screening tool for cognitive impairment. Its inclusion in the synthetic dataset for SMC vs LMCI and EMCI vs LMCI points to its utility in general cognitive screening, despite being less frequently selected compared to other detailed scales like ADAS and CDRSB.

• Immediate recall score from the rey auditory verbal learning test: This score measures short-term auditory-verbal memory. Its selection in EMCI vs LMCI, SMC vs AD, EMCI vs AD and CN vs AD datasets reflects its importance in assessing memory functions, particularly those affected early in cognitive impairment.

• Learning score from the rey auditory verbal learning test: This score assesses the ability to learn new information, which can be impaired in AD. Its inclusion in SMC vs AD, EMCI vs AD, LMCI vs AD and CN vs AD highlights its role in evaluating cognitive functions related to learning.

• Forgetting score from the rey auditory verbal learning test: This score measures the rate of forgetting learned information. Although not highlighted in the provided datasets, it can be crucial for understanding specific memory deficits in AD.

• Total score from the longitudinal data entry list: This score captures comprehensive cognitive performance over time. Its frequent selection (e.g., CN vs LMCI, SMC vs LMCI, SMC vs AD, EMCI vs LMCI, EMCI vs AD, MCI vs AD) demonstrates its robustness in tracking cognitive decline.

• Total score from the test of recent abstraction: This score assesses abstract thinking and problem-solving abilities, which can decline in dementia. Its selection in various datasets (e.g., CN vs SMC, SMC vs EMCI, EMCI vs AD) indicates its relevance in evaluating cognitive flexibility and higher-order thinking.

• Functional activities questionnaire score: FAQ measures the ability to perform daily activities and is a critical indicator of functional decline in dementia. Its consistent selection across multiple datasets (e.g., CN vs SMC, CN vs EMCI, CN vs LMCI, CN vs AD, SMC vs AD, EMCI vs LMCI, EMCI vs AD, LMCI vs AD, MCI vs AD) highlights its importance in assessing real-world impacts of cognitive impairment.

Incorporating features such as AGE, PTGENDER and PTMARRY into ML models offers significant real-life applications by addressing key aspects of cognitive health (Sindi et al., 2021). AGE helps in predicting cognitive decline and Alzheimer’s risk, aiding in early intervention. PTGENDER considers gender differences in disease prevalence, enabling tailored screening and treatment. PTMARRY reflects social support levels, which impact cognitive function, guiding targeted interventions. CDRSB, ADAS11 and ADAS13 provide critical measures of dementia severity and cognitive impairment, facilitating accurate diagnosis and personalized treatment plans. ADASQ4 focuses on memory recall, essential for early detection of cognitive issues (Yi et al., 2023). MMSE, a widely used cognitive screening tool, helps in initial assessments, while RAVLT.immediate, RAVLT.learning and RAVLT.forgetting offer insights into memory functions and deficits. LDELTOTAL tracks comprehensive cognitive performance over time and TRABSCOR assesses abstract thinking and problem-solving abilities. FAQ measures daily functional abilities, indicating real-world impacts of cognitive impairment. Collectively, these features enhance ML models’ ability to support early diagnosis, personalized care and continuous monitoring, improving outcomes in real-world clinical settings.

Conclusions

In conclusion, the proposed methodology showcases the effectiveness of using ML models for feature selection and classification of subjects based on their cognitive impairment levels. By leveraging data from the ADNIMERGE dataset and employing a rigorous selection process focusing on neuropsychological assessments, the study successfully addressed data imbalances and established a continuum of cognitive transitions across disease stages. The use of RFE and ensemble multivariate models enabled the identification of sensitive neuropsychological assessments for each disease stage.

The study’s results highlight the importance of features such as “ADAS13”, “CDRSB” and “AGE” in classifying patients with different cognitive states, as they were frequently selected in the custom models. These features are well-established in individual diagnostic assessments, emphasizing their significance in cognitive impairment classification. The study’s statistical validation metrics, including sensitivity, specificity and accuracy, further validate the effectiveness of the ML models.

In comparison to previous studies, the models proposed in this work achieve comparable performance without relying on medical imaging features. This highlights the potential of using neuropsychological assessments as a diagnostic tool, especially in settings where access to medical imaging studies is limited. Overall, the models presented in this study offer a practical and viable alternative for diagnosing and classifying patients with different cognitive states, particularly for those who are unable or unwilling to undergo invasive tests.

While the results of this initial analysis are promising, several opportunities for future research are envisioned. These include refining synthetic data generation methods, validating model performance in larger cohorts and exploring additional clinical applications of synthetic data analysis in Alzheimer’s research and healthcare.

As future work we plan to explore new ensemble techniques and ML models to extend the implementations made here. Also, we will compare different information criteria, such as the Bayesian Information Criterion, to the one proposed by Akaike, to further enhance our methodology.

While RFE AIC has been valuable in our current study, exploring other feature selection methods that can be combined, providing additional insights and potentially improving our model’s performance and robustness. LASSO is known for its ability to perform both feature selection and regularization, which helps to enhance the prediction accuracy and interpretability of the statistical model it produces. Unlike RFE, which recursively eliminates features based on model performance, LASSO can shrink the coefficients of less important features to zero, thus selecting a simpler model that is less prone to overfitting.

Genetic algorithms are particularly useful for optimization problems and can explore a large search space efficiently. This makes them suitable for identifying optimal subsets of features that contribute most significantly to the model’s performance. Utilizing genetic algorithms can lead to the discovery of novel feature combinations that may not be apparent through traditional methods. This approach can enhance the robustness of our feature selection process by considering a wider array of potential solutions and optimizing for the best possible feature set.

Incorporating LASSO and genetic algorithms into our feature selection process will provide several advantages: enhanced model performance, reduced overfitting and comprehensive analysis. Each method offers unique strengths in selecting features that are most predictive, potentially leading to improved model accuracy and robustness. These techniques can help mitigate overfitting by either regularizing less important features (LASSO) and exploring a diverse set of feature combinations (genetic algorithms). The combined insights from these diverse methods will enable a more thorough understanding of feature importance and interactions, leading to better-informed decisions in model development.

This future work aims to build on the promising results of our current study, with the goal of developing more robust and applicable models for clinical practice. By continuing to refine our methods and validate our findings in diverse and larger populations, we hope to contribute significantly to the field of Alzheimer’s research and healthcare.

Supplemental Information