IMPACT: an interactive multi-disease prevention and counterfactual treatment system using explainable AI and a multimodal LLM

Data collection

The initial and critical phase necessitates collecting data sets from various hospitals and healthcare centers to develop a recommendation system to prevent multiple diseases. Due to the comprehensive nature of these data sets, categorization into specific disease groups is imperative. Upon organization, these categorized data sets are rendered significantly more viable for analyzing multiple diseases.

Let the number of hospitals be denoted by $n$. Several disease datasets were collected from each hospital. All hospitals do not need the same number and type of disease datasets. For instance, hospital $h1$ possesses $m$ disease datasets, hospital $h2$ has $i$, and hospital $hn$ has $j$ different disease datasets, with $m\ne i\ne j$ being a likely scenario. Similarly, the $d1$ disease dataset from hospital $h1$ does not have to be identical to the $d1$ disease dataset from hospital $h2$ ( $D_{h1,d1}\ne D_{h2,d1}$). In Eq. (1), datasets are segregated according to each disease ( $D_i$), where $\alpha$ and $\beta$ represent two different hospitals. In Eq. (2), $D_{t}ot$ is the set of all the disease datasets, with $k$ representing the total number of diseases.

(1) $$\begin{array}{c}{D}_i=\bigcup _{k=1}^n{D}_{hkdi},\\ where{D}_{h\alpha ,d1}=D_{h\beta ,d1}and\alpha \ne \beta \end{array}$$

(2) $$\begin{array}{c}D\mathrm{\_}tot=\{D_1,D_2,D_3,\dots D_i,\dots ,D_k\},\\ wherei\in \{1,2,\dots k\}.\hfill \end{array}$$

This recommendation system for preventing multiple diseases is meticulously designed to be highly personalized, adapting seamlessly to each user’s unique characteristics and needs. Initially, the user selects a specific set of diseases from those available in the system’s disease-wise datasets. This selection is designated as the user’s target disease. Personalization is emphasized by allowing different users to have distinct sets of target diseases, addressing their health concerns and risks. Equation (3) defines the target diseases $D\mathrm{\_}tg{t}_i$ for each user $U_i$, which must be within $D\mathrm{\_}tot$.

(3) $$\mathcal{D}\mathrm{\_}tg{t}_i\subseteq D\mathrm{\_}tot.$$

The requirement for personalized data extends to specific user information, which is crucial for each target disease ( $D\mathrm{\_}tg{t}_i$). Among the features of these target diseases, certain personal information, such as age, gender, weight, job type, residence type, and smoking status, must be provided by the IMPACT system during user interaction, as these features vary among individuals. Furthermore, wearable devices contribute valuable data by monitoring dynamic health indicators such as hypertension (H), blood sugar (BS), and oxygen levels, thereby further enriching the user profile.

The system differentiates between changeable features, such as weight and smoking status, and immutable ones, such as age, gender, and marital status, a distinction crucial for its recommendation mechanism. The system aims to minimize the risk of targeted diseases by proposing optimal values for adjustable features. For example, a suggested body mass index (BMI) target of 20.68153 can effectively lower associated health risks. Users can specify a range for any changeable feature to enhance personalization, such as a BMI range of 22 to 30. The system then recommends a value within this range, like 24.456, optimizing health outcomes within the user’s specified constraints. This method ensures recommendations are scientifically sound, practically achievable, and tailored to individual circumstances.

System model

This healthcare IMPACT system offers a highly personalized approach to disease prevention that adapts to each user’s unique features and preferences. The system model is depicted in Fig. 1. Utilizing disease data sets to predict multiple diseases begins with collecting diverse data ( $Master\mathrm{\_}Dataset(M_D)$) from various hospitals. This comprehensive data is meticulously organized by disease type, resulting in a structured repository ( $D\mathrm{\_}tot$). Users seeking to prevent multiple diseases can select a set of diseases from this repository ( $D\mathrm{\_}tot$). This user-specific selection is mathematically defined in Eqs. (3) and (4), where each user, denoted as $U_i$, has a unique set of targeted disease data ( $D\mathrm{\_}tg{t}_i$).

(4) $$\mathcal{D}\mathrm{\_}tg{t}_i=\{{\mathcal{D}}_1,{\mathcal{D}}_2,\dots ,{\mathcal{D}}_n\}.$$

Sensor data from smartwatches, fitness trackers, and biosensors are collected and integrated into the system, providing physiological and behavioral features such as heart rate, blood pressure, oxygen levels, sleep patterns, and physical activity. Users can also input additional health information through an interactive interface. The collected data undergoes preprocessing to ensure compatibility with the ML framework. Time-series sensor data is processed using statistical feature extraction (mean, standard deviation, and entropy), while discrete and categorical inputs are normalized using Min–Max Scaling and One-Hot Encoding. Missing data are handled through forward filling and KNN-based imputation. These approaches align with best practices in handling missing data in AI-driven models for complex systems, as discussed in Sharifi et al. (2024). The ML model employs a multi-input neural network, where convolutional layers extract temporal patterns while fully connected layers process structured inputs. These feature representations are then used to define the population for the multi-disease prevention problem, ensuring an effective and scalable health prediction system.

A spectrum of sophisticated machine learning models is deployed to ensure accurate predictions of the targeted diseases. The most effective model for each disease is determined through rigorous evaluative metrics, providing the best fit for each disease. Given the variety of diseases, denoted as $n$, a corresponding number of machine learning models ( $n$) is required. Each model is tailored to the characteristics of a specific disease dataset. For any user, $U_i$, the array of models employed can vary, adapting to the unique combination of diseases they are addressing, as represented in Eq. (5). These multiple models define numerous objectives for the multi-disease prevention problem.

(5) $${\mathcal{M}}_i=\{M_1,M_2,\dots ,M_n\}.$$

Prediction models for all target diseases and user feature information are obtained. Subsequently, a multi-dimensional counterfactual explanation is designed as in Fig. 2, utilizing the sophisticated multi-objective optimization technique known as the Non-dominated Sorting Genetic Algorithm II (NSGA-II) (Kalyanmoy, 2002). It starts by generating an initial population based on feature information. It is followed by iterative processes of calculating multiple objective functions using trained machine learning models, performing non-dominated sorting, and selecting elite populations. The process continues until convergence is reached, leading to optimized and personalized health recommendations, minimizing the possibility of simultaneous attacks of multiple targeted diseases.

Interactive model

With the help of Google Gemini Pro (as in Fig. 1), the interactive model of the proposed IMPACT system can convert user natural language input to SQL by leveraging its advanced natural language understanding and generation capabilities. The system can accurately interpret intent and context by processing a user’s natural language query and then generating the corresponding SQL query to retrieve the desired data from a database. This lets users interact with databases using natural language, simplifying data access and analysis.

Initially, when a user requests the interface through natural language input, it is processed by the Google Gemini Pro. A second input is required from the prompt, which commands the conversion of the user’s natural language input to SQL. The specific command varies based on the requirement. For instance, if the user intends to retrieve information from the database (in this case, Sqlite3 DB), the command will be converted to a select query. This allows the user to obtain feature details and their values, which are then returned as a response from the model’s interface. In other scenarios, when the user provides feature values, the information is returned as a result of the user data and processed for further operations. The interactive model in Fig. 1 illustrates these interactions between the user and the model.

Similarly, the user interacts with the model using natural language through the interface and gets responses or shares the user data as feature information to the multi-dimensional counterfactual model as it is defined in Algorithm 1. This multi-dimensional model generates the result in the form of personalized health recommendations. Finally, this recommendation is provided to the user to prevent the simultaneous attack of multiple diseases.

Multi-dimensional counterfactual model

In Explainable AI for disease prediction, counterfactual explanations fall short when dealing with multiple diseases simultaneously, as they typically focus on a single disease model. A multi-dimensional counterfactual model that integrates various disease prediction models ( ${\mathcal{M}}_i$ for a user $U_i$ as in Eq. (5)) is crucial for reducing the risk of concurrent disease outbreaks. NSGA-II, a multi-objective optimization algorithm, is utilized in the proposed multi-dimensional counterfactual model as in Algorithm 2 of the IMPACT system to minimize the simultaneous occurrence of multiple diseases. The optimization framework defines the objective functions as the predicted probabilities of heart stroke ( $P_{HS}$) and diabetes ( $P_{DM}$), aiming to reduce both concurrently. To balance these competing objectives, a penalty function is introduced to discourage counterfactual solutions that significantly improve one health condition while worsening another. And it is presented in the next section.

As there is $n$ number of diseases, the possibility of an attack of each disease is found from the outcome of each prediction model as $M_i(X_i)$ where $i=1,2,3,\dots ,n$ and $X_i$ represent a set of feature variables for each disease. It is possible that a set of feature variables for one disease may or may not be the same as another, and there are some common features between these sets.

(6) $$X_i\ne X_{j}wherei\ne jandi,j=\{1,2,3,\dots ,n\}$$

(7) $$X_i\cap X_j\ne \mathrm{\Phi }wherei\ne jandi,j=\{1,2,3,\dots ,n\}.$$

To act on all the prediction models $M_i(X_i)$ simultaneously, it is necessary to make the union of all feature sets of all disease data sets. It is defined in Eq. (8).

(8) $$X_{\mathrm{\Delta }}=\{X_1\cup X_2\cup X_3\cup \dots \cup X_n\}.$$

For each feature $x_{\delta }$ (where $x_{\delta }\in X_{\mathrm{\Delta }}$), the boundary value can be defined as lower bound (LB) and upper bound (UB). The value of the feature ( $val(x_{\delta })$) can be in between this boundary. It can be represented as in Eq. (9).

(9) $$L{B}_{\delta }\le val(x_{\delta })\le U{B}_{\delta }where{x}_{\delta }\in X_{\mathrm{\Delta }}and\delta =\{1,2,3,\dots ,k\}wherek=|X_{\mathrm{\Delta }}|.$$

Like the traditional counterfactual explanation, the user can also provide a list of feasible and infeasible features from the set of all features. For the infeasible features, the user can fix them to constant values ( $C_{\delta }$).

(10) $$val(x_{\delta })=L{B}_{\delta }=U{B}_{\delta }=C_{\delta }where\delta =\{1,2,3,\dots ,k\}.$$

The user can also provide a boundary according to their choice for the feasible features.

(11) $$L{B}_{\delta }^{\mathrm{\prime }}\le val(x_{\delta })\le U{B}_{\delta }^{\mathrm{\prime }}where\delta =\{1,2,3,\dots ,k\}.$$

A random population of potential solutions (individuals) is initialized. After combining all the feature sets, the random population is generated by considering the boundary values of each feature. The randomly generated population can be defined as $P_0(X_{\mathrm{\Delta }})$.

NSGA-II starts by generating a random initial set of solutions evaluated using multiple objective functions for different target diseases ( $M_i(X_i)$ for $i=1,2,3,\dots ,n$). An offspring population is created using non-dominated sorting and crowding distance selection, ensuring genetic diversity through crossover and mutation. The parent and offspring populations are combined into a new population $R_0=P_0\cup Q_0$. Based on rank and crowding distance, the best solutions form the new parent set $P_1$ for the next iteration. NSGA-II iteratively refines this process to converge on the Pareto-optimal front, providing a set of non-dominated solutions that balance multiple objectives. The final output is a set of recommended feature values to minimize the risk of multiple diseases, making this model crucial for personalized disease prevention.

Pre-processing and modeling data sets

This proposed IMPACT system generates actionable, personalized feature values to significantly reduce the risk of simultaneous attacks from critical chronic conditions like diabetes and heart stroke. Focusing on these diseases and analyzing data from widely recognized and available in public repositories (Soriano, 2019; UCI Machine Learning Repository, 2024).

The stroke and diabetes datasets, encompassing 5,110 and 768 instances, respectively, offer a comprehensive foundation for analysis. Each instance includes distinct attributes, designated as “stroke” and “outcome” to signify their primary outcome features. A rigorous data cleaning process revealed 201 null entries in the ‘BMI’ column of the stroke dataset, which were replaced with the median value to maintain accuracy. The datasets were systematically organized into dependent and independent features, with ‘stroke’ and ‘outcome’ identified as the independent features for their respective datasets. To enhance data usability, five categorical features in the stroke dataset namely, ‘gender’, ‘ever_married’, ‘work_type’, ‘Residence_type’, and ‘smoking_status’ were effectively transformed through one-hot encoding. The ‘id’ feature, deemed non-essential, was excluded from the analysis. Details of the feature sets are presented in Table 1. Among these datasets, three common features were identified between Stroke and diabetes datasets, namely, ‘age’, ‘avg_glucose_level’, and ‘BMI’. Traditional counterfactual explanations face a challenge with these common features as recommendations to mitigate the risk of one disease. For example, diabetes might inadvertently increase the risk of stroke. Therefore, the development of a multi-dimensional counterfactual model is imperative. Such a model would minimize the likelihood of being afflicted by both diabetes and stroke simultaneously by recommending changes in both these common features and other relevant feature values to the patient.

Evaluation method

Tables 2 and 3, present the performance metrics of various machine learning models on stroke and diabetes datasets, respectively. Key metrics evaluated include mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), $R^2$ score, ROC AUC score, and accuracy. Among the models, XGBoost achieved the highest ROC AUC score for the stroke dataset, while LightGBM performed best for the diabetes dataset, indicating strong predictive capability across both datasets based on the ROC AUC score criterion.

Selection method

Both datasets were divided into training and test sets, maintaining an 80:20 ratio. This division facilitated training in various advanced machine learning models, including logistic regression, gradient boosting, XGBoost, AdaBoost, CatBoost, LightGBM, and support vector machine. Through rigorous training, XGBoost and LightGBM emerged as the standout models for the stroke and diabetes datasets. Their superior performance, indicated by the highest accuracy and ROC AUC scores, highlighted them as prime candidates for inclusion in this multi-objective problem-solving framework.

User interaction

After getting the models for each disease, the proposed IMPACT system will work by making simple natural language interaction between the user and the interface. The system facilitates interaction between the interface and users, as described in the sequence diagram depicted in Fig. 3. Initially, the user requests the interface to display all features using simple and natural language such as English. The interface interprets this request, which converts the user’s input into SQL code through a prompt command. The system processes the inputs, which generates an SQL query. This query is executed in an SQLite3 database, and the retrieved information is sent back to the user via the interface. Subsequently, the interface prompts the user to select a feature to be held constant. The interface then displays the feature values by querying the database, and the user provides the desired constant values. These values are stored in the feature information. The user also inputs values for features with continuous values, which are similarly stored. The user specifies the number of counterfactuals needed, and the system generates the corresponding feature recommendations. These recommendations are derived from a multi-dimensional counterfactual model that inputs feature information and disease models.

Computing infrastructure

All experiments were conducted on a personal computer running Windows 11 Education. The hardware included an Intel(R) Core(TM) i7-10700 CPU @ 2.90 GHz with 8.00 GB of RAM. The implementation was performed using Python 3.11.9 within a Conda environment (version 24.7.1). The core libraries and tools used for the experiments include NumPy 1.26.4, Pandas 2.2.1, Scikit-learn 1.5.1, and Matplotlib 3.8.3. SQLite3 was used for database management, while Google Gemini Pro 1.5 served as the generative model. For cloud deployment, the application was hosted using Streamlit v1.38.0. This setup facilitated both local experimentation and cloud-based system accessibility.

Reproducibility and README documentation

To reproduce this work, acquire the datasets in the “Pre-processing and Modeling Data Sets” subsection and pre-process them as described. Once the data is prepared, proceed with model training and evaluation following the steps in the “Evaluation Method” subsection, then select the top-performing models for each disease, as detailed in the “Selection Method” subsection. Using NSGA-II, generate multi-dimensional counterfactual explanations to derive Pareto-optimal feature values for targeted diseases, following Algorithm 2, which can serve as health recommendations. The proposed IMPACT system facilitates user interaction through Google Gemini Pro 1.5, as illustrated in Algorithm 1 and in the sequence diagram (Fig. 3). The entire codebase is shared on Zenodo (https://doi.org/10.5281/zenodo.14891893) and GitHub (https://github.com/iprasantmohanty/presonalized-health), and deployment to the cloud can be easily achieved using Streamlit (https://streamlit.io/). With the aforementioned computing resources, this process enables full reproducibility of the work.

Results obtained

Personalized health recommendations will be generated by the proposed IMPACT system and provided to users seeking simultaneous prevention from all targeted diseases. Flexibility is afforded to users to make adjustments to specific features within specified ranges according to their requirements or to opt out of modifying any features. The system will also offer feature recommendations in scenarios where all features are left open to modification within their default boundary values. However, these results are not practically useful.

Features such as age and gender, typically unchangeable for a user, are referred to as non-feasible features for the user and are kept constant by the user. However, features such as hypertension, heart disease, work type, residence type, and smoking status may be identified as feasible for users to modify to minimize the possibility of diseases. For features that are not feasible for change, including age, gender, and marital status, constant values are requested from the user. In other scenarios, health information such as blood pressure and oxygen levels is obtained from wearable IoT devices. For features with continuous values, the recommendation system facilitates the selection of value boundaries, set to $\pm 10\mathrm{\%}$ of the feature value chosen by the user. This functionality mirrors what is provided in traditional counterfactual explanations. Similarly, other continuous feature values can be offered based on the boundary values.

In this proposed system, the results are generated for a user with the following fixed features: age 32, male gender, married status set to no, and work type private. The probabilities of heart stroke and diabetes attacks for various combinations of population sizes, maximum number of generations, and each counterfactual (CF) are provided in Table 4. Similarly, the user can select any other set of continuous features, and the boundary values will be established according to it. Considering these boundaries of the selected continuous features, the model will recommend feature values. In this system, the user chooses a continuous feature, namely ‘BMI’ of 20.10; its boundaries are set to $\pm 10\mathrm{\%}$, i.e., 10.3 to 29.9. Two different recommendations are generated for each combination of population and generation size as the chosen number of counterfactuals is two. However, the user can also select any recommendations by choosing the positive number of counterfactuals in the interface.

To ensure a balanced reduction in disease risks, a penalty function is formulated as follows:

(12) $$\mathcal{L}(\mathbf{x})=w_1{P}_{HS}(\mathbf{x})+w_2{P}_{DM}(\mathbf{x})+\lambda max(0,P_{HS}(\mathbf{x})-P_{DM}(\mathbf{x}))$$where $w_1$ and $w_2$ are equal weights (set to 1) ensuring balanced optimization, and $\lambda =1$ penalizes cases where minimizing one disease risk leads to a disproportionate increase in another. The penalty values are computed based on the probabilities provided in Table 4.

For example: For a population size of 500, with 20 generations (CF #1)

(13) $$\begin{array}{rl}\mathcal{L}(\mathbf{x})=& 0.00001095+0.00025517+max(0,0.00001095-0.00025517)\\ =& \mathrm{0.00026612.}\end{array}$$

For a population size of 2,000, with 40 generations (CF #2):

(14) $$\begin{array}{rl}\mathcal{L}(\mathbf{x})=& 0.00000876+0.00010403+max(0,0.00000876-0.00010403)\\ =& \mathrm{0.00011279.}\end{array}$$

These computed values demonstrate that the optimization framework effectively balances the risk reduction of multiple diseases while avoiding extreme trade-offs. This penalty mechanism ensures that counterfactual recommendations remain clinically meaningful and beneficial for multi-disease prevention.

The feature values recommended to minimize these diseases are shown in Table 5. These results are produced when the user opts for two recommendations. Any other set of feasible features can be chosen, while the user will provide constant values for the remaining infeasible features. The system will provide recommended feature values for the possible features.

Figure 4 presents the execution time in seconds required to generate each recommendation as counterfactuals (CF#1 and CF#2) for various combinations of population size and maximum generations.

Figure 5 illustrates the Pareto fronts for both diseases in each recommendation. This is achieved by increasing population size and increasing the number of generations. Here, one of the two counterfactual results is plotted. The figures in 6 include six separate graphs depicting how the disease probabilities of the optimal solution decrease and approach near zero as the number of generations increases in each scenario.

Analysis and discussions

Table 4 depicts that, by varying combinations of population and maximum generations, probabilities for heart stroke and diabetes were reduced to 0.00000876 and 0.00010403, respectively. The user can choose any of the two recommendations of a specific combination obtained in Table 5 to get the minimum possibility of diseases. For example, the system suggests particular feature values, such as age 32, an average glucose level of 93.174, and a BMI of 20.68197, based on a population size of 500 and a maximum generation of 40. Figure 4 shows that execution time increases with larger population sizes and generations.

An ablation study is conducted to further understand the contribution of different components in our proposed system, and the results are summarized in Table 6. The table presents multiple experimental setups where key components such as SQL queries, dataset size, and machine learning model choices are modified to assess their impact. The results highlight that the full system configuration (baseline) performs best with the highest accuracy and AUC scores while maintaining the lowest attack probabilities for stroke and diabetes. These findings confirm the necessity of the selected system components in achieving optimal results.

Another ablation study was conducted to validate further the contributions of different components in our proposed model. Table 7 presents the impact of excluding SQL queries, reducing dataset sizes, and varying machine learning models from Table 6. The table also includes standard deviations ( $\sigma$) to reflect performance variability across multiple runs and p-values to indicate the statistical significance of differences. The p-values were obtained using a two-sample t-test, comparing each ablation setting to the baseline. The results demonstrate that removing key components significantly affects performance, confirming the need for each module in our framework.

Justification

Pareto fronts represent the probability of attacks by these diseases, which minimizes both simultaneously and are plotted in Fig. 5. Each point signifies an individual solution to this multi-objective problem, with the green point identified as the optimal solution that reduces the attack probabilities of both diseases. The aim is to generate the set of feature values a user can adopt to achieve this optimal solution. It can be observed in Fig. 6 that the best solutions for disease probability fluctuation become minimal after 30–40 generations for most of the population size. Also, their execution time increases as shown in Fig. 4. It is also observed that there is no further significant decrease in the probability of diseases from the population size of 1,000 to 2,000, as shown in Table 4. Hence, in this experiment, recommendations are generated till population size 2,000, and a maximum generation of 40.

Limitations/validity

This work only generates recommendations for two diseases, and the user cannot choose more. However, future research could expand this work by incorporating more diseases to minimize attack probabilities further, highlighting its potential for related applications.