Enhancing machine learning-based forecasting of chronic renal disease with explainable AI

Dataset

In our work, we utilize the CRD dataset, readily available from the open data repository curated by the University of California, Irvine (UCI) (Rubini & Eswaran, 2015). There are 400 instances in this dataset, 24 attributes, and 1 target class attribute. The first attribute is an ID and has been removed from the dataset since it is not required for the model. The other 23 features include demographic and medical measurements like age, blood pressure, specific gravity, albumin, sugar, red blood cells, pus cells, bacteria, blood glucose, blood urea, serum creatinine, sodium, potassium, haemoglobin, packed cell volume, white blood cell count, red blood cell count, hypertension, diabetes mellitus, coronary artery disease, appetite, pedal edema, and anaemia. Each feature’s type varies, including integer, continuous, categorical, and binary, with some missing values. The target variable, ‘class’ indicates the presence of CRD and contains no missing values. The dataset contains 250 cases labeled as ‘yes’ (indicating CRD) and 150 instances labeled as ‘no’ (indicating no CRD). The existence of missing values, a prevalent issue in real-world datasets, poses a serious barrier for this dataset. Random imputation techniques are employed to address this issue. It should be noted that out of the 400 instances in the dataset, only 158 instances contain no missing values. Figure 4 illustrates the attribute correlation heatmap representation, using Pearson correlation to provide insights into the relationships between various attributes.

The characteristics in this dataset are classified into two types: numerical features and categorical features. Appetite, pus cell clumps, pus cells, diabetes mellitus, pedal edema, bacteria, red blood cells, hypertension, coronary artery disease, and anaemia are among the categorical characteristics. These characteristics, combined with the class attribute, determine the presence of CRD. Conversely, the remaining characteristics are categorized as numerical attributes. The class attribures are specifically used to determine whether an individual has CRD or not.

Missing value imputation

A random value imputation technique is used to address missing values for specific variables such as “anaemia”, “appetite”, “bacteria”, “coronary_artery_disease”, “diabetes_mellitus”, “red_blood_cells”, “pus_cells”, “peda_edema”, “pus_cell_clumps”, and “hypertension”. The requirement to manage missing data in these categorical variables motivates the usage of “random value imputation” in the CRD prediction study. Here, for a feature X with missing values, let N be the total number of missing values and X_observed be the set of observed (non-missing) values in feature X. Now, randomly sample N values from X_observed, resulting in N random values, denoted as R_i, where i ranges from 1 to N. Assign these random values to the corresponding missing values in feature X. (1)${\displaystyle X_{missing}=RandomSample\left(X_{observed},N\right).}$ In summary, the random value imputation method is essential for dealing with missing data. It helps ensure that the CRD prediction model is built using as much complete and representative data as possible, particularly for categorical characteristics that play a role in the analysis.

 
_______________________ 
Algorithm 1 Proposed Method for CRD Prediction__________________________________ 
Input: CRD dataset 
Output: Predicted class label 
Begin 
Step 1:  Load the Dataset - Load the CRD dataset into a variable, such as 
“crd_data”. 
Step  2:  Handle  Missing  Values  -  Locate  and  handle  missing  data  in  the 
“crd_data” field. 
Step 3: Textual Data Encoding - Convert category text data in “crd_data” to 
numerical format. 
Step 4: Split and Scale the Data - Split the “crd_data” variable into subgroups 
for training and testing. If required, scale the data. 
Step 5: Hyperparameter tuning - Adjust the machine learning models’ hyper- 
parameters. 
Step 6: Apply ML Models - Train and test the machine learning models. 
Step 7: Determine Validation Scores - Evaluate model performance using val- 
idation scores. 
Step 8:  Interpreting ML models - Use SHAP and LIME explainability and 
interpretability to understand model decisions. 
End_____________________________________________________________________________________    

Feature encoding

Once the missing values have been addressed, the dataset undergoes a transformation process where textual data is converted into a numerical format. This transformation is achieved using a feature encoding method called as“LabelEncoder”.

The LabelEncoder method serves the following purposes:

• Conversion of categorical to numeric: It is utilized to change categorical features, which originally contained text-based categories into a numerical representation. Each category received a distinct numeric code.

• Preservation of ordinal information: LabelEncoder maintains the relative order of the categories. This ensures that the assigned numeric codes reflect the original order or ranking of the categories. For instance, if the categories have a meaningful order like “low”, “medium”, and “high”, LabelEncoder retains this order when converting them into corresponding numeric codes (e.g., 0, 1, and 2).

The specific features that undergo this transformation include “pus_cell_clumps”, “red_blood_cells”, and “pus_cell”, which are transformed into two categories. Additionally, “anaemia”, “appetite”, “bacteria”, “diabetes_mellitus”, “hypertension”, “coronary_artery_disease”, and “peda_edema” are also transformed. The utilization of the LabelEncoder technique is crucial for converting this specific kind of data into numerical representations. Regarding studies on CRD, this improvement facilitates further data analysis, modeling, and forecasting operations. A numerical representation is being established for each of these attributes, simplifying data processing and enabling their application to a diverse variety of data-driven activities.

Splitting and scaling the data

Out of the 400 instances, the models are trained on seventy percent of the dataset (280 instances) and their performance is assessed on the remaining thirty percent (120 instances). In the training set, there are 180 ‘yes’ cases and 100 ‘no’ cases, while in the testing set, there are 70 ‘yes’ cases and 50 ‘no’ cases. The Standardisation and min-max scaling are two essential techniques that ensure the data features are centered around their respective means and maintain consistent scales. To create reliable and accurate projections, it is essential to engage in extensive information preparation. The MinMax Scaler is a data transformation technique that scales attributes to a specified range, often between 0 and 1. It maintains the arrangement of the initial distribution while modifying the data to a certain extent. (2)${\displaystyle A=\frac{a-b}{c-b}}$

where: A - is the scaled/normalized value of the feature. a - is the feature’s initial value. b - is the smallest score of the feature in the dataset. c - is the feature with the highest score in the dataset.

Using this formula, MinMax Scaler rescales each data point within the range [0, 1] depending on the dataset’s minimum and maximum values. It is frequently used to standardize characteristics and scale them to make them appropriate for machine learning techniques.

Standard scalar: StandardScaler follows the Standard Normal Distribution (SND), adjusting the data to have a unit variance and setting the mean to 0. This transformation is crucial for various machine learning algorithms as it helps to achieve better convergence and performance. (3)${\displaystyle \mathbf{Standardization\; :}Z=\frac{x-\mu }{\sigma }.}$

Here, x represents the original data values, μ denotes the mean, and σ represents the standard deviation. The Standardization equation transforms the data values (x) into the standardized form (Z) with a mean of 0 and a standard deviation of 1.

Handling class imbalance

Within the course of our study on CRD prediction, we discovered a significant class imbalance in which CRD patients are significantly underrepresented relative to non-CRD cases. Recognizing the possible negative impact of this imbalance on our models’ predictive efficiency, we used the SMOTE, Synthetic Minority Over-sampling Technique proposed by Chawla et al. (2011) as a resampling approach, which is commonly utilised in medical contexts. By synthetically boosting instances of the minority class, SMOTE effectively alleviated the class imbalance, enabling a more balanced distribution for training purposes. This accurate method is designed to prevent the models from showing a bias towards the majority group, ensuring that they are capable of recognizing patterns across both CRD-positive and CRD-negative situations. By assuring a balanced representation of positive and negative instances, this method improves model predictability. Figure 5 illustrates how the data imbalance has been handled in the current process, resulting in more robust and unbiased predictions. After applying SMOTE to the training data, the class distribution was adjusted to correct class imbalances. Initially, the training data consisted of 280 instances, with 180 labelled as ‘yes’ (showing the presence of CKD) and 100 labelled as ‘no’ (indicating the absence of CKD). Following SMOTE, the data was resampled to ensure an equal number of cases in both classes. The training dataset now has 180 cases labelled ‘yes’ and 180 instances labelled ‘no’, resulting in a balanced dataset of 360 occurrences.

Hyper parameter tuning

The study examined several distinct facets of machine learning algorithms, with the primary goal of improving their performance through the difficult process of hyperparameter tweaking. This procedure is vital for fine-tuning the critical settings of these models to attain the best prediction accuracy. A parameter grid is methodically created to explore this hyperparameter field systematically. A variety of hyperparameters are included, including the number of estimators (trees), the maximum depth of these trees, and the minimum sample required for node splitting.

GridSearchCV (Pirjatullah et al., 2021), (Gill & Gupta, 2023) is a strategy that rigorously investigates numerous hyperparameter arrangements to discover the optimum set for a machine learning model. It comprises generating a set of hyperparameter values as a grid, testing all possible combinations, evaluating model performance, and picking the best one, therefore assisting in model optimization. Figure 6 depicts how GridSearchCV hyperparameter tuning works. The parameter grid and GridSearchCV best outcome for all the models are tabulated in Table 2.

ANN

An artificial neural network (ANN) with an input layer, two hidden layers incorporating the rectified linear unit (ReLU) activation function, and an output layer utilizing the sigmoid activation function for binary predictions is used in the context of binary classification for chronic renal disease (CRD). The network design is subjected to a grid search for hyperparameter tuning, which involves experimenting with various combinations of hidden layer sizes, activation functions (ReLU, tanh), solvers (Adam, SGD), regularisation strengths (alpha), and starting learning rates.

The Adam optimizer is used during model training to repeatedly change the model parameters. The grid search tests multiple hyperparameter combinations systematically using 5-fold cross-validation, to discover the configuration with the highest predictive performance.

The grid search discovers the most effective design by examining several sets of hidden layer sizes, including (64,), (32,), (128, 64), and (64, 32, 16). The chosen model obtains a outstanding test accuracy of 99.0%, suggesting its capacity to properly detect and forecast the existence of CRD.

Extra tree classifier

The Extremely Randomized Trees also known as extra trees classifier (Geurts, Ernst & Wehenkel, 2006), which is highly skilled in group learning, is applied carefully. The GridSearchCV technique (Pirjatullah et al., 2021) is used with great care in this implementation, methodically exploring a large range of hyperparameter combinations. To prevent overfitting during this exhaustive process, deliberate adjustments are made to several critical parameters. These adjustments include determining the minimum sample size required to initiate splitting in a tree node, limiting the maximum depth of individual trees, and specifying the total number of trees within the classifier’s “forest”. Additionally, balanced data is consistently utilized throughout our analysis.

The principal objective of this systematic investigation is to identify the ideal hyperparameter combination that would optimize the model and allow it to ascertain the presence of CRD, with the highest possible degree of accuracy. The results of this thorough investigation are extremely impressive. An astounding accuracy of 99.04% on the test dataset is achieved by using the grid search to identify the ideal hyperparameter configuration. With its ability to capture the nuances of the dataset and produce incredibly accurate predictions, this outstanding accomplishment demonstrates the extra trees classifier’s potential as a potent tool in CRD prediction.

Logistic regression

Logistic regression (LR) is a fundamental linear modelling technique in predictive analytics. Critical variables such as solver, penalty, and regularisation strength were thoroughly examined using the GridSearchCV approach.

The solver parameter defines the optimisation strategy used in logistic regression, whilst the penalty determines the type of regularisation used. The emphasis was mostly on L2 regularisation, a variation of the elastic net technique that supplements the algorithm’s loss function to prevent overfitting and encourage a well-generalized model.

The LR model obtained 99.07% accuracy on the experimental dataset by carefully tuning hyperparameters. This outstanding accuracy puts LR on par with other top-performing models, reaffirming its efficacy in predicting CRD.

Furthermore, the stability of LR’s higher performance solidifies its position as the preferable model for CRD prediction. Its consistent supply of accurate findings, which outperform competing models, demonstrates its applicability for real-world applications. As a result, LR was chosen as the foundation for building the entire programme, offering trustworthy predictions that are critical for influencing clinical decision-making and improving patient outcomes. The mathematical intuition is as follows: (4)${\displaystyle \text{Loss}\left(w\right)=-\sum _{i=1}^N\left[a_{i}log\left(p\left(a_i\mid x_i,w\right)\right)+\left(1-a_i\right)log\left(1-p\left(a_i\mid x_i,w\right)\right)\right]+\lambda \parallel w{\parallel }^2}$

Loss(w) is the logistic regression loss function with L2 regularization.

• N is the overall amount of observations.

• a_i is the actual binary label for data point i.

• x_i is the data point (i) feature vector.

• w is the vector of model parameters (weights).

• p(y_i − x_i, w) is the probability of the positive class predicted by the model for data point i.

• lambda(λ) is the regularization strength, determining the influence of the L2 penalty on the weight of the model.

• ∥w∥² reflects the weight vector w’s L2 norm (Euclidean norm).

L2 term λ||w||² has been added to the standard logistic regression loss function to prevent overfitting. It penalizes large values of the model’s weights (w), effectively encouraging smaller, more balanced weight values. This helps in producing a more robust and generalized logistic regression model.

 
__________________________________________________________________________________________ 
Algorithm 2 ML Model Training using GridSearchCV Hyperparameter Tuning 
  1:  Input:    Sampled  data  Xtrain_resampled,   Xtest_resampled,   ytrain_resampled, 
     ytest_resampled = resample_data(Xtrain,ytrain) 
  2:  Create a scaling pipeline: 
  3:  Xtrain_scaled,Xtest_scaled = scale_data(Xtrain_resampled,Xtest_resampled) 
  4:  Define hyperparameter grid for grid search 
  5:  Initialize best accuracy variable, best_test_acc = 0 
  6:  Iterate over hyperparameter grid 
  7:  for each hyperparameters in hyperparameter_grid do 
 8:       Build ML model with specified hyperparameters 
  9:       model = build_ml_model(hyperparameters) 
10:       Train the model 
11:       model.fit(Xtrain_scaled, ytrain_resampled) 
12:       Evaluate on the test set 
13:       test_acc = model.evaluate(Xtest_scaled, ytest_resampled) 
14:       Update best accuracy and best model if the current model is better 
15:       if test_acc > best_test_acc then 
16:            best_test_acc ← test_acc 
17:            best_model ← model 
18:       end if 
19:  end for 
20:  Display results 
21:  display_results(best_model, best_test_accuracy)_____________________________________    

Random forest

In this research, we used the RandomForest classifier because of its shown success in creating highly accurate predictions by using the collective knowledge of decision trees. This methodology builds on our previous usage of the extra trees classifier, emphasizing the importance of ensemble learning approaches in solving complicated challenges like predicting CRD. To improve the model’s prediction performance and also to remove overfitting, we tested several parameters in our RandomForest implementation. Bootstrap, Minimum Samples Split, Minimum Samples Leaf, Maximum Features, Maximum Depth, and number of estimators are among the factors used. Each hyperparameter influenced many parts of the model’s behavior, ranging from resampling strategies to tree depth and branching. The comprehensive hyperparameter tuning efforts attempted to establish the optimal configuration for fully realizing the model’s potential in CRD prediction. Notably, these efforts are rewarded, as the RandomForest classifier attained an exceptional accuracy value of 99.05% on the test dataset. This outstanding accomplishment illustrates the model’s exceptional capacity to generate extremely accurate forecasts and solidifies its expertise in the field of CRD prediction.

LGBM

The study used the LightGBM classifier, a gradient-based learning algorithm known for its effectiveness in predictive modelling (Ke et al., 2017). The model obtained outstanding predicted accuracy of 98.15% on the test dataset by meticulously optimising crucial hyperparameters such as learning rate, maximum tree depth, and boosting rounds using GridSearchCV. This achievement highlights LightGBM’s robustness in CRD prediction, proving its potential to produce exact prognostications. Furthermore, LightGBM’s approach to minimising an objective function that includes both prediction errors and model complexity guarantees a balanced and reliable framework for CRD prediction problems, making a substantial contribution to the growth of predictive analytics in healthcare. Indeed, the description presented provides a concise synopsis of the LightGBM algorithm’s fundamental mechanisms. At its core, LightGBM seeks to minimise an objective function that consists of two key components: the loss term and the regularisation term.

1. Loss term: This component assesses the prediction error for each data point and works to reduce these mistakes collectively. The loss term helps to refine model predictions by efficiently capturing the differences between projected and actual values, thus boosting overall accuracy.

2. Regularisation term: The regularisation term is designed to reduce model complexity and prevent overfitting. This term ensures that the model generalises successfully to unknown data by setting constraints on its structure and parameters, hence increasing its dependability and resilience. (5)${\displaystyle Objective=\sum L\left(y,\hat{y}\right)+\sum \Omega \left(g\right)}$

where:

L(y, ŷ) is the loss term measuring prediction errors.

Ω(g) is the regularization term penalizing complex decision trees.

LightGBM employs gradient descent to reduce this function and increase model accuracy. To summarise, it combines effective approaches with gradient boosting to obtain accurate prediction.

Decision tree

In the decision tree classifier, meticulous hyperparameter tuning using GridSearchCV was employed on resampled data to enhance model performance. While the decision tree exhibited notable efficacy, achieving an accuracy of 97.22%, its performance slightly trailed behind other advanced models utilized in our study. This detailed investigation highlights the complexities of CRD prediction, needing a careful examination of numerous algorithms and configurations to determine their respective strengths and limits. Despite its slightly poorer performance, the decision tree’s interpretability and ability to capture subtle relationships within the data provided vital insights into our overall goal of developing a robust CRD prediction model.

Recall

True positive rate (TPR) and sensitivity are other names for recall (R). By dividing the total number of true positives by the number of true positive and false negative predictions, it is calculated. It is crucial to recognize the strong correlation between the number of positive samples detected and the recall measure. (6)${\displaystyle R=\frac{\text{True Positives}}{\text{True Positives + False Negatives}}.}$

An ML model’s recall is termed poor when the total amount of true positives (TP) and false negatives (FN) in the denominator exceeds the TP in the numerator. When the TP in the numerator surpasses the TP+FN in the denominator, the model’s recall is deemed high. This implies that the number of FNs should be low for having a higher recall.

TN: Instances correctly classified as negative by the model. TP: Instances correctly classified as positive by the model. FN: Instances incorrectly classified as negative by the model. False Positives (FP): Instances incorrectly classified as positive by the model. If an instance is classified as negative, it means no kidney disease is identified from the provided attribute information. If classified as positive, it means kidney disease is identified.

Specificity

True negative rate (TNR), another name for specificity, is obtained by dividing the number of true negative predictions by the sum of true negatives and false positives. The number of FPs should be low for having a higher specificity. (7)${\displaystyle S=\frac{\text{True Negatives}}{\text{True Negatives + False Positives}}.}$

Precision

Precision (P) is characterized as the ratio of actual positive samples rightly identified among all samples classified as positive by the model. (8)${\displaystyle P=\frac{\text{True Positives}}{\text{True Positive + False Positives}}.}$

The precision of an ML model will be high when the value of TP (numerator) ≥ TP+FP (denominator) and it will be low if TP+FP (denominator) ≥ TP (numerator). This implies that the number of FPs should be low for having a higher precision.

F1-score

The F1-score is an evaluation statistic that is used to assess the efficacy of the ML model. It utilizes both precision and recall, making it appropriate for datasets with imbalances. (9)${\displaystyle F1-score=2\cdot \frac{\text{P * R}}{\text{P + R}}.}$

The F1-score, which ranges from 0 to 1, represents the performance of the model, with 0 being the worst situation and 1 indicating the most favorable case. By offering a weighted average of precision and recall, this score compensates for both erroneous positives and false negatives.

The F1-score will be high if both precision and recall exhibit elevated values. In contrast, if both precision and recall are poor, the F1-score will be minimal. When one of these indicators is low and the other is high, the F1-score will be in the medium range.

Accuracy

In the field of AI and ML, we evaluate a model’s performance using accuracy (A), which is commonly stated as a percentage. It measures the proportion of correct predictions to total predictions, providing a measure of the model’s accuracy. (10)${\displaystyle A=\frac{\text{Number of Correct Predictions}}{\text{Total Number of Predictions}}.}$