A systematic hybrid machine learning approach for stress prediction

Dataset description

In our proposed approach first, we acquired the stress detection dataset from Kaggle (https://www.kaggle.com/code/ingridai/stress-detection-prediction/data). The dataset consists of the five target classes such as low/normal, medium-low, medium, medium-high, and high. The collected dataset by Rachakonda et al. (2020, 2018) consists of the following parameters such as snoring range of the user, body temperature, respiration rate, limb movement rate, eye movement, blood oxygen levels, heart rate, number of hours of sleep, and stress level. Figure 2 shows the range of the parameter values.

The used dataset is highly balanced as the dataset contains an equal number of each class. The description of the dataset variable is shown in Tables 2 and 3 showing the sample of the dataset.

The stress dataset consists of the eight features for the training of machine learning models as shown in Table 2. To check that, dataset features are correlated to the target class or that some features are not important to predict the stress level we deployed and calculate the feature importance. We find the feature importance using the extra tree classifier (Gupta, 2020) and the feature importance graph is shown in Fig. 3. All features present in the dataset are important for stress prediction so we used all features in the training and testing set as all features are equally important.

Data splitting

This section contains a discussion about data splitting. We split the dataset with an 80:20 splitting ratio because the dataset size is small and we use most of the data for training purposes. While the remaining 20% of the data we used for the testing purpose. The dataset consist of a total of 630 records and 126 belong to each class. The ratio of training and testing set with respect to each target class is shown in Table 4.

Machine learning models

We deployed several machine learning models in comparison with each other and selected the best in terms of performance to make a hybrid model. We used three tree-based models such as RF, GBM, and two linear models LR, and SVM. We tuned these models with their best hyper-parameters setting to achieve their best results. We tuned models between specific ranges using the grid search method and hyper-parameter setting of machine learning models is shown in Table 5.

LR

LR is a statistical model used for the classification of data. LR find the relationship between dependent and independent variables (Rustam et al., 2020a). LR is used to perform computation for the probability of outcomes given an input variable. LR used the black box function Softmax to find the probabilities basis on the relationship between input variables and outcomes. Softmax function can be defined as:

(1) $$\sigma (z{)}_i={\displaystyle \frac{e_i^z}{{\sum }_{u=1}^v{e}_j^z}}$$

Here, $\sigma$ is softmax, $\overrightarrow{z}$ is input vector, $e^{z_i}$ input vector standard exponential function $v$ is the number of target classes, $e^{z_j}$ is output vector standard exponential function $e^{z_j}$ output vector standard exponential function.

RF

RF is the tree-based model used for classification and regression. It is an ensemble model which combined the number of decision trees under majority voting criteria (Rustam et al., 2021). Decision trees work as weak learners in RF and then predictions from these weak learners will be combined for voting and the target class with more number predictions will be the final prediction by RF. We can define RF mathematically as:

(2) $$RF=mode\sum _{n=1}^{N}Tre{e}_i$$

Here, N is the number of decision trees in RF. We used RF with 200 n_estimators in the hyper-parameters setting which means that 200 decision trees will participate in the prediction procedure.

GBM

GBM is also an ensemble model used for regression and classification tasks (Natekin & Knoll, 2013). GBM also combined weak learners such as decision trees to make prediction processes significant. Unlike RF, GBM use boosting method, and each tree will combine sequentially. Error rate from first weak learners will be transferred to second and then further so to reduce it and get optimal training. We used GBM with 200 decision trees which means that 200 decision trees will participate in the prediction procedure and each tree is restricted to max 50 level depth using max_depth hyper-parameters which will help to reduce the complexity in learning models.

ADA

ADA is a tree-based model similar to GBM because it also used boosting method to enhance the performance (Owusu, Zhan & Mao, 2014). We used decision trees as weak learners with ADA and trained them with 0.2 learning_rate. We used 200 decision trees under boosting methodology with 50 level depth of each tree to reduce complexity.

SVM

A supervised machine learning algorithm that is used both for classification and regression (Rupapara et al., 2021). SVM draws a hyperplane to classify the data into their corresponding classes. It works well depending on how much data is linearly separable. It put the feature space in n dimension and then draws multiple hyperplanes with calculation and selects the best hyperplane which separates the target classes with the best margin. We used SVM with linear kernel hyper-parameter because our dataset is linearly separable.

HB

The proposed HB is an ensemble model combination of GB and RF. This model we used for the prediction of stress levels. We used soft voting criteria to combine the machine learning models (Rupapara et al., 2022; Rustam et al., 2020b). The architecture of HB is shown in Fig. 4. We select GB and RF based on their performance individually in terms of accuracy and efficiency.

In Fig. 4, P_0, P_1, P_2, P_3, and P_4 are the prediction probabilities by the GBM and RF for each class. Avg P_0, Avg P_1, Avg P_2, Avg P_3, and Avg P_4 are the average probabilities for each target class while Max_p is the target class with max probability. In the proposed model, GB and RF are used with the same hyper-parameters setting such as 200 decision trees, and 50 max depth as we used in the individual model. The mathematical explanation of HB is given below:

(3) $$G\mathrm{\_}0,G\mathrm{\_}1,G\mathrm{\_}2,G\mathrm{\_}3,G\mathrm{\_}4=GBM(data)$$

Here, in Eq. (3), $G\mathrm{\_}0,G\mathrm{\_}1,G\mathrm{\_}2,G\mathrm{\_}3,G\mathrm{\_}4$ are the probabilities by the GBM for 0, 1, 2, 3, 4 target classes respectively. Similarly, in Eq. (4), $R\mathrm{\_}0,R\mathrm{\_}1,R\mathrm{\_}2,R\mathrm{\_}3,R\mathrm{\_}4$ are the probabilities by the RF for 0, 1, 2, 3, 4 target classes respectively.

(4) $$R\mathrm{\_}0,R\mathrm{\_}1,R\mathrm{\_}2,R\mathrm{\_}3,R\mathrm{\_}4=RF(data)$$

So, the average probabilities for each class will be:

(5) $$AvgP\mathrm{\_}0={\displaystyle \frac{G\mathrm{\_}0+R\mathrm{\_}0}{2}}$$

(6) $$AvgP\mathrm{\_}1={\displaystyle \frac{G\mathrm{\_}1+R\mathrm{\_}1}{2}}$$

(7) $$AvgP\mathrm{\_}2={\displaystyle \frac{G\mathrm{\_}2+R\mathrm{\_}2}{2}}$$

(8) $$AvgP\mathrm{\_}3={\displaystyle \frac{G\mathrm{\_}3+R\mathrm{\_}3}{2}}$$

(9) $$AvgP\mathrm{\_}4={\displaystyle \frac{G\mathrm{\_}4+R\mathrm{\_}4}{2}}$$

Now, these average probabilities will pass through the argmax function to find the target class with maximum probabilities. Here, in Eq. (17) Max_P is the target class with maximum probability which will be the final prediction.

(10) $$Max\mathrm{\_}p=argmax\{AvgP\mathrm{\_}0,AvgP\mathrm{\_}1,AvgP\mathrm{\_}2,AvgP\mathrm{\_}3,AvgP\mathrm{\_}4\}$$

This computation by the model is for a single prediction. As each model is deeply involved in the prediction procedure so if one model will be a little bit wrong the second model will help to regain better results. Let’s dry run this prediction procedure on a dataset example. We take a sample from Table 3 which is 93.8, 25.68, 91.84, 16.6, 89.84, 99.6, 1.84, 74.2.

(11) $$0.1,0.4,0.5,0.8,0.5=GBM(data)and,0.2,0.2,0.3,0.7,0.3=RF(data)$$

(12) $$ForClass0={\displaystyle \frac{0.1+0.2}{2}=0.15}$$

(13) $$ForClass1={\displaystyle \frac{0.4+0.2}{2}=0.3}$$

(14) $$ForClass2={\displaystyle \frac{0.5+0.3}{2}=0.4}$$

(15) $$ForClass3={\displaystyle \frac{0.8+0.7}{2}=0.75}$$

(16) $$ForClass4={\displaystyle \frac{0.5+0.3}{2}=0.4}$$

All prediction probabilities will pass through argmax function to pick the highest and according to the results, 0.75 is the highest probability belonging to class 3 so the prediction is three.

(17) $$Max\mathrm{\_}p=argmax\{0.15,0.3,0.4,0.75,0.4\}$$

Evaluation criteria

This study used four evaluation parameters to measure the performance of learning models and also used a confusion matrix for this purpose. The main evaluation parameter is accuracy which is the total number of correct predictions divided by the total number of predictions to show how much a model is correct.

(18) $$Accuracy={\displaystyle \frac{Totalnumberofcorrectpredictions}{Totalnumberofpredictions}}$$

We also used confusion matrix for the evaluation and confusion matrix consist on TP (true positive), TN (true negative), FP (false positive), and FN (false negative) (Rustam et al., 2020a). We can also find the four evaluation parameters accuracy, precision, recall, and F1 score using these confusion matrix terms such as:

(19) $$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(20) $$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(21) $$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(22) $$F1Score={\displaystyle \frac{Precision\ast Recall}{Precision+Recall}}$$

Results of machine learning models

This section contains the results of machine leanings models such as RF, LR, SVM, GBM, ADA, and proposed HM. The performance of all learning models is good just the ADA is low in accuracy score. Tree-based ensemble models RF and GB outperform with a significant accuracy score of 0.99 accuracies while linear models SVM and LR both achieved 0.95 accuracy scores respectively. The performance of LR and SVM is not good because linear models required a large dataset with a large feature set. ADA is not good in terms of accuracy score and other evaluation parameters because it’s a boosting algorithm that combines the decision trees sequentially so training of the model required a large dataset.

Tables 6–11 showing the results of all models. The proposed model HB outperforms all individual models with a 100% accuracy score. The proposed model HB is significant in comparison to all other used models because we combine the best performer RF and GBM under soft voting criteria. Both individually perform well and give only one wrong prediction by each as shown in Fig. 5. One model weakness is another model strength as RF predicts the medium stress (2) as medium-high stress (3) while the GBM model GBM model predicts medium stress (2) as medium-low stress (1). When we combined both models they make better calculations and resolve the issue of one wrong prediction with 100% correct predictions. The performance of the ADA model is poor for the 0 target class as shown in Table 8. It achieved 0.00 scores in terms of all evaluation parameters for 0 target class. ADA model required some large dataset to get a good fit our used dataset is small in size so its overall performance is poor as well as for 0 target class.

The performance of models is too significant because of the dataset feature correlation with the target class. As shown in Fig. 6, all targets are linearly separable. Figure 6A show feature space using the ‘sr’, ‘rr’, ‘t’ and all target are clearly separable. Figure 6B shows the feature space using the ‘lm’, ‘bo’, ‘rem’ variables are there is a little bit of overlap in the target class with these three variables, and in Fig. 6C we used ‘rem’, ‘srh’, ‘hr’ variables and there is still little overlapping but when we used all feature together than all target are linearly separable and there is a good correlation between features and target classes as shown in Fig. 6D.

Deep learning results

This section contains the results of deep learning models in comparison with used machine learning models. These deep learning models are used with their state-of-the-art architecture taken from previous studies worked on similar kinds of datasets (Rupapara et al., 2022). The architecture of the used models is shown in Table 12. All models start with their embedding layer consisting of vocabulary size 1,000 and output size 100. Each model consists of a dropout layer with a dropout size of 0.5 which helps to reduce complexity in learning models (Reshi et al., 2021). In the end, each model compiles with categorical cross-entropy loss function because of multi-class data and adam optimizer. Models are fitted with a batch size of 8 and 100 epochs.

Tables 13–16 showing the results of deep learning models. According to the results, all deep learning models equally perform well with a 0.99 accuracy score. All deep learning models give wrong predictions for medium stress (2) and medium-low stress (1) target classes. Figure 7 shows the accuracy loss graph for each deep learning model according to per epochs.

Deep learning models are efficient as well as individual machine learning models even on small datasets also but they are not efficient in terms of computational time. Deep learning models take more time for computation while machine learning models take low time. Our proposed models outperform deep learning both in terms of accuracy and efficiency with low computational time and high accuracy scores as shown in Table 17.

K-fold cross-validation results

To show the significance of our proposed approach we have also done k-fold cross-validation using all models on the used dataset. We used 10 fold in k-fold validation and the results of 10 fold cross validation are shown in Table 18. Performance of all models is approximately similar as was in train test split testing validation but Adaboost is poorer in 10 Fold cross-validation with 0.67 mean accuracy and 0.12 standard deviation (SD). Our proposed models outperform other models in 10 fold cross validation approach also with 1.00 mean accuracy and 0.00 SD which show the significance of the proposed approach.

Comparison with other studies

In this section, we compared our proposed approach with previous studies approaches. We select studies from the literature that have done work on stress detection and worked on hybrid models. We deployed their approaches on our used dataset to make a fair comparison. We implement previous studies’ approaches in the same environment as we deployed our approach. Then we perform the experiment on the same dataset which we used in this study for our proposed approach validation. We used study Salazar-Ramirez et al. (2018) for comparison because it also has worked on stress detection and used Gaussian SVM as a learning model. Similarly, Alberdi et al. (2018) and Lim et al. (2018) also have done work on stress detection and deployed SVM and ANN as learning models respectively. Rupapara et al. (2022) used LVTrees for blood cancer prediction. They combined LR, SVM, ETC for better results and we deploy this model on our used dataset. The comparison with other studies methods shown in Table 19.

Proposed approach results using another dataset

We experimented with another dataset to show the significance of the proposed approach. We used the “Human Stress Detection” dataset available on the public data repository, Kaggle (Rachakonda, 2022). This dataset consists of physiological features such as “Humidity, Temperature, Step count” and three target classes low stress (0), normal stress (1), and high stress (2). The dataset contains the total 2,001 samples and the ratio for each target class in the dataset is shown in Fig. 8.

We performed experiments using all models and the results are shown in Table 20. Results show that tree-based models are significant on this dataset with 100% accuracy and the HB model also outperforms in terms of all evaluation parameters because it’s a combination of tree-based models. Linear models such as LR and SVM are also good with 0.95 and 0.98 accuracy scores. All deep learning models are good with a 0.99 score in terms of all evaluation parameters. Our all proposed models are significant with the highest accuracy on both used datasets in this study.

Statistical T-test

We have done a statistical T-test to show the significance of the proposed approach. T-test rejects or accepts the null hypothesis based on the given two inputs. We compared the results of the proposed HB with all other used methods and find the T-score and critical value (CV). If the T score is less or equal to CV then its T-test accepts the null hypothesis else rejects the null hypothesis and accepts the alternative hypothesis.

• Accepted null hypothesis means there is no statistical difference in compared results.

• Accepted alternative hypothesis means there is a statistical difference in compared results.

Table 21 show the T-test value for each compared approach. In each case, T-test reject the null hypothesis and accepted the alternative hypothesis which means that HB is statistically significant as compared to other techniques. T-test gives 7.20 $e^{-17}$ CV and 2.3 T-score in each case which means a T-score greater than a CV in each compared case.