Enhancing intrusion detection performance using explainable ensemble deep learning

Applications of machine learning models for intrusion detection

Machine learning models were widely used for intrusion detection (Ahmad et al., 2021; Liu & Lang, 2019). Common machine learning models used to detect attacks include decision trees (DT), random forest (RF), support vector machines (SVM) and k-nearest neighbors (KNN). Debicha et al. (2021) evaluated the performance of different machine learning models for intrusion detection. The experimental results showed that random forest achieved the best accuracy compared to decision trees and SVM for binary classification of attacks. Dina, Siddique & Manivannan (2022) proposed a machine learning-based intrusion detection system to classify attacks. The experimental results showed that the decision tree model outperforms RF and SVM models for multi-class classification of attacks.

Sathesh (2019) proposed an intrusion detection method based on a combination of feature selection, clustering, and classification techniques. This method applied a fuzzy rule-based system to analyze the features followed by the application of DT model to select important features. To reduce the computational complexity, data are clustered using K-means to minimize the number of data in the training dataset. The SVM classifier is then used to categorize the attacks on the network. Soheily-Khah, Marteau & Béchet (2018) proposed a hybrid intrusion detection method based on K-means and random forest algorithm. The obtained results showed a better accuracy of attack prediction using the clustering technique. Sahu & Mehtre (2015) developed several DT-based models for intrusion detection such as C4.5, ID3 and CART. These models employ various techniques for feature selection, pruning, and handling missing values to build an efficient DT model for classifying network traffic.

Aburomman & Reaz (2016) proposed a machine learning-based intrusion detection model. They formulate the intrusion detection problem using the SVM classifier. The authors also used a radial basis function (RBF) kernel to categorize attacks into predetermined classes. The obtained results confirmed the suitability of SVM for intrusion detection tasks given its high accuracy compared to conventional machine learning methods. Li et al. (2014) also proposed an intrusion detection method based on KNN model. In the first step, the k-means clustering algorithm is used to build a cluster center for each group. Then, KNN is applied as a classification technique to determine the nearest neighbors. The experimental results showed that their proposed KNN-based classifier performs better than SVM in terms of classification accuracy. Azimjonov & Kim (2024) presented a novel approach for designing a lightweight intrusion detection system specifically for IoT networks. They utilized SVM as the underlying machine learning algorithm and employed feature selection methods to enhance the efficiency of the system. Four feature selection techniques were applied namely Importance Coefficient, Forward, Backward, and Correlation Coefficient. These methods were used to identify the most significant and effective features for detecting IoT botnet attacks.

Although the application of machine learning models has shown good accuracy in the detection and classification of attacks, it requires affordable efforts for pre-processing and configuring input features in a real-world attack investigation. Conventional machine learning models need predefined input features given as inputs for machine learning models. This problem is solved by using deep learning models that automatically extract and learn features from any input data (Koumakis, 2020). This important characteristic explains the effective and wide use of deep learning models in intrusion detection systems.

Applications of deep learning models for intrusion detection

Deep learning models have been well applied and implemented in intrusion detection systems. Several deep learning models for intrusion detection were proposed in the literature (Liu & Lang, 2019; Ahmad et al., 2021). Laghrissi et al. (2021) proposed a deep learning approach for intrusion detection using LSTM. The authors studied the performance of the model for binary and multi-class attack classification. The experimental results showed the suitability of LSTM to improve the classification accuracy of attacks compared to conventional machine learning methods. Muhuri et al. (2020) developed a method for intrusion detection to classify attacks by combining a genetic algorithm (GA) with LSTM neural network. They found that LSTM classifier combined with an optimal feature set improves the accuracy of intrusion detection. The experimental results performed on NSL-KDD dataset showed that GA can largely improve the classification accuracy of LSTM for both binary and multi-class classification. Obtained results using LSTM classifier outperformed those obtained using SVM and RF.

Zhang et al. (2023) introduced a novel approach that combines bidirectional long short-term memory (BiLSTM) with an attention mechanism for network intrusion detection. The proposed method leverages the advantages of BiLSTM in capturing dependency relationships between features. Additionally, an attention mechanism is employed to analyze the network traffic classification generated by the BiLSTM model. Experimental results on a real dataset demonstrate that the proposed method achieves higher detection accuracy compared to existing intrusion detection methods. Nguyen & Kim (2020) proposed a deep intrusion detection method based on CNN model. The authors first partitioned the features into four subsets using fuzzy c-means clustering and then converted them into grayscale format data. Subsequently, the CNN model was utilized for identifying attacks. Experimental results on the NSL-KDD dataset showed that the proposed CNN model achieves good accuracy compared to both machine learning and deep learning methods.

Halbouni et al. (2022) introduced a deep learning-based method for intrusion detection systems by combining various types of deep learning models such as CNN and LSTM. The authors also employed a feature selection technique to build the feature set. Experimental results on the NSL-KDD dataset demonstrated that the LSTM model achieved higher accuracy compared to the CNN model. Cao et al. (2022) proposed an intrusion detection approach utilizing a recurrent neural network based on GRU. The authors developed an efficient system that classifies network flow instances as normal or attack. A Pearson correlation feature selection algorithm was utilized to optimize the computational complexity. Experimental results on the CICIDS2018 dataset indicated that the GRU-based RNN outperformed existing deep learning models.

Shettar et al. (2021) also presented a multi-layer perceptron (MLP) model for classifying attacks. Experimental results on the NSL-KDD dataset demonstrated that the MLP model achieved higher accuracy compared to traditional machine learning models. Jeyanthi & Indrani (2023) proposed an intrusion detection system for healthcare applications using a deep learning model with custom features. The method incorporated a recurrent neural network (RNN) and a BiLSTM algorithm to detect and classify intrusion attacks. Custom features were extracted from incoming data streams and used to train the deep learning models. Experimental results showed the effectiveness of the proposed system in detecting and mitigating security threats in a healthcare environment.

Despite the high accuracy of deep-learning-based intrusion detection systems, they often operate as a “black-box” system that only gives a final classification of attacks without providing explanations for the reasoning behind the classification. Explanations may help security experts to transparently and effectively determine the correct attacks and implement the right strategies to protect the network.

Explainable intrusion detection systems

Several works studied the explainability of intrusion detection systems (Neupane et al., 2022) through using explainable artificial intelligence methods such as SHAP (Lundberg & Lee, 2017) and Local Interpretable Model-agnostic Explanations (LIME) (Ribeiro, Singh & Guestrin, 2016). Wang et al. (2020) proposed a framework that enhances the explainability of intrusion detection systems. Their method leverages SHAP to generate both local and global interpretations. The local interpretation focuses on explaining the reasoning behind a specific decision using a single data record, whereas the global interpretation utilizes the entire dataset to provide insights into the overall structure of the model.

Younisse, Ahmad & Abu Al-Haija (2022) also proposed an explainable intrusion detection system. They combined deep neural networks and the interpretability of the model predictions. The proposed system utilized SHAP to provide both local and global explanations. Keshk et al. (2023) proposed an explainable intrusion detection system in IoT networks. They developed a deep-learning intrusion detection approach based on LSTM and generated explanations using several techniques such as SHAP, Permutation Feature Importance, Individual Conditional Expectation, and Partial Dependence Plot. The proposed system achieved high accuracy and high interpretability of output results compared to other IDS systems.

Khan et al. (2021) introduced an explainable auto-encoder-based detection framework for identifying attacks in IoT networks. The framework utilizes CNN and LSTM models to detect both known and zero-day attacks. The data are initially processed by an auto-encoder-based LSTM to extract temporal features, followed by the CNN model for generating predictions. The LIME is employed to provide further explanations regarding the detected attacks. Also, in the work of Barnard, Marchetti & DaSilva (2022), an explainable intrusion detection method is proposed which integrates eXtreme Gradient Boosting (xgboost) as a classifier and SHAP as an explainable method. Despite the high detection accuracy, the proposed method has a high computational complexity due to the complexity of the explanation step. To reduce the computational cost, the authors utilized the PDP plots to eliminate features that cannot be adequately explained while maintaining nearly the same predictive accuracy.

Bashaiwth, Binsalleeh & AsSadhan (2023) proposed an explainable intrusion detection method that utilizes the LSTM model as a classifier. The authors combined both SHAP and LIME to enhance explainability. The combined XAI methods provide insights into the model’s decision-making process and help identify the intrinsic features necessary to differentiate attacks. The proposed model offers a deeper understanding of the factors contributing to the classification of attacks. Sharma et al. (2024) also introduced a deep neural network approach for intrusion detection that utilized a filter-based approach to reduce the dimensionality and emphasize the most important aspects. This approach selects the most relevant features and limits the number of inputs to the model. Furthermore, the authors applied a set of XAI techniques to explain the attack detection. Furthermore, Oseni et al. (2022) introduced an explainable deep learning-based intrusion detection framework to enhance the transparency of deep learning-based IDS in IoT networks. The framework incorporates the SHAP method to interpret the decisions made by the deep learning-based IDS. The authors provide insights and explanations regarding the reasoning behind the IDS’s predictions which allows for a better understanding of the underlying factors contributing to the classification of intrusions. An overview of existing explainable approaches for intrusion detection systems is given in Table 1. All the described studies focused on enhancing the interpretability of intrusion detection systems using various methods such as SHAP, LIME, and ensemble models. While these approaches have shown promising results in providing explanations for detected attacks, they are usually based on a single deep learning model such as CNN, RNN, and LSTM. We will explore in the next sections the use of explainable techniques to interpret the results of ensemble deep-learning-based models in intrusion detection applications.

LSTM

LSTM is an improved version of RNN (Hochreiter & Schmidhuber, 1997) that solves the problem of long-term dependencies within RNN-based networks. This problem consists of the inability of RNNs to retain information from previous input data over longer sequences to improve the prediction. To overcome this limitation, LSTM includes a memory cell to learn which information will be retained or forgotten. As described in Fig. 1, the memory cell is controlled by three primary gates: the forget gate, the memory gate, and the output gate. These gates enable LSTM to effectively manage and regulate the flow of information within the network.

First, the forget gate determines which information from the previous cell state should be discarded. It takes as inputs the previous input layer $x_{t-1}$, the previous hidden layer $h_{t-1}$, and the previous cell state $c_{t-1}$. The forget gate applies the $sigmoid$ function to these inputs, producing a real number between $0$ and $1$. This number represents the forget gate’s decision regarding the amount of information from the previous cell state that should be forgotten. Mathematically, the forget gate operation can be described as follows:

(1) $$f_t=sigm(W_f\cdot [h_{t-1},x_{t-1}]+b_f)$$where $f_t$ is the forget gate output at time-step $t$, $sigm$ is the $sigmoid$ function, $W_f$ is the weight matrix, $[h_{t-1},x_{t-1}]$ is the concatenation of the previous hidden layer and the previous input layer, and $b_f$ is the bias term.

After that, in a second step, the memory gate determines which new information should be stored in the current cell state. It consists of two components: the input gate and the new candidate values. The input gate decides which values should be updated while the new candidate values are potential new values that could be added to the cell state. The input gate and candidate values are computed using the $sigmoid$ and hyperbolic $tanh$ activation functions, respectively. Mathematically, the memory gate operations can be described as:

(2) $$i_t=sigm(W_i\cdot [h_{t-1},x_{t-1}]+b_i)$$

(3) $${\tilde{C}}_t=\tanh (W_C\cdot [h_{t-1},x_{t-1}]+b_C)$$where $i_t$ and ${\tilde{C}}_t$ are respectively the input gate output and the candidate value at time-step $t$, $W_i$ and $W_C$, are the weight matrices, $[h_{t-1},x_{t-1}]$ is the concatenation result of the previous hidden layer and the previous input layer, and $b_i$ and $b_C$ are the bias terms.

Finally, the output gate regulates the output information from the LSTM cell. It takes as inputs the current input layer $x_t$, the previous hidden layer $h_{t-1}$, and the current cell state $C_t$. Similarly to the forget gate, the output gate uses the $sigmoid$ function to determine which parts of the cell state should be returned as output. Additionally, the output gate applies the $tanh$ function to the cell state to generate values between $-1$ and $1$, which are then multiplied by the output of the $sigmoid$ function. Mathematically, the output gate operation can be described as:

(4) $$o_t=sigm(W_o\cdot [h_{t-1},x_t]+b_o)$$

(5) $$h_t=o_t\cdot \tanh (C_t)$$where $o_t$ and $h_t$ are respectively the output gate and the current hidden layers outputs at time-step $t$, $h_t$, is the current hidden layer output at time step $t$, $W_o$ is the weight matrix, $[h_{t-1},x_t]$ is the concatenation of the previous hidden layer and the current input layer, $b_o$ is the bias term, and $C_t$ is the current cell state.

We note here that other activation functions, such as rectified linear unit (ReLU) and Softmax, can be used as activation functions for the LSTM model. Our choice of $sigmoid$ and hyperbolic $tanh$ is based on their common use in RNN-based networks (Hochreiter & Schmidhuber, 1997; Zhang et al., 2023) and their successful applications in malware and intrusion detection contexts (Muhuri et al., 2020; Nguyen & Kim, 2020). Although $sigmoid$ and hyperbolic $tanh$ activation functions have certain limitations, such as being computationally consuming compared to other functions like ReLU and having a lower convergence speed during training, their widespread use within the LSTM model can be attributed to several reasons. Firstly, both functions are non-linear and capable of capturing complex patterns and relationships in the data. Secondly, the gradients of these functions can be easily computed, unlike other functions such as softmax, which allows to effectively optimize the network parameters during the learning process.

SHAP

The SHAP technique was introduced by Lundberg & Lee (2017) to explain predictions built using supervised learning methods. It is based on the idea of Shapley value from game theory which quantifies the contribution of each feature to build the final predictions. The Shapley value measures the difference in prediction when a specific feature is included or excluded. It calculates the marginal contribution of each feature by considering all possible combinations of features and their respective Shapley values. Suppose we have the following:

• $\mathcal{F}$ a set of features in a dataset

• $|\mathcal{F}|$ the total number of features.

• A a subset of features used in the prediction model.

• |A| the number of features in the subset A.

• ${\mathrm{\Delta }}_i(A,x)$ is the difference between the prediction generated when including the feature $i$ and all the other possible predictions generated without including the feature $i$.

The Shapley value for a particular feature $i$, denoted as $S{V}_i(x)$, is calculated as the sum over all subsets A that include the feature $i$ as follows:

(6) $$S{V}_i(x)=\sum _{A\subseteq \mathcal{F}\setminus i}\frac{|A|!\times (|\mathcal{F}|-|A|-1)!}{|\mathcal{F}|!}\times {\mathrm{\Delta }}_i(A,x).$$

The calculated Shapley value $S{V}_i(x)$ provides insights into the marginal contribution of feature $i$ to the prediction of a single data record $x$. The importance of feature $i$ in the determination of the class label of the data record $x$ is quantified by considering all possible subsets of features with and without including that feature in the learning model. The contribution $S{V}_i(x)$ can be either positive or negative. A positive value indicates that feature $i$ positively influences the prediction, whereas a negative value suggests a negative influence.

The calculated $S{V}_i(x)$ only gives an interpretation of the importance of feature $i$ in determining the class label of the data record $x$ (Qaffas et al., 2023a). However, to build a global overview of the importance of each feature across all data records, global Shapley values can be calculated (Qaffas et al., 2023b) by summing local Shapley values for all data records $x\in X$ as in the following:

(7) $$\overline{S{V}_i(x)}=\frac{\sum _{x\in X}|S{V}_i(x)|}{|X|}$$where |X| is the number of data records in the dataset and $|S{V}_i(x)|$ is the absolute value of the local Shapley value for the data record $x$ on feature $i$. Global Shapley values provide a comprehensive understanding of features’ importance and their variation across the whole predictive model enabling better interpretation of the learning model.

Phase 1: Data detector modeling

The data detector modeling phase involves an ensemble deep learning model utilizing three LSTM structures. This phase is divided into two main stages: (1) the base-learner stage, where three distinct LSTM neural networks are employed, and (2) the meta-learner stage, which focuses on applying a machine learning algorithm to effectively aggregate the different results.

LSTM model training

Deep learning models have shown superior accuracy compared to traditional machine learning models (Lansky et al., 2021). Specifically, LSTM-based models offer several advantages including ease of training and satisfactory performance even with limited computational resources. These characteristics make them highly suitable for addressing the challenges associated with intrusion detection (Zhang et al., 2023). We leverage the advantages of LSTM networks by proposing a design that includes three distinct LSTM configurations as base learners. The outputs of these base learners will be then aggregated using a meta-learner to build improved final results. The use of multiple LSTM base learners enables to explore different representations and variations in the data and increases the ability to capture complex relationships.

Initially, a simple LSTM classifier with a single LSTM layer is developed. Then, additional LSTM layers are added by using two key techniques: batch normalization and dropout. Batch normalization is applied to normalize the activation within each mini-batch to guarantee stable and consistent learning across the network (Bjorck et al., 2018). On the other hand, the dropout is applied to reduce network complexity by randomly deactivating neurons during training. This allows for preventing over-reliance on specific connections and improving generalization (Baldi & Sadowski, 2013). Furthermore, we introduce a flattened layer before the fully connected dense layer. This flattening operation reshapes the input data to ensure compatibility with the subsequent layers. The output of the last layer is considered as the final returned predictions. The proposed layers of the LSTM classifiers are illustrated in Figs. 3–5. These visual representations provide a clear overview of the architectural configuration for each classifier and highlight the arrangement and connectivity of the LSTM layers.

Concerning the activation of the output dense layer, we use the Softmax (Liu et al., 2016) function which can be defined as follows:

(8) $$\sigma (x{)}_i=\frac{e^{x_i}}{{\sum }_{j=1}^C{e}^{x_j}}$$where $x$ represents the input vector and C denotes the total number of classes. The Softmax function plays a crucial role in transforming the raw scores into a normalized probability distribution across multiple classes. It is important to note that the Softmax function ensures a valid probability distribution given that all probabilities across all classes sum up to $1$.

Concerning the loss function, we opt for the Categorical Cross-Entropy (CCE) (Rusiecki, 2019) due to its effectiveness in handling multi-class classification tasks. The CCE loss function is well-suited for LSTM models and has the purpose of quantifying the dissimilarity between the predicted probability distribution and the true class label distribution. CCE is mathematically defined as:

(9) $$CCE=-\frac{1}{N}\sum _{i=1}^N\sum _{c=1}^C(p_{ic}\log (y_{ic}))$$where $p_{ic}$ is a binary indicator function that indicates whether the data record $i$ belongs to category $c$ and $y_{ic}$ represents the predicted probability distribution for the data record $i$ belonging to class $c$. By minimizing the CCE loss, the model is trained to assign higher probabilities to correct classes and lower probabilities to incorrect ones.

Each LSTM model is trained using a $k$-fold cross-validation approach, where $k$ represents the number of folds. The training data is split into $k$ folds: $D=\{D_1,D_2,\dots ,D_k\}$. Each fold $D_i$ is used as a validation set and the remaining ( $k-1$) folds are used as a training set for each LSTM model. Once the models are trained, predictions from each LSTM model for the data records in fold $D_i$ are obtained. These predictions are denoted as $c{1}_i$, $c{2}_i$, and $c{3}_i$, representing the predicted class labels for LSTM1, LSTM2, and LSTM3, respectively. Given the crucial role of LSTM parameters in building effective learning models, we propose to explore different values for each parameter. We used the GridSearch algorithm (Gozzoli, 2018) to optimize all the LSTM parameters given its ease of implementation and its effective computational complexity. Selected parameter values for the three LSTM classifier configurations are reported in Table 2. Each classifier is designed with a unique combination of parameter values.

Table 2:

LSTM classifiers’ architectures.

Parameter	LSTM-1	LSTM-2	LSTM-3
No. of LSTM layers	1	2	3
No. of filters	32	64	128
Batch normalization	–	$\surd$	$\surd$
Dropout rate	–	0.1	0.1
Network optimizer	Adam	Adam	Adam
Kernel size	3	5	7
Batch size	32	32	32
Epochs	50	100	100

DOI: 10.7717/peerj-cs.2289/table-2

Phase 2: Model explaining

This phase aims to provide explanations for the classes generated in the previous phase. These explanations will allow cybersecurity experts to better understand and interpret the resulting classification of attacks at both local (individual traffic data) and global (attack class) levels.

Concerning local explanations, it focuses on explaining the predictions made for individual data traffic records and the reasons behind assigning them to specific attack class. To achieve this, we compute the Shapley value for each data record and each feature. The calculated Shapley values quantify the contribution of each attack feature, for each traffic record, towards the final prediction. These local explanations support security experts in comprehending the reasons behind assigning a connection to a particular normal or attack class. Furthermore, experts can analyze the positive and negative contributions of each feature for each connection, providing insights into the impact of individual features on the predicted class labels.

Let us consider a small traffic dataset with three features: “Source IP”, “Destination IP”, and “Protocol”. We will focus on explaining the predictions made for individual data traffic records using local Shapley values. Suppose we have the following three data records:

• Record 1: Source IP: ‘192.168.0.1’, Destination IP: ‘10.0.0.1’, Protocol: ‘TCP’

• Record 2: Source IP: ‘192.168.0.2’, Destination IP: ‘10.0.0.2’, Protocol: ‘UDP’

• Record 3: Source IP: ‘192.168.0.3’, Destination IP: ‘10.0.0.3’, Protocol: ‘ICMP’

For each record and each feature, we will calculate the Shapley value with respect to the assigned class (normal or attack). Let us assume the assigned class for each record is as follows:

• Record 1: Assigned Class $\to$ Normal

• Record 2: Assigned Class $\to$ Attack

• Record 3: Assigned Class $\to$ Normal

To compute the Shapley values, we analyze the contribution of each feature for each record towards the final prediction. Let us assume the Shapley values are calculated as follows:

• Record 1:

  • – Shapley Value (Source IP) $=0.1$

  • – Shapley Value (Destination IP) $=-0.2$

  • – Shapley Value (Protocol) $=0.3$

• Record 2:

  • – Shapley Value (Source IP) $=-0.4$

  • – Shapley Value (Destination IP) $=0.5$

  • – Shapley Value (Protocol) $=0.2$

• Record 3:

  • – Shapley Value (Source IP) $=0.2$

  • – Shapley Value (Destination IP) $=-0.3$

  • – Shapley Value (Protocol) $=-0.1$

These Shapley values quantify the contribution of each feature for each record. Positive values indicate a positive contribution towards the assigned class, whereas negative values indicate a negative contribution. Based on these local explanations, security experts can better understand the reasons behind assigning each connection to a specific normal or attack class. They can analyze the positive and negative contributions of each feature for each connection and gain insights into the impact of individual features on the predicted class label. For example, in Record $1$, the “Source IP” and “Protocol” features have a positive contribution towards the assigned class while the “Destination IP” feature has a negative contribution.

To provide security experts with summarized class interpretations, we generate further explanations at the class level. These explanations allow identifying the most critical features when building each attack class and then offer valuable insights into the workings of the learning model. To determine the importance of each feature, we calculate the average of the absolute Shapley values locally for each class across all traffic records. The calculated global shapley values are typically sorted in descending order to show the most important ones for each class. These important attack features provide insights into the proposed prediction model and its ability to distinguish between different attack classes which can help in better understanding the automatic recommendations built by the intrusion detection system.

Let us consider the small example of three records described above, global shapley values for each class can be calculated as follows:

• Assigned Class $\to$ Normal:

  • – Global Shapley Value (Source IP) $=|0.1+0.2|/2=0.15$

  • – Global Shapley Value (Destination IP) $=|(-0.2)+(-0.3)|/2=0.25$

  • – Global Shapley Value (Protocol) $=|0.3+(-0.1)|/2=0.1$

• Assigned Class $\to$ Attack:

  • – Global Shapley Value (Source IP) $=|-0.4|/1=0.4$

  • – Global Shapley Value (Destination IP) $=|0.5|/1=0.5$

  • – Global Shapley Value (Protocol) $=|0.2|/1=0.2$

These global Shapley values represent the average absolute Shapley values per feature for each assigned class. These global Shapley values provide insights into the overall impact of each feature on the predictions for each class. They can help in understanding the relative importance of features for differentiating between attack classes. For example, for the ‘Normal’ class, the most important feature is ‘Destination IP’, followed by the ‘Source IP’ and ‘Protocol’ and also for the second class, “Attack”, the most important feature is ‘Destination IP’, followed by the ‘Source IP’ and ‘Protocol’.

Evaluation methodology: datasets description and evaluation measures

Empirical experiments were conducted on a Dell Inspiron machine with an Intel^(R) Core^(TM) i7-1165G7 processor running at $2.80$ GHz and $8.00$ GB of RAM. All methods were implemented within Python $\mathrm{3.9.7}$ by using TensorFlow, Pandas, and Keras libraries and evaluated on two datasets, NSL-KDD (https://github.com/Mamcose/NSL-KDD-Network-Intrusion-Detection/tree/master) and UNSW-NB15 (https://research.unsw.edu.au/projects/unsw-nb15-dataset), which are specifically designed for intrusion detection systems. The first dataset, NSL-KDD, is an enhanced version of the widely used KDD-CUP99 dataset (https://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html). Several improvements were made to ensure its suitability for robust intrusion detection analysis. Firstly, redundant and duplicate records were eliminated to prevent biases and ensure a balanced representation of data. Secondly, the distribution of records across different attack categories in the NSL-KDD dataset was adjusted to address the imbalances in the original KDD-CUP99 dataset. The number of records in each category was adjusted inversely proportional to the percentage of records in the KDD-CUP99 dataset. This adjustment helps to create a more realistic and representative dataset for training and testing intrusion detection models. Each data record in the dataset comprises a total of $41$ features, including three symbolic, seven binary, and $31$ numerical features. Detailed descriptions of these features can be found in Dhanabal & Shantharajah (2015) where authors provide valuable insights into the dataset composition and characteristics. The NSL-KDD dataset classifies network traffic into normal traffic and four types of attacks, namely denial of service (DOS), probe (PROB), remote to local (R2L), and user to root (U2R).

Concerning the second dataset, UNSW-NB15, was created by Moustafa & Slay (2015) to address the need for a realistic and representative dataset for evaluating intrusion detection models. Unlike NSL-KDD dataset, UNSW-NB15 contains new attack and standard network traffic data. Each data record in the dataset is described by $47$ features, including three symbolic, two binary, and $42$ numerical features. Detailed descriptions of these features can be found in Moustafa & Slay (2015) where authors provide valuable insights into the dataset description. The UNSW-NB15 dataset classifies network traffic into normal traffic and nine types of attacks, namely Exploits, Generic, Reconnaissance, Analysis, Shellcode, DoS, Worms, Fuzzers and Backdoors.

To assess the effectiveness of the proposed method, we utilized four evaluation metrics: accuracy, precision, recall, and F1-score. These metrics provide a comprehensive analysis of the intrusion detection system’s accuracy. Given that multiple classes are defined in each dataset, we considered the calculation of these measures for each class $i$ separately using the following formula:

(10) $$Accurac{y}_i=\frac{TP+TN}{TP+TN+FP+FN}$$

(11) $$Precisio{n}_i=\frac{TP}{TP+FP}$$

(12) $$Recal{l}_i=\frac{TP}{TP+FN}$$

(13) $$F1-Scor{e}_i=\frac{2\ast Precision\ast Recall}{Precision+Recall}$$where a true positive (TP) indicates that the intrusion detection system accurately detects a specific attack that has occurred, a true negative (TN) indicates that the system correctly identifies a normal connection, a false positive (FP) indicates that the system erroneously detects an attack that did not occur and a false negative (FN) occurs when the intrusion detection system fails to detect an intrusion after a specific attack. $Accurac{y}_i$ represents the proportion of accurate predictions local to class $i$. $Precisio{n}_i$ measures the proportion of true positives compared to all correct predictions while $Recal{l}_i$ measures the proportion of actual correctly predicted positive instances. The last measure, $F1-scor{e}_i$, is a harmonic mean of both $Precisio{n}_i$ and $Recal{l}_i$. All these described measures are calculated for each class $i$ and then an average over all classes (macro-averaging) is computed as in the following:

(14) $$Accuracy=\frac{1}{N}\sum _{i=1}^{N}Accurac{y}_i$$

(15) $$Precision=\frac{1}{N}\sum _{i=1}^{N}Precisio{n}_i$$

(16) $$Recall=\frac{1}{N}\sum _{i=1}^{N}Recal{l}_i$$

(17) $$F1-Score=\frac{1}{N}\sum _{i=1}^{N}F1-Scor{e}_i$$where N is the total number of classes. All of these measures provide a quantitative assessment of the performance of each evaluated method, where higher values indicate better performance in identifying network intrusions.

Performance evaluation of ensemble learning

In this section, we present the evaluation of the performance of our proposed ensemble deep learning method. Firstly, we conducted experiments to assess the accuracy of the meta-learner algorithm used in EED. We evaluated the performance of EED by using various machine meta-learning algorithms, including DT, RF, K-NN, and SVM. Obtained results on NSL-KDD and UNSW-NB15 datasets are summarized in Table 3. The reported results indicate that using Random Forest as a meta-learner for the proposed EED method achieved the highest accuracy for both datasets. This is explained by its robustness and ability to handle various data types and structures. Consequently, the Random Forest will be considered as the default meta-learner algorithm for all the next presented experiments to assess EED performance.

Then, we evaluated the accuracy when varying the k-fold cross-validation parameter. The evaluation results are presented in Table 4. Reported results of NSL-KDD demonstrate that EED achieved the highest accuracy when $k$ = 8 and $k$ = 6 with $96.99$ and $97.05$ respectively. However, the obtained results of UNSW-NB15 dataset clearly show that EED achieved the highest accuracy when $k$ = 8 with $95.15$. Consequently, we decided to continue evaluations with $k$ = 8.

Next, we evaluated the accuracy for each LSTM classifier as outlined in “Phase 1: Data detector modeling”. We individually assessed the performance of each LSTM classifier and compared it to the performance of the ensemble model. The comparative results on the NSL-KDD and UNSW-NB15 datasets are presented in Table 5. Reported results on both datasets demonstrate that our proposed ensemble learning model outperformed all individual LSTM classifiers in terms of all evaluation measures. These experiments confirm that combining the outcomes of different LSTM structures significantly enhances the overall model performance. The Random Forest algorithm as the meta-learner and the combination of diverse LSTM classifiers contribute to achieving superior performance compared to individual LSTM models.

Comparison between EED and existing intrusion detection methods

We assessed the accuracy of the proposed EED method compared to existing machine learning and deep learning-based intrusion detection methods. The comparative results are presented in Table 6. Reported results of NSL-KDD dataset demonstrate that EED achieved the highest accuracy among the compared methods. Our model attained an accuracy value of $97.05$% surpassing all machine learning algorithms such as the decision tree-based method that achieved an accuracy of $95.17$%, the K-nearest neighbor approach with $94.13$% and the SVM method with $94.58$%. Similarly, the accuracy achieved by our model on UNSW-NB15 dataset was 95.15% outperforming other machine learning algorithms, such as the decision tree-based method with $93.87$% and the SVM method with $94.12$%. These results can be explained by the effectiveness of the designed LSTM layers in building and extracting better input features from the dataset.

Furthermore, when compared to deep learning-based models, our EED model also demonstrated superior accuracy for both datasets. For example, the Multilayer Perceptron achieved an accuracy of $94.18$%, the CNN achieved $95.77$%, and GRU achieved $96.57$% for NSL-KDD dataset. The higher accuracy of our EED model compared to deep-learning-based methods can be explained by the strength of ensemble learning to combine results of multiple deep models to enhance the overall performance. Therefore, the obtained results highlight the effectiveness and superiority of the proposed EED method compared to existing machine learning and deep learning-based intrusion detection methods. The utilization of ensemble learning, along with the designed LSTM layers, has resulted in enhanced intrusion detection capabilities.

Then, we evaluated the running times of the proposed method compared to existing methods. The comparative results are presented in Fig. 6. Machine learning based method spends less time on each dataset, but its performance is not as good as other deep learning methods. Most of deep learning methods are time-consuming on the two datasets. Although the running time values of deep learning methods are high, their performance is excellent compared to machine learning. Our method showed a relatively comparable running times to deep learning-based methods while achieving superior accuracy. Thus, our proposed EED method is relatively time consuming with enhanced detection accuracy.

Generating local and global explanations

To provide local explanations, we used the force plot which shows the contribution of each feature in the model’s decision-making process. Figure 7A presents a local explanation for an average of $10$ data records from NSL-KDD datasets, correctly classified by the model as a DOS attack. The feature same_srv_rate with a value of $0.01$ significantly influences the decision to assign records to this class. A lower value of same_srv_rate increases the probability of assigning the record to the DoS attack class. Additionally, diff_srv_rate with a value of $0.08$ and count with a value of $103$ also contribute to the assignment of records to the DoS attack class. Figure 7B displays a local explanation for an average of $10$ data records classified as a Probe (PROB) attack. The plot reveals that feature values such as src_bytes equal to $0$, dst_host_rerror_rate equal to $0.25$ and dst_bytes equal to $0$ predominantly contribute to the decision of classifying a record as a PROB attack. It is important to note that dst_host_same_srv with a value of $0.52$ decreases the probability of assigning a record to the PROB attack class. Concerning Figure 7C, it illustrates a local explanation for $10$ data records correctly classified as R2L attack. The values of features such as num_compromised = 2, hot = 3, and src_bytes = 2.42 significantly influence the decision to classify a record as an R2L attack. Higher values of these features largely increase the probability of classifying the record as an R2L attack. The last Fig. 7D presents a local explanation for $10$ data records correctly assigned to the User to U2R attack class. The feature duration with a value of $12$ positively contributes to the final classification indicating that a large duration value increases the likelihood of the connection being assigned to the U2R attack class. Moreover, dst_bytes = 0 and src_bytes = 7.045 also positively contribute to this decision.

Figure 8A presents a local explanation for an average of $10$ data records from UNSW-NB15 dataset which are correctly classified by the model as a Generic attack. The feature sbytes with a value of $1.76$ significantly influences the decision to assign records to this class. A higher value of sttl increases the probability of assigning the record to the Generic attack class. In addition, smean with a value of $638$ and ct_dst_src_ltm with a value of $2$ also contribute to the assignment of records to the Generic attack class. Concerning Fig. 8B, it shows a local explanation for $10$ data records correctly classified as Analysis attack. The values of features such as dbytes = 0.001, sttl = 62, and dmean = 439 significantly contribute the decision to classify a record as an Analysis attack. Higher values of these features largely increase the probability of assigning the record as an Analysis attack. Figure 8C illustrates a local explanation for an average of $10$ data records classified as a DoS attack. This figure shows that feature values such as ct_dst_src_ltm equal to $4$, sbytes equal to $200$ and seman equal to $100$ scientifically increase the probability of classifying a record as a DoS attack. Lastly, Fig. 8D presents a local explanation for $10$ data records correctly assigned to the Backdoors attack class. The feature sbytes with a value of $114$ positively contributes to the final classification indicating that a large value of this feature increases the likelihood of the data record being classified to the Backdoors attack class. In addition, a lower value of ct_dst_sport_ltm increases the probability of assigning the record to the Backdoors attack class.

In order to provide insights into the important features that were crucial in building attack classes, we build global explanations of EED results by using the SHAP feature-summary-plot as schematized in Figs. 9 and 10 for NSL-KDD and UNSW-NB15 datasets respectively. In these plots, each dot corresponds to a data record in the dataset. The vertical position of a dot represents a specific feature, while the horizontal position indicates the impact of that feature’s value on the model’s classes (local to each class individually). The color of each dot represents the value of the corresponding feature for that record in the dataset, with red indicating high values, purple indicating medium values, and blue indicating low values. Figure 9A focuses on the identification of DOS attacks. It shows that high values of the src_bytes feature increase the probability of a connection being classified as a DOS attack by $20\mathrm{\%}$ to $30\mathrm{\%}$. Additionally, low values of the dst_bytes feature contribute to the identification of DOS attacks. Moving to the PROB attack class in Fig. 9B, explanations reveal that low values of the src_bytes feature increase the probability of a prediction being classified as a PROB attack by $30\mathrm{\%}$. Moreover, large values of the dst_host_same_srv feature raise the probability of the connection being classified as a PROB attack by $10\mathrm{\%}$ to $30\mathrm{\%}$.

Concerning Fig. 9C, it demonstrates the global explanation of the R2L attack class. It indicates that low values of the dst_srv_count feature increase the probability of a prediction being classified as an R2L attack, with a range of $0\mathrm{\%}$ to $30\mathrm{\%}$. Also, large values of the logged_in feature increase the probability of the connection being classified as an R2L attack, ranging from $0\mathrm{\%}$ to $30\mathrm{\%}$. Lastly, Fig. 9D provides the global explanation of the U2R attack class. It shows that low values of the dst_srv_count feature increase the probability of considering records as U2R attacks, ranging from $0\mathrm{\%}$ to $30\mathrm{\%}$. Additionally, large values of both dst_host_same_src and logged_in features increase the probability of the connection being classified as a PROB attack, with a range of $0\mathrm{\%}$ to $20\mathrm{\%}$.

Figure 10A displays the global explanation of Generic attack class. It shows that low values of the sttl feature increase the probability of a connection being classified as a DOS attack by $20\mathrm{\%}$ to $30\mathrm{\%}$. Moreover, high values of the smean feature influence the classification of Generic attacks. However, reported results in Fig. 10B show that high values of the smean feature raise the probability of assigning a connection as an Analysis attack by $15\mathrm{\%}$. In addition, low values of the sbytes feature increase the probability of the data record to be classified as an Analysis attack by $20\mathrm{\%}$. Moving to Fig. 10C that gives explanations of the DoS attack class, reported results show that low values of ct_srv_dst feature contribute significantly to the classification of data records as a DoS attack, with a range of $0\mathrm{\%}$ to $30\mathrm{\%}$. Similarly, large values of the sbyte feature increase the probability of the connection to be classified as a DoS attack with a range of $5\mathrm{\%}$ to $20\mathrm{\%}$. Finally, Fig. 10D displays the global explanation of the Backdoors attack class. It indicates that low values of both sbytes and smean features increase the probability of the connection being classified as a Backdoors attack, with a range of $20\mathrm{\%}$ to $30\mathrm{\%}$. Moreover, large values of the ct_dst_sport_ltm feature increase the probability of a connection being classified as Backdoors attacks, ranging from $10\mathrm{\%}$ to $20\mathrm{\%}$.