An optimized ensemble model with advanced feature selection for network intrusion detection

Genetic algorithms

GAs are adaptive heuristic search algorithms based on the principles of natural selection and genetics. They are particularly effective for optimization problems where the solution space is vast and not explicitly defined. GAs operate by maintaining a population of candidate solutions, which are iteratively evolved to produce better solutions over time. The evolution process in GAs involves several key steps. First, a selection process is carried out, where the best-performing individuals, or solutions, are chosen based on a fitness function that evaluates their quality. Next, a crossover operation is applied, where selected individuals are recombined to create new offspring by combining traits from both parents. This step helps explore new regions of the solution space. Finally, a mutation process introduces random modifications to the offspring to increase diversity within the population, prevent premature convergence, and avoid getting trapped in local optima. Together, these steps ensure that the algorithm effectively navigates the solution space to find optimal or near-optimal solutions.

In the domain of NIDS, GAs are employed to optimize the feature selection process by navigating through the feature space to identify the most relevant attributes. This approach enables the proposed “Opt-Forest” model to construct more compact and accurate decision trees, thereby improving its ability to detect sophisticated and evolving cyber threats. GAs help in balancing exploration and exploitation during feature selection, ensuring that the model does not get stuck in suboptimal solutions and can adapt to diverse attack patterns.

Random forests

Random forests are an ensemble learning technique primarily used for classification and regression tasks. The model builds multiple decision trees during training, each constructed using a random subset of the training data (bagging) and a random subset of features at each split. This randomness introduces diversity among the trees, which reduces the variance and helps prevent overfitting. Each tree in the forest independently votes on the class of a given input, and the final prediction is determined by majority voting (in classification) or averaging (in regression). The strength of Random Forests lies in their ability to handle large datasets with higher dimensionality and their robustness against noise and overfitting.

In the proposed “Opt-Forest” model, random forests serve as the base classifier, with the decision-making process enhanced through optimization techniques like genetic algorithms. By integrating GAs based optimization with the traditional random forest approach, the model achieves greater accuracy and robustness, particularly in detecting subtle and evolving network intrusions. This combination leverages the strengths of both techniques. Random forests ability to generalize across varied data and GAs capacity to optimize feature selection, resulting in a more efficient and reliable intrusion detection system.

System model

To address the limitations of traditional machine learning techniques in network intrusion detection, we introduce an optimized ensemble model for creating decision forests. The primary goal of our study is to offer an ensemble method for network intrusion detection (NID) that uses GAs and decision forests. The proposed system is structured around three core components: Data Gathering and Preliminary Processing, Feature Selection, and Model Training and Evaluation. Figure 1 illustrates the overall system architecture, showing the flow of data and the interactions between components.

• Data gathering and preliminary processing: The proposed system begins with the collection of a dataset sourced from the Kaggle repository, which serves as the foundation for the network intrusion detection process. Once the data is gathered, it undergoes a preprocessing step to address any missing values and prepare it for subsequent analysis. This involves cleaning the data to ensure its quality and consistency, followed by organizing it into a format suitable for further processing. The preprocessing phase is critical to ensure that the data is accurate and ready for feature extraction, which directly impacts the effectiveness of the intrusion detection system.

• Feature selection: After preprocessing, the system employs advanced feature selection techniques to identify and select the most relevant features from the dataset. This process includes methods such as Best-First Search, Particle Swarm Optimization (PSO), Evolutionary Search, and GS. By integrating these optimization techniques, the system enhances the accuracy, robustness, and efficiency of the decision forest model. The feature selection phase is crucial for improving model performance, as it reduces dimensionality and focuses on the most informative attributes, thereby enhancing the model’s ability to detect network intrusions effectively.

• Model training and evaluation: The core of the system involves training the “Opt-Forest” model using 70% of the dataset, while the remaining 30% is reserved for testing the model’s efficacy. The ensemble model integrates decision forests with genetic algorithms to optimize feature selection and enhance detection performance. Following training, the system evaluates the model by comparing it with contemporary machine learning models, including AdaBoostM1 (AbM1), J48, K-nearest neighbor (KNN), logistic model tree (LMT), multilayer perceptron (MLP), naïve Bayes (NB), and stochastic gradient descent (SGD), based on metrics such as true positive rate (TPR), false positive rate (FPR), precision, Matthews correlation coefficient (MCC), recall, and accuracy. The evaluation also includes a comprehensive performance assessment using measures like accuracy, precision, recall, and F-measure. This thorough analysis ensures that the model is robust and effective in identifying network breaches, and its adaptability and reliability in maintaining network security are well validated.

Threat model

The threat model for our proposed “Opt-Forest” model encompasses a broad range of cyber threats, including Denial of Service (DoS) and Distributed Denial of Service (DDoS) attacks, probing and scanning activities. The system is designed to address both external adversaries, who attempt to exploit network vulnerabilities from outside and internal malicious actors, who misuse their legitimate access to execute insider attacks. To counter these evolving threats, the proposed model leverages a combination of ensemble learning and optimization techniques to ensure timely, accurate detection while minimizing false positives and negatives, thus providing robust protection against a diverse spectrum of intrusions.

Proposed “Opt-Forest” model

In this section, we introduce the proposed algorithm, named “Opt-Forest.” This algorithm is specifically designed to optimize an ensemble of decision forests, such as Random Forest, using a GAs framework. This approach leverages genetic algorithms for optimization and decision forests for creating the ensemble model in the context of NID. This combination of techniques aims to improve the accuracy and efficiency of intrusion detection systems by harnessing the strengths of both genetic algorithms and decision forests. The algorithm consists of several phases, each aimed at refining the population of decision trees.

Iterative refinement

The algorithm proceeds through J iterations, each consisting of following multiple steps aimed at refining the population of decision trees.

• Initial population selection: In each iteration, the algorithm performs initial population selection. For odd-indexed chromosomes, stratified sampling is performed to create strata $S1$, $S2$, and $S3$. From these strata, M trees are randomly selected using disproportionate stratified sampling (DSS) to form high-quality chromosomes. For even-indexed chromosomes, M trees are randomly selected from the entire forest, ensuring diversity in the population.

• Crossover and Mutation: Next, the algorithm performs “Crossover and Mutation.” Chromosome pairs are selected for crossover, where a 1-point crossover technique generates new offspring chromosomes by combining genetic material from the parent chromosomes. These offspring undergo a 1-bit flipping mutation, introducing variability and aiding the exploration of the solution space.

• Elitist Operation: Following this, the “Elitist Operation” step is executed to preserve high-quality solutions. Chromosomes are duplicated into $P_{TempCur}$, and the best chromosome in the current iteration is stored as $CrCurrBest$. A crossover is applied again to create $P_{Mod}$, a modified population. The best chromosome in $P_{Mod}$ ( $CrModBest$) is compared with $CrSFB$; if $CrModBest$ has a better evaluation score, it replaces $CrSFB$. Additionally, the worst chromosome in $P_{Mod}$ is replaced with $CrCurrBest$ if the latter has a better evaluation score, ensuring that elite solutions are carried forward.

• Chromosome Selection for the Next Iteration: The algorithm then moves to “Chromosome Selection for the Next Iteration.” A new pool of chromosomes ( $P_{Pool}$) is created by combining $P_{cur}$ and $P_{Mod}$. From this pool, $20$ chromosomes are selected using the roulette wheel selection method, which favors chromosomes with higher fitness scores, ensuring that the next generation starts with a strong set of potential solutions.

• Rectification of So Far Best Chromosome: Finally, the Rectification of So Far Best Chromosome (SSO) step is performed. Sequential Search Operations are applied to $CrSFB$. Each bit in the chromosome is systematically flipped from $1$ to $0$ or from $0$ to $1$, checking if the evaluation score improves with each flip. This fine-tuning process helps in identifying and solidifying the best possible solution.

The iterative process continues until J iterations are completed. Throughout the iterations, the combination of stratified sampling, crossover, mutation, elitist operations, and sequential search ensures that the algorithm effectively explores and exploits the solution space, gradually improving the quality of the decision forest ensemble. The final output is a robust ensemble model optimized for detecting network intrusions with high accuracy and low false positives. The Algorithm 1 and Fig. 2 provides a detailed explanation of the proposed “Opt-Forest” algorithm.

Algorithm 1 :

Opt-Forest algorithm.

1:  Initialization:

2:  Create Pcurr, PTempCurr, and PMod.

3:  Initialize CrSFBest.

4:  for $i=1$ to I (Iterations) do

5:   Preliminary Population Selection:

6:   Selection for odd chromosomes:

7:   To create strata $S_{t1}$, $S_{t2}$, $S_{t3}$, do stratified sampling.

8:   From strata, randomly select M trees using disparate stratified sampling (DSS) to form CrOdd.

9:   Selection for even chromosomes:

10:   To form CrEven, randomly select M trees from the forest.

11:   Crossover and Mutation:

12:   Select pairs $(C{r}_i,C{r}_j)$ of chromosomes for crossover.

13:   To generate offspring apply 1-point crossover: $CrOffspring=C{r}_i\oplus C{r}_j$.

14:   Do 1-bit flipping mutation on CrOffspring.

15:   Discriminatory Operation:

16:   Double chromosomes to PTempCurr.

17:   Save the finest chromosome as CrCurrBest.

18:   To create PMod, apply crossover.

19:   Compare CrSFBest with CrModBest:

20:   if EA(CrModBest) > EA(CrSFBest) then

21:      $CrSFBest=CrModBest$

22:   end if

23:   if EA(CrCurrBest) > EA(CrWorst) then

24:     $CrWorst=CrCurrBest$

25:   end if

26:   Update CrCurrBest from PMod.

27:   Select Chromosome for the Next Iteration:

28:   Create PPool by combining Pcurr and PMod: $P_{Pool}=P_{curr}\cup P_{Mod}$

29:   Using roulette wheel selection, from PPool select 20 chromosomes.

30:   Refinement of So Far Best Chromosome:

31:   Put on Sequential Search Operations:

32:   for each bit bi in CrSFBest do

33:    if $b_i=1$ and EA(CrSFBest) improves then

34:      $b_i=0$

35:    else if $b_i=0$ and EA(CrSFBest) improves then

36:      $b_i=1$

37:    end if

38:   end for

39:  end for = 0

DOI: 10.7717/peerj-cs.2472/table-101

Data gathering and preliminary processing

The study’s data was obtained from the Kaggle repository, which can be accessed at https://www.kaggle.com/datasets/sampadab17/network-intrusion-detection. The dataset consists of 22,544 rows with 41 columns which initially contains some missing values. To handle these missing values, we applied the mean imputation technique, a common statistical method for handling missing data points.

The mean imputation technique replaces each missing value with the mean (average) value of the observed data within the same feature. Mathematically, this can be represented as:

(1) $$\mathrm{M}\mathrm{e}\mathrm{a}{\mathrm{n}}_{\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{u}\mathrm{t}\mathrm{e}\mathrm{d}}=\frac{\sum _{i=1}^n{x}_i}{n}$$where, $x_i$ represents the mean values of the features and $n$ represents the observed data points within the features. This equation calculates the average value by summing all observed values and dividing by the number of observations. To illustrate, consider a feature F with missing values $x_m$. For each missing value $x_m$ in F:

• Identify all non-missing values $\{x_1,x_2,...,x_n\}$ within F.

• Compute the mean of these values:

  $$\mathrm{M}\mathrm{e}\mathrm{a}{\mathrm{n}}_{\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{u}\mathrm{t}\mathrm{e}\mathrm{d}}=\frac{\sum _{i=1}^n{x}_i}{n}$$

• Replace each $x_m$ with $\mathrm{M}\mathrm{e}\mathrm{a}{\mathrm{n}}_{\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{u}\mathrm{t}\mathrm{e}\mathrm{d}}$.

This method assumes that the data is missing at random, which provides a reasonable approximation for the missing values based on the existing data, thereby reducing the potential bias introduced by missing values and improving the robustness of subsequent analysis and modeling. However, not all of these features significantly contribute to the network intrusion detection process, as some provide limited useful information regarding attacks. To ensure that only the most relevant features are preserved for efficient NID analysis, we performed a careful selection of features utilizing advanced feature extraction approaches, which are covered in the following sections. The selected attributes are presented in Table 1.

Feature selection

Feature selection is a critical step to improve the performance of the intrusion detection system. We employ a combination of Best-First Search, Particle Swarm Optimization (PSO), Evolutionary Search, and Genetic Search to select the most relevant features from the dataset. The technique of feature extraction involves reducing the dimensionality of a dataset by choosing or manipulating significant variables, typically enhancing model performance and computing efficiency while maintaining vital information for analysis or modelling tasks. This hybrid approach ensures that the selected features maximize the classification accuracy while maintaining the computational efficiency of the model. The selected features are then used to construct the decision forest. The overall methodology for feature selection is presented in Fig. 3. Using each of the previously mentioned searching strategies, a different amount of features is picked. Then the ending process is used, in which the features are chosen by majority vote.

Dataset overview

We utilized the UNSW-NB15 dataset for training and testing our models, which is widely recognized in the field of network intrusion detection. This dataset includes a comprehensive collection of network traffic data that represents both normal activities and various types of intrusions, making it an ideal benchmark for evaluating the performance of intrusion detection systems (IDS). The UNSW-NB15 dataset contains 49 features, including packet-based and flow-based attributes, ensuring a robust and versatile resource for developing and evaluating IDS models. For our study, the dataset was divided into 70% for training and 30% for testing. This split allowed for a substantial amount of data to be used in the training phase, where the model identified patterns and correlations within the data to develop a predictive model. The training subset included both benign traffic and multiple categories of attack traffic, enabling the model to learn the distinguishing characteristics of each. Following training, the model was tested using the testing subset to evaluate its performance and ability to generalize to new, unseen data. In addition to the proposed model, various other models were evaluated as benchmarks to provide a comprehensive evaluation of their effectiveness in achieving the study objectives (Moustafa & Slay, 2015, 2016; Moustafa, Creech & Slay, 2017).

The UNSW-NB15 dataset is critically important due to its comprehensive and diverse collection of network traffic data, which provides a balanced representation of real-world network conditions. Unlike other datasets that may focus solely on specific types of attacks or normal traffic, UNSW-NB15 includes modern attack types such as DoS, fuzzers, analysis, backdoors, exploits, generic, reconnaissance, shellcode, and worms. This diversity ensures that models trained and tested on UNSW-NB15 are well-prepared to handle contemporary security challenges. Additionally, the dataset’s extensive documentation and established benchmarks facilitate reproducibility and comparison across different studies, enhancing its value to the research community. By using the UNSW-NB15 dataset, researchers and practitioners can develop models that are more accurate, reliable, and capable of generalizing to real-world network environments, thereby advancing the state-of-the-art in network security (Moustafa, Slay & Creech, 2017; Sarhan et al., 2021).

Evaluation criteria

In the evaluation of models, the training and testing phases were executed using a 70–30 dataset split. This evaluation encompasses a thorough comparison between the proposed model and a selection of bench-marked machine learning models, which are AbM1 (Subasi & Kremic, 2020), J48 (Maulana & Defriani, 2020; Ortega et al., 2020; Posonia, Vigneshwari & Rani, 2020), KNN (Alroobaea, 2020; Zhang & Li, 2021), LMT (Verma, Yadav & Monia, 2022; Sujal, Nanthini & Reddy, 2022; Bhoyar et al., 2021), MLP (Asiri et al., 2022; Tolstikhin et al., 2021; Yu et al., 2022), NB (Alroobaea, 2020) and lastly, SGD (Toğaçar, Ergen & Cömert, 2020; Stich & Karimireddy, 2020; Upadhyay et al., 2020). The few evaluation criteria are given below:

Precision: The precision of the model is determined by dividing all of its positive predictions by the percentage of true positive rate. It evaluate the model’s accuracy in identifying positive instances without mistakenly labeling negative instances as positive. The precision formula is as follows:

(3) $$Precision=\frac{TP}{TP+FP}$$

Recall: Recall is a statistic used in classification tasks to assess a model’s capacity to properly identify positive cases from the total number of actual positive examples in the dataset. It is also referred to as sensitivity or true positive rate. Mathematically, recall is calculated as:

(4) $$Recall=\frac{TP}{TP+FN}$$

F-measure: A statistic known as the F1-score merge accuracy and recall into just one score, hence creating a balance between the two. It is especially effective when the dataset has an unequal class distribution (class imbalance). The formula for the F1-score is:

(5) $$F-Measure=\frac{2\ast Precision\ast Recall}{Precsision+Recall}$$

Matthew’s correlation coefficient: The validity of binary classifications can be gauged using Matthew’s correlation coefficient (MCC). Its range is from −1 to 1. where 0 denotes a prediction that is no better than the random prediction, 1 denotes a perfect prediction, and −1 denotes a complete difference between the prediction and the observation. The formula for MCC is:

(6) $$MCC=\frac{TP\ast TN-FP\ast FN}{\sqrt{(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P})(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N})(\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P})(\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N})}}$$

Accuracy: Performance assessment by comparing a predictive model’s predictions to the actual values of the target variable in the dataset, accuracy describes how accurate or how often a prediction is accurate. This statistic is frequently employed to evaluate the overall efficacy of a model in producing accurate predictions. The formula of accuracy is:

(7) $$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$$

Results and comparative analysis

To validate the superiority of the “Opt-Forest” model, we compared its performance against several benchmark models, including AbM1, J48, KNN, LMT, MLP, NB, and SGD. In the process of analyzing outcomes obtained from different machine learning algorithms applied to the enhanced dataset containing clearly defined and distinct features, we prioritized identifying the most suitable intrusion detection algorithms, including a novel model. We investigated each of them for finding how much they are accurate to figure out network anomalies and normal activities which are the basic criteria to detecting network intrusions. Models are thoroughly analyzed under critical measurements which are key to ML and which are discussed and analyzed as follows: Firstly, we have calculated the confusion matrix as depicted in Table 2, for each of the algorithms which shows the accuracy of the models in terms of detecting normal activities as “normal” and anomalies as “anomaly” in network intrusion detection.

After analyzing the results of different machine learning algorithms, we expanded our research to include crucial metrics required for efficient intrusion detection. We investigated additional performance evaluation like precision, recall, and F1-score in addition to accuracy evaluations. These metrics provide more detailed information on how well the algorithms are able to distinguish between normal and abnormal network activities. Through close analysis of these metrics along with the confusion matrix results, we were able to obtain a thorough understanding of the effectiveness of each method in distinguishing between normal and malicious network activity. The foundation of our methodology for choosing the best intrusion detection algorithms for reliable network security is this kind of thorough assessment.

The algorithms with superior performance are detailed in Table 2. Among them J48 algorithm accurately recognized 4,038 occurrences of the “Normal” class as “Normal,” but incorrectly classed 21 examples as “Anomaly”. It accurately identified 3,462 occurrences as “Anomaly” but incorrectly classed 37 instances as “Normal” for the “Anomaly” class. LMT precisely recognized 4,034 instances of the “Normal” class as “Normal,” but incorrectly classed 25 examples as “Anomaly.” It accurately identified 3,464 occurrences of the “Anomaly” class as “Anomaly,” but erroneously classified 35 cases as “Normal.” The out-performer remained Opt-Forest with the most optimum results. It properly categorized 4,046 occurrences of the “Normal” class as “Normal,” but misclassified 13 instances as “Anomaly.” It accurately recognized 3,470 occurrences of the “Anomaly” class as “Anomaly,” but incorrectly classed 29 instances as “Normal.” The ensemble approach used during data features selection might well be the reason for its better performance.

We extracted results for various machine learning algorithms deployed on a refined dataset with well-distinguished and distinct features. Our focus was on identifying those most suitable for intrusion detection, which included AbM1, J48, KNN, LMT, MLP, NB, SGD, and Opt-Forest model. We investigated each of them for finding how much they are accurate to figure out network anomalies and normal activities which is the basic criteria to detecting network intrusions. Models are thoroughly analyzed under critical measurements which are key to ML and which are discussed and analyzed as: Firstly, we have calculated the confusion matrix as depicted in Table 2, for each of the algorithms which shows the accuracy of the models in terms of detecting normal activities as “normal” and anomalies as “anomaly” in network intrusion detection. The algorithms with better performance are discussed here. Among them, the J48 algorithm accurately recognized 4,038 occurrences of the “Normal” class as “Normal,” but incorrectly classified 21 examples as “Anomaly.” It accurately identified 3,462 occurrences as “Anomaly” but incorrectly classed 37 instances as “Normal” for the “Anomaly” class. LMT precisely recognized 4,034 instances of the “Normal” class as “Normal,” but incorrectly classed 25 examples as “Anomaly.” It accurately identified 3,464 occurrences of the “Anomaly” class as “Anomaly,” but erroneously classified 35 cases as “Normal.” The outperform remained Opt-Forest with the most optimum results. It properly categorized 4,046 occurrences of the “Normal” class as “Normal,” but misclassified 13 instances as “Anomaly.” It accurately recognized 3,470 occurrences of the “Anomaly” class as “Anomaly,” but incorrectly classed 29 instances as “Normal.” The ensemble approach used during data features selection might well be the reason for its better performance. Furthermore, our analysis extends past classification accuracy to consider each algorithm’s robustness and scalability in handling large-scale network datasets. We analyze computing requirements and scalability to ensure that the algorithm chosen is compatible with the organization’s infrastructure and operational requirements.

Performance evaluation is an important phase in machine learning since it allows you to examine the accuracy and efficiency of your model’s predictions. It assists you in determining how well your model is operating and if it satisfies the necessary goals. In this study where, the count of true-positive classification is denoted by TP, while the count of false-negative classification is presented by FN, TN is the count of true-negative classification, and FP is the count of false-positive classifications. In our assessment of each model’s performance, we focused on two crucial metrics, true positive rate (TPR) and false positive rate (FPR), illustrated in Fig. 4, two essential measures in ML, particularly in binary classification issues. These metrics are frequently used to assess the effectiveness of categorization models. TPR indicates how successfully your model distinguishes positive instances from the overall number of positive cases. The fraction of true negative cases that the model mistakenly labels as positive is measured by FPR. Opt-Forest remained an out-performer as it offered 0.994 TPR and a least 0.006 FPR values. This means it classified almost 100% correct occurrences for their true nature and misclassified them for only a fraction of 0.006%.

Furthermore, the Opt-Forest exceptional performance highlights how well it can distinguish between positive and negative instances, which makes it an effective option in intrusion detection systems. Its remarkably high TPR and quite low FPR figures show how reliable it is at accurately detecting real events while reducing false alarms. Whereas J48 and LMT results are satisfactory, their FPR values were slightly higher than those of the Opt-Forest, suggesting more likely to have for misclassification. This is quite remarkable as compared to other ML approaches evaluated J48 and LMT provided satisfactory results followed by AbM1 and KNN as shown in above Fig. 4.

A separate set of important measurements, including precision, recall, and F-measure, was also extracted for each model on the given dataset and is briefly discussed in the following sections. These metrics serve several functions and provide useful information about the performance of a classification model. Table 3 and Fig. 5 show the critical performance measures, such as precision, recall, and F-measure, for various models utilized in the context of NIDS. Notably, the “Opt-Forest” model consistently outperforms the other algorithms. Its exceptional accuracy (0.994) suggests a low rate of false positives, which is critical in avoiding false alerts for non-intrusive activities. The high recall (0.994) illustrates its capacity to detect real intrusions while reducing the likelihood of missing true threats. Furthermore, the F-Measure (0.994) exhibits a fair choice between precision and recall, confirming the model’s capacity to maintain a harmonic balance between false alarm reduction and intrusion detection. “Opt-Forest” is most certainly benefiting from decision trees’ ensemble nature, utilizing several models to improve accuracy and resilience. This thorough and expert examination highlights the “Opt-Forest” model’s supremacy in NIDS, making it an appealing choice for intrusion detection due to its great overall performance and harmonious balance of precision and recall.

Models deployed in this study were investigated for another important ML parameter known as MCC. This is illustrated in Fig. 6. MCC is an acronym for Matthew’s Correlation Coefficient, and It’s a machine learning metric for evaluating how well binary classification models work. Proposed method yielded the most optimum results of 0.989 followed by J48 and LMT with 0.985 and 0.984 respectively. The overall accuracy is a popular indicator for assessing the performance of a classification model. It calculates the percentage of properly identified cases throughout the full dataset. Figure 7 presented the accuracy analysis of each employed model compared with the proposed model. Proposed method outperformed its counterparts in terms of accuracy by achieving 99.45%, followed by J48 and LMT gaining 99.23% and 99.21%, respectively. KNN achieved an overall accuracy of 98.72% followed by AbM1 with 93.37%. Opt-Forest algorithm clearly shows its effectiveness here as this model used an ensemble technique for collecting and analyzing features from the given dataset and this could be the reason behind its top performance.

Percentage difference (PD) is used for extracting model performance comparison (MPC) and finding the algorithms which perform closer to the proposed model. This gives a clue that the models having minimum PD with the proposed model are also suitable and can also be deployed at a similar job. Figure 8 shows PD of the proposed model with all other tested models. PD score of Opt-Forest with J48 is 0.22% and with LMT it is 0.24%. These PD scores give indications that J48 and LMT can also be deployed to attain satisfactory results in the same task. PD scores of other algorithms like NB, SGD, and AbM1 were high and hence cannot be recommended for achieving satisfactory results. Furthermore, PD scores also provide beneficial indicators for finding algorithms that closely match the effectiveness of the proposed method. Via examining PD scores of the various algorithms, researchers can gain insights into which methods offer comparable efficiency in addressing the task at hand. In the context of our study, the relatively low PD scores for J48 and LMT are potential or viable alternatives to the proposed model. However, it’s essential to conduct further analysis to assess the robustness and generalizability of these findings across diverse datasets.

Threats to validity

Threats to validity encompass factors that can undermine the accuracy, generalizability, or reliability of research findings.

• 1.

  External validity: External validity is critical since it pertains to the generalizability of this research. While the UNSW-NB15 dataset provides a modern foundation, it is important to acknowledge its potential limitations in capturing the full spectrum of network behaviours, as different datasets, network configurations, and industry-specific contexts can result in distinct traffic patterns and intrusion behaviours. As a result, the findings may not apply generally to networks with dramatically different profiles or designs, emphasizing the importance of cautious interpretation and consideration of context-specific differences in network traffic patterns.

• 2.

  Internal validity: The integrity of the experimental design and data analysis techniques is important to the internal validity of this study. Particular attention should be paid to any biases or mistakes introduced during data preparation, such as managing missing values and feature selection since these variables might impact the model’s accuracy and validity of the study’s. Furthermore, the effect of parameter and hyper-parameter settings on machine learning model performance, including the “Opt-Forest” model, highlights the importance of explicit documentation of these settings in order to ensure the robustness and replicable of the given findings.

• 3.

  Construct validity: Construct validity is concerned with variable measurement and manipulation. In this study, it is crucial to recognize that feature engineering, like choosing and selecting features, have a major influence on the input data for the models. Variations in model performance might result from different feature engineering decisions. The exact collection of characteristics chosen is critical in defining the outcomes and, as a result, the construct validity of the study.

• 4.

  Data quality: The accuracy of the data utilized is critical to the research’s credibility. The UNSW-NB15 dataset, albeit more recent in this investigation, may still have limits and potential data quality concerns. Inaccuracies or biases in the dataset have the potential to have a major impact on the study results. Recognizing these constraints and resolving any data quality concerns is critical for maintaining the research’s credibility.

Limitations and future work

The proposed “Opt-Forest” model demonstrates substantial potential for improving network intrusion detection, but several limitations remain. The model’s reliance on high-quality training data means that any biases or gaps in the dataset can adversely impact detection accuracy. Furthermore, the computational complexity of combining multiple optimization techniques, such as genetic algorithms, particle swarm optimization, and evolutionary search, may result in longer training times and require substantial processing power, which could limit its use in resource-constrained environments. Additionally, the model’s effectiveness against entirely novel attack patterns not represented in the training data may be limited, and its performance in real-time scenarios could be affected by network latency and other operational constraints.

Expanding on these limitations, several promising avenues for future research in Network Intrusion Detection Systems (NIDS) emerge. Leveraging advanced machine learning techniques like transfer learning and reinforcement learning could enhance the model’s adaptability to new threats while reducing false positives. Developing larger, more diverse datasets that reflect modern network traffic patterns is also crucial for improving the robustness of intrusion detection algorithms. Additionally, optimizing NIDS deployment in evolving architectures, such as cloud and edge computing, may lead to more effective security solutions. Lastly, future work should prioritize ethical and legal considerations, including privacy and data protection compliance, to ensure responsible NIDS usage. Addressing these areas will enhance the flexibility, efficiency, and ethical integrity of NIDS in safeguarding digital ecosystems.