Employing SAE-GRU deep learning for scalable botnet detection in smart city infrastructure

Significant contributions

The primary objective of this research is to push the boundaries of IoT security through the introduction of a novel deep learning architecture. The proposed Stacked Autoencoder–Gated Recurrent Unit (SAE-GRU¹ ) model aims to enhance both the accuracy and efficiency of intrusion detection within networks. This article details the design, implementation, and deployment of the proposed model in a smart city context to furnish a comprehensive evaluation of its performance across several metrics to validate its effectiveness. Hereby, the main contributions of proposed research are listed below:

1. Introduced a hybrid deep learning model, SAE-GRU, that combines Stacked Autoencoders for dimensionality reduction² and gated recurrent units for temporal pattern recognition to detect botnet activities in smart city IoT networks.

2. Implemented model pruning and weight quantization techniques to reduce computational complexity and improve the efficiency of the intrusion detection system without compromising accuracy.

3. Developed an emulation environment replicating a smart city network using diverse IoT devices and communication protocols, which enabled comprehensive testing of the proposed system.

4. Integrated sparse matrix multiplication and batch processing to optimize the inference process to minimize computational overhead and guarantee real-time detection capabilities.

5. Applied truncated backpropagation through time in the GRU to manage sequential data processing effectively to effectively reduce latency in real-time applications.

6. Conducted rigorous performance evaluations using k-fold cross-validation and various datasets to demonstrate superior accuracy, precision, recall, and AUC values compared to traditional models.

7. Employed feature importance analysis techniques, including SHAP values to identify key features like packet size and traffic volume to contribute significantly to detection accuracy.

8. Demonstrated the model’s scalability and robustness in handling high data volumes and diverse network conditions to make it suitable for large-scale smart city implementations.

9. Highlighted the model’s resilience against zero-day attacks and its ability to adapt to evolving botnet strategies through advanced deep learning techniques.

10. Provided practical insights for future research by suggesting methods to enhance computational efficiency and extend the system’s capability to handle more complex IoT network environments.

The remainder of this article is organized to provide a detailed exploration of each component of the proposed IDS. Following this introduction, the article presents a review of related works to set the context for this study, followed by an explanation of fundamental concepts essential for understanding IoT botnet detection. The subsequent sections describe the proposed methodology, emulation setup, results, and an in-depth analysis of the key features identified during the study. The article concludes with a summary of findings and discusses their implications for future research and practical applications.

Vulnerability context for smart city networks

Smart city infrastructures are attractive targets for cyberattacks due to the convergence of critical services and interconnected devices that make it crucial to understand the potential attack vectors, targets, and impacts. Prominent attack vectors include DDoS which overwhelms systems with excessive traffic, Command and Control (C&C) exploits that manipulate devices for malicious activities, and Advanced Persistent Threats (APTs) which involve prolonged infiltration to compromise network security and disrupt essential services. These attacks often target critical infrastructure such as power grids and emergency services, aiming to cause significant disruptions and compromise public safety. Botnets, a key component in many attacks, are networks of compromised devices controlled by attackers. They can be structured in centralized, decentralized, or hybrid architectures, each influencing their resilience and control mechanisms. Malware employed in these attacks includes polymorphic malware that changes its code to evade detection, and techniques like cross-layer exploits that target multiple layers of a system. Infection vectors, such as vulnerability exploitation, supply chain compromise, and social engineering, are used to infiltrate devices. Attackers also utilize defense evasion techniques to conceal their malicious activities.

Challenges in IoT botnet detection

Our investigation revealed several limitations and challenges, one of the major challenges in IoT botnet detection is the extreme heterogeneity of IoT devices, which introduces a wide variety of communication protocols, processing capabilities, and security vulnerabilities across networks. Each IoT device can behave differently depending on its configuration which proves its’ eligibility in creating a highly diverse attack surface that complicates the modeling of normal and malicious behavior. Another significant challenge arises from the resource constraints inherent in IoT devices, such as limited computational power, memory, and energy. These limitations obstruct the deployment of advanced security measures such as encryption and real-time anomaly detection. As a result, IDS must depend on lightweight solutions that maintain a balance between detection accuracy and computational efficiency. IoT devices generate a substantial volume of network traffic due to their continuous operation. This results in massive data streams that must be processed and analyzed in real time. It was evident that high data throughput demands real-time detection capabilities, which places a significant computational load on detection systems. These systems must operate efficiently to handle the traffic without causing latency. A further challenge lies in obtaining labeled datasets necessary for training machine learning models. IoT botnets often exhibit subtle behavioral changes that are difficult to distinguish from legitimate traffic. Herewith, the absence of comprehensive labeled datasets limits the effectiveness of supervised learning methods in detecting emerging threats. Likewise, modern IoT botnets add to this complexity by frequently adapting their attack strategies to evade detection mechanisms. Techniques such as encryption, polymorphism, and traffic obfuscation make IDS ineffective when relying solely on signature-based or traditional anomaly detection methods.

Traditional IDS limitations

Traditional IDS face several limitations (Hajiheidari et al., 2019; Khraisat & Alazab, 2021; Najafli, Haghighat & Karasfi, 2024) when applied to modern IoT environments. For instance:

1. It lacks the capability to manage the evolving and diverse nature of IoT devices and protocols. This results in significant gaps in detecting targeted and complex attacks.

2. High false positive rates are a common issue due to rigid rule-based detection methods that fail to adapt to varying traffic patterns.

3. These systems often exhibit poor scalability in large-scale IoT environments. This leads to performance degradation and delays in threat detection.

4. Limited real-time analysis capabilities arise because majority of traditional IDS rely on batch processing methods, which are unsuited for the continuous flow of IoT data.

5. Resource-constrained IoT devices are incompatible with traditional IDS due to the high computational demands of these systems.

6. Signature-based detection methods in traditional IDS cannot identify novel or previously unknown threats effectively.

7. Contextual awareness is insufficient in traditional IDS. They fail to recognize the holistic behavior of IoT devices within their operational settings, resulting in incomplete assessments of threats.

8. Integration challenges occur with IoT-specific communication protocols and encryption mechanisms, leading to undetected vulnerabilities in traffic analysis.

Features of ideal DL architectures for intrusion detection

Our investigation revealed the following advantages of optimum DL for intrusion detection (Najafli, Haghighat & Karasfi, 2024; Muneer et al., 2024), which includes, but not limited to:

1. Ability to extract features from raw network traffic directly from data without requiring manual feature engineering. This ensures suitability for high-dimensional IoT datasets.

2. The capability to identify and interpret intricate patterns within data that enable detection of sophisticated and evolving attack strategies is often undetected by traditional methods.

3. High adaptability to diverse IoT devices and communication protocols. This ensures robustness across heterogeneous network environments.

4. Scalability that allows effective handling of large data volumes. This makes these methods ideal for real-time intrusion detection in smart city networks.

5. Potential to significantly reduce false positives by identifying subtle traffic deviations. This improves alerting precision and operational effectiveness.

6. Resilience against adversarial tactics through advanced modeling of non-linear relationships and the integration of diverse data sources.

7. Ability to perform well even with limited labeled data by employing semi-supervised or unsupervised approaches. This addresses challenges in scenarios where labeled datasets are scarce.

Applied preprocessing and optimization techniques

To ensure the integrity, efficiency, and effectiveness of the proposed intrusion detection framework, a combination of data preprocessing and model optimization techniques was employed throughout the design and training phases. Each technique was selected to address specific challenges associated with high-dimensionality, data sparsity, computational efficiency, and model generalization in considered network environment. The following definitions provide concise explanations of the methods utilized within this study to support feature representation, model training, and real-time inference:

• → Min-max normalization: A scaling technique that transforms numerical features into a fixed range, typically [0, 1], to ensure uniform contribution to the model’s learning process.

• → Mutual information and variance thresholding techniques: Feature selection methods used to retain informative attributes and eliminate low-variance or weakly relevant features which usually leads to reducing redundancy in high-dimensional IoT traffic data.

• → Imputation techniques: Methods applied to handle missing data by substituting null values with statistical estimates, such as mean for numerical and mode for categorical features, to maintain dataset consistency.

• → Feature scale normalization: A transformation approach that adjusts the magnitude of numerical features to a common scale, preventing dominant attributes from skewing model training.

• → One-hot encoding: A categorical encoding scheme that converts non-numeric variables into binary vectors which allow the models to process protocol types and device states without assuming ordinal relationships.

• → Weight decay: A regularization technique that penalizes large weight values during training to prevent overfitting and promote model generalization in noisy IoT environments.

• → L2 regularization: A specific form of weight decay that adds the squared magnitude of weights to the loss function to discourage complexity and improve robustness.

• → Dropout and early stopping: Regularization strategies where dropout randomly disables neurons during training and early stopping halts training once validation performance ceases to improve and reduce overfitting.

• → Pruning optimization: A technique that removes non-contributing or weakly active neural connections to reduce model size and improve inference speed without compromising detection accuracy.

• → Weight quantization: The process of converting model weights from high-precision to lower-precision formats (e.g., 32-bit to 8-bit) to decrease memory footprint and accelerate computations.

• → Sparse matrix multiplication: An inference-time optimization that leverages zero-valued weight sparsity to skip unnecessary computations to enhance real-time performance on edge devices.

Data collection and preprocessing

Raw network traffic data were collected from a diverse set of IoT devices operating within the smart city infrastructure, including but not limited to smart meters, environmental sensors, traffic monitoring systems, and surveillance cameras. These devices were selected due to their heterogeneous communication protocols (i.e., includes but not limited to: Zigbee, Wi-Fi, LoRaWAN, RTSP (Real-Time Streaming Protocol), etc.,) and varying data generation rates, representing the complexity of an interconnected smart city ecosystem. The collection process spanned over a period of 4 weeks to ensure temporal diversity and capture data variations in network behavior. Data capture was conducted using network monitoring tools (i.e., Wireshark & TCPDUMP) capable of passive traffic analysis, which allowed for continuous recording without interfering with device operations. Packet sniffers (i.e., as illustrated in Fig. 1) were deployed at multiple network access points across the emulated smart city setup to capture all inbound and outbound traffic to ensure that the dataset reflected both normal and anomalous behavior from different sectors of the emulated city. This approach allowed for the monitoring of device-level communications, network congestion patterns, and potential cyber threats such as DDoS attempts, while maintaining data integrity. To ensure comprehensive representation, network traffic from high-traffic nodes such as public Wi-Fi hotspots was included alongside traffic from low-traffic IoT devices such as smart parking meters. This approach captured a broad spectrum of traffic patterns and device interactions. The collection framework incorporated various IoT protocols including MQTT, CoAP, and HTTP, which are integral to smart city infrastructure. Random sampling was applied across different time slots and network segments to avoid overrepresentation of any specific device type or network traffic pattern. In this context, we observed that data bias may arise from unequal representation of certain patterns or features within the dataset that can potentially lead to skewed or inaccurate model outcomes. Also, overrepresentation of traffic from high-traffic IoT devices like public Wi-Fi hotspots could hinder the detection of attacks targeting fewer common devices. Similarly, an imbalance in protocol representation, such as focusing on MQTT over HTTP was observed with the ability to reduce the model’s capacity to detect threats in underrepresented communication types.

In our investigation, data preprocessing was essential for maintaining the quality and usability of collected IoT network traffic data for intrusion detection. We realized the importance of preserving the integrity and statistical reliability of the dataset, and for this reason, the mean imputation was used for numerical features by replacing null entries with the arithmetic average of observed values, which preserved the statistical distribution and prevented distortion in features such as packet size, byte counts, and flow duration—attributes critical for distinguishing volumetric and behavioral anomalies in botnet detection.

To further strengthen the data preprocessing, we also applied Mode imputation to categorical features such as protocol type or device state, where the most frequent class was substituted to retain consistency in discrete attribute distribution. These strategies prevented information loss while maintaining model input uniformity. It is worth highlighting that for in botnet detection, numerical values are especially important as they capture subtle deviations in traffic volume, session timing, and data transfer behavior that often precede or accompany coordinated botnet activity, thus allowing the model to learn and identify patterns that are otherwise undetectable through categorical inspection alone. In accordance, the feature selection improved model efficiency by eliminating irrelevant attributes while preserving essential patterns in network behavior.

We also applied mutual information along with a fixed variance thresholding technique set at 0.01 to eliminate features exhibiting low variability across observations, as such features contribute minimal discriminative power and can introduce noise into the learning process. This dual-step approach enabled the selection of highly informative attributes that not only improve classification accuracy by focusing the model on relevant patterns but also reduce computational complexity by removing redundant or static input dimensions. The applied dataset covered various intrusion types, including DDoS attacks, Command and Control activities, traffic injection attempts, data exfiltration, spoofing events, malware propagation, credential harvesting, brute force attacks, firmware exploitation, and eavesdropping on communication channels.

The heterogeneous nature of IoT devices, as presented in Table 3, introduced variations in data formats, requiring specific methods to handle missing values. In this context, imputation techniques were applied to maintain dataset integrity, where mean imputation was used for numerical features and mode imputation for categorical attributes to address data sparsity. Whereas the feature scale normalization was applied to account for differences in magnitude, including packet sizes and traffic volumes. Min-max normalization adjusted feature values within a unified scale, typically between 0 and 1, which guaranteed consistency across the dataset and improving the effectiveness of model training.

(1) $$x_{norm}={\displaystyle \frac{x-x_{min}}{x_{max}-x_{min}}.}$$

As per Eq. (1), the $x_{norm}$ is normalized value, $x$ is the original feature value, $x_{min}$ & $x_{max}$ are the minimum and maximum values of the feature, respectively. This normalization is crucial in preventing features with larger ranges from dominating the training process.

To convert categorical variables, such as protocol types, into numerical formats, one-hot encoding (Rezvan et al., 2024) was employed. This method transformed each categorical feature into a binary vector, where each category was represented as a distinct dimension that has allowed the model to interpret the categorical information without imposing artificial ordinality. The choice of one-hot encoding was particularly relevant given the non-hierarchical nature of many categorical features in IoT traffic, where no intrinsic ranking exists between different protocols or device types.

We selected preprocessing algorithms such as the Pandas and Scikit-learn libraries due to their proven efficiency and scalability. These qualities were critical for handling the extensive volume and high dimensionality of IoT data. The preprocessing steps (such as: Missing value imputation; Normalization of feature scales; One-hot encoding for categorical variables; Data cleaning; Standardization; Feature extraction; and Feature dimensionality reduction) standardized the dataset while eliminating inconsistencies. This process ensured a robust foundation for feature extraction and effective model training.

Rationale for employing stacked autoencoder

The adoption of SAEs in this research stems from their capacity to process high-dimensional data effectively while maintaining essential features. The emulated IoT environment generated extensive network traffic with a mix of significant and redundant information. Hereby, the SAE proved effective in this context due to its ability to uncover latent patterns such as recurring traffic bursts, synchronized connection attempts, and consistent session durations by encoding these into compact, information-rich representations (e.g., protocol activity footprint; connection uniformity pattern, etc.). Through dimensionality reduction, the autoencoder transformed correlated features: like ‘packet size, flow duration, and byte count’ into a unified latent variable capturing their combined variance. This abstraction enabled the model to detect subtle yet coordinated anomalies in communication behavior that are often obscured in the original high-dimensional feature space especially for those which are indicative of botnet activity. This compression process minimized noise and preserved critical characteristics without compromising analytical depth (i.e., feature importance analysis; performance metrics evaluation; temporal pattern recognition; model generalization assessment; real-time detection capability; false positive and false negative analysis; cross-validation; and dataset diversity evaluation). In our resource-constrained IoT system, this dimensionality reduction demonstrated its value by enabling computational efficiency and facilitating real-time analysis. The compressed data produced by SAEs reduced the computational demands of subsequent tasks, thereby it improved the overall performance of implemented intrusion detection model.

In the proposed emulated settings, SAE, the encoder function transforms the high-dimensional input data $x\in R^n$ into a lower-dimensional hidden representation $h\in R^m$. This procedure (i.e., as exhibited in Eq. (2)) was important to reduce redundancy and emphasize essential feature interactions which enabled the model to focus on patterns that are most relevant for distinguishing between normal and malicious traffic behavior.

(2) $$h=f\left(W_{e}x+b_e\right)$$where $W_e\in R^{m\times n}$ is the weight matrix, $b_e\in R^m$ is the bias term, and $f(\cdot )$ is the activation function, typically ReLU (Rectified Linear Unit). The hidden representation $h$ encapsulates the essential features in a compressed form. The decoder then attempts to reconstruct the input $x$ from $h$ using the Eq. (3):

(3) $$x^{\mathrm{\prime }}=f\left(W_{d}h+b_d\right)$$where $W_d\in R^{n\times m}$ and $b_d\in R^n$ are the decoder’s weight matrix and bias term, respectively. The objective of training the SAE is to minimize the reconstruction error, which measures the difference between the original input $x$ and its reconstruction $x^{\mathrm{\prime }}$. The reconstruction error is often expressed as the mean squared error (MSE) and is minimized as follows (i.e., expressed in Eq. (4)):

(4) $$L\left(x,x^{\mathrm{\prime }}\right)={\displaystyle \frac{1}{n}\sum _{i=1}^n(x_i-x_i^{\prime }{)}^2}$$where $L\left(x,x^{\mathrm{\prime }}\right)$ is the loss function, and $x$ and $x^{\mathrm{\prime }}$ represent the original and reconstructed values of the input data, respectively. It is worth noting that minimizing the MSE during the training of the SAE ensured that the reconstructed output closely matches the original input for preserving critical traffic patterns. Lower MSE indicates that the encoder has captured the most informative features needed to differentiate normal from botnet-induced anomalies. Thus, an encoder compresses input data into a lower-dimensional space and a decoder reconstructs the original input. The objective is to retain the most relevant features while discarding noise which facilitates efficient pattern recognition in high-dimensional datasets.

The SAE was trained in an unsupervised manner by optimizing the loss function using the gradient-based optimization algorithm ‘Adam’ (Reyad, Sarhan & Arafa, 2023). This algorithm was selected for training the SAE because it combined the benefits of both ‘AdaGrad and RMSProp (i.e., are the optimization algorithms used to update model parameters during training)’ by adaptively adjusting learning rates for each parameter using first and second moment estimates of gradients. This made it well-suited for handling sparse features and non-stationary objectives common in high-dimensional IoT traffic data. Regularization technique, weight decay or $L2$ regularization, was applied to the weight matrices during training to mitigate overfitting. Our investigation revealed that this approach is particularly critical when handling large and noisy datasets that are common in IoT networks. Weight decay introduced a penalty to the loss function based on the magnitude of the weights. This penalty helped prevent the model from becoming overly complex and improved its ability to generalize to unseen data. The applied loss function with $L2$ regularization is expressed in Eq. (5) as:

(5) $$L_{reg}\left(x,x^{\mathrm{\prime }}\right)=L\left(x,x^{\mathrm{\prime }}\right)+\lambda \parallel W{\parallel }_2^2$$where $\lambda$ is the regularization parameter and $\parallel W{\parallel }_2^2$ is the squared $L2$ norm of the weight matrices.

After training the SAE, the encoder component processed high-dimensional input data to generate lower-dimensional feature representations. These compact feature vectors retained the most critical information for subsequent analysis while minimizing computational complexity in later stages of intrusion detection. This reduction in dimensionality enhanced computational efficiency and concentrated on the most crucial patterns which eventually led to improved accuracy in identifying malicious activities.

Underlying principle for exercising gated recurrent unit network

The GRU network was implemented to detect temporal patterns (e.g., repeated connection intervals, synchronized packet bursts, periodic data uploads, uniform session durations, cyclic protocol switching, timed beacon signals, consistent idle-to-active transitions, recurring port access sequences, repetitive handshake patterns, scheduled command transmissions) in IoT network traffic data. Herewith, the structured gated recurrent units handled sequential data (i.e., time-stamped network flows, session-wise traffic logs, packet arrival sequences, protocol exchange orders, connection attempt histories, authentication sequences, port scanning timelines, device communication intervals, traffic burst timings, malware propagation traces) efficiently by maintaining long-term dependencies while reducing computational demands. Temporal patterns and sequential data were important because they captured the timing and order of network activities, which are critical for identifying behaviors characteristic of botnet operations. Such patterns revealed coordinated actions, delayed triggers, and repetitive access sequences that static feature analysis could not detect.

Our comprehensive evaluation showed that GRUs address vanishing gradient challenges that frequently affect traditional recurrent neural networks. This property strengthens their effectiveness in real-time intrusion detection for IoT environments. Cross-validation confirmed the capability of GRUs to capture sequential dependencies with greater stability, whereas conventional recurrent networks struggled to retain long-range information. Thus, this implementation enabled the emulated system to learn patterns efficiently and identify both normal and malicious activities within the traffic. Normal activities include regular data transmission intervals, periodic updates, scheduled maintenance tasks, consistent communication behaviors, synchronized sensor readings, uniform packet sizes, and steady bandwidth utilization. Whereas, the malicious activities encompassed sudden surges in traffic volume, irregular packet timings, anomalous data flows, unscheduled device interactions, inconsistent protocol behaviors, rapid data bursts, significant packet size deviations, coordinated surges across multiple devices, repeated connection attempts to external servers, frequent contact with command-and-control servers, brute force login attempts, high-frequency port scans, periodic data exfiltration bursts, prolonged DDoS attacks, unexpected increases in device-to-device communication, synchronized anomalies from geographically dispersed devices, abnormal encryption patterns, frequent traffic destination changes, and suspicious outbound connections to unverified domains. We have also observed that the GRUs require fewer parameters than LSTMs, demonstrated reduced computational overhead, which was vital in environments with limited resources. This efficiency and robust ability to process sequential dependencies established GRUs as an ideal solution for intrusion detection within the proposed architecture.

In our emulated settings, the functionality of a GRU (i.e., as evident from Fig. 1) is controlled by two primary gates: the update gate and the reset gate, which govern how much past information is passed to the future. Herewith, the ‘update gate’ determines how much of the previously observed network behavior (e.g., normal or suspicious traffic patterns) is carried forward for further analysis in the detection process. For botnet detection, this helped track long-term anomalies, such as coordinated attacks that evolve over time. Whereas the ‘reset gate’ decides how much of the past information should be ignored, enabling the model to focus on immediate and relevant traffic patterns. In our applied detection, this allowed the GRU to disregard irrelevant or benign variations, such as normal fluctuations in traffic, while concentrating on identifying emerging botnet behavior.

Accordingly, the update gate, denoted as $z_t$, determines the extent to which the hidden state from the previous time step $h_{t-1}$ should be carried forward. This phenomenon is represented in Eq. (6):

(6) $$z_t=\sigma \left(W_z{x}_t+U_z{h}_{t-1}+b_z\right)$$where $W_z$ and $U_z$ are the weight matrices for the input $x_t$ and the previous hidden state $h_{t-1}$, respectively, and $b_z$ is the bias term. The sigmoid function $\sigma$ ensures that the values of $z_t$ remain between 0 and 1, effectively controlling the weight of the previous hidden state in the current computation.

The reset gate, denoted as $r_t$, is responsible for deciding how much of the previous hidden state should be forgotten. As stated earlier, this gate plays a critical role in allowing the GRU to selectively reset portions of its memory when modeling sequential dependencies, as exhibited in Eq. (7):

(7) $$r_t=\sigma \left(W_r{x}_t+U_r{h}_{t-1}+b_r\right).$$

Here, $W_r$ and $U_r$ are the weight matrices, and $b_r$ is the bias associated with the reset gate. By modulating the reset gate, the GRU can ignore irrelevant parts of the past sequence when they are not useful for future predictions. With this, as exhibited in Eq. (8), the candidate activation, denoted as $\widetilde{h_t}$, represents the new memory content to be added to the hidden state. This candidate is computed based on both the current input and the reset hidden state, which allows the GRU to conditionally forget or remember parts of the sequence:

(8) $$\widetilde{h_t}=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}(W_h{x}_t+U_h\left(r_t\odot h_{t-1}\right)+b_h.$$

Herewith, the $W_h$, $U_h$, and $b_h$ are the corresponding weights and biases for the candidate activation, and $\odot$ represents element-wise multiplication. The reset gate $r_t$ ensures that only relevant historical information influences the new candidate activation. Ultimately, the hidden state at time step $t$ is updated as a linear interpolation between the previous hidden state and the candidate activation, controlled by the update gate:

(9) $$h_t=\left(1-z_t\right)\odot h_{t-1}+z_t\odot \widetilde{h_t}.$$

Equation (9) determines the final output of the GRU unit at each time step by balancing the influence of the new candidate activation $\widetilde{h_t}$ with the previous hidden state $h_{t-1}$, depending on the value of the update gate $z_t$. This interpretation allowed GRUs to adaptively retain relevant information over long time sequences which no doubt proved to be a critical feature for identifying patterns of malicious activity within sequential IoT network traffic.

Herewith, the Table 4 provides an algorithmic and structured breakdown of the SAE-GRU-based intrusion detection workflow by detailing each step from data preprocessing to real-time threat classification. This formal representation enhances the clarity of the proposed methodology.

Temporal pattern recognition in IoT network traffic

The compressed feature vectors generated by the SAE were passed as input into the GRU network to model temporal dependencies³ . These lower-dimensional feature vectors, which represented condensed and denoised versions of the original high-dimensional network traffic, encapsulated critical information relevant to identifying both normal and malicious activities. The GRU network processed these sequential inputs by maintaining a hidden state that captured information from previous time steps. This mechanism allowed it to learn long-term dependencies and temporal correlations within the data. Each feature vector in the input sequence passed through the update and reset gates to enable the GRU to determine how much past information should be retained or discarded at each time step. Through this modulation, the GRU dynamically adjusted its memory of past events, which was essential for distinguishing between normal traffic patterns and anomalies indicative of botnet activity. As the GRU iterated through the sequence of feature vectors, it identified patterns that signaled malicious behaviors such as coordinated spikes in traffic or abnormal data flows while filtering out routine variations in the network. The ability of the GRU to maintain context over extended periods enabled it to detect botnet activities that unfold over long time frames. This temporal modeling was crucial for differentiating between benign and malicious behaviors, as it accounted for both short-term fluctuations and long-term trends in order to ensure high accuracy in threat identification.

Supervised training process of the GRU network

The supervised training process of the GRU network was conducted using labeled data, where each sequence of input feature vectors derived from the SAE was associated with a corresponding class label indicating whether the traffic pattern represented normal behavior or malicious activity. The use of labeled data allowed the GRU to learn from ground-truth examples and distinguish between benign and anomalous patterns in network traffic. The cross-entropy loss function was employed during training to quantify the difference between the predicted class probabilities and the actual class labels. This loss function was suitable for binary classification tasks, such as detecting botnet attacks, and is represented in Eq. (10):

(10) $$L\left(y,\hat{y}\right)=-{\displaystyle \frac{1}{N}\sum _{i-1}^N[y_i\log \left({\hat{y}}_i\right)+(1-y_i)\mathrm{l}\mathrm{o}\mathrm{g}\left(1-{\hat{y}}_i\right)]}$$where $y_i$ is the true label, ${\hat{y}}_i$ is the predicted probability, and $N$ represents the number of samples. Herewith, we applied the Adam optimizer to compute individual adaptive learning rates for each parameter and combines the benefits of both the AdaGrad and RMSProp algorithms (as exhibited in Table 5), ensuring stable convergence even in complex, high-dimensional spaces.

To enhance the generalization capability of the GRU network and prevent overfitting, two techniques were applied: Dropout and Early Stopping. Dropout was introduced during training by randomly setting a fraction of the units in the hidden layers to zero at each iteration. This forced the network to learn redundant representations and reduced its dependency on specific neurons. The dropout rate was carefully selected to achieve a balance between underfitting and overfitting, typically ranging from 0.3 to 0.5 for optimal performance. Early Stopping was employed as a regularization method that monitored the model’s performance on a validation set and halted training when the validation loss stopped improving. This ensured that the GRU network stopped training at its optimal state and avoided overfitting the training data. Our observation revealed that by applying these strategies, the GRU network achieved high accuracy and robustness in detecting anomalies across diverse and complex IoT traffic patterns and demonstrated strong generalization to unseen data during deployment.

Architecture of the combined SAE-GRU model

From the preceding discussion, it is evident that we have effectively conceptualized & favorably designed optimal SAE-GRU model to efficiently process high-dimensional IoT network traffic data and extracted temporal dependencies for effective intrusion detection. Herewith, at this stage, the SAE acted as a dimensionality reduction layer to reduce noise and compressed the raw high-dimensional input data into lower-dimensional feature vectors. These feature vectors are then passed into the GRU network, which models the temporal relationships between the sequences of traffic data. We designed the SAE as a three-layer architecture: an input layer, a hidden layer, and an output (reconstruction) layer. The number of hidden units in each layer was empirically determined, with the hidden layer of the encoder having 128 units to balance expressiveness and computational efficiency. The rectified linear unit (ReLU) was used to introduce non-linearity and enhance the model’s ability to learn complex patterns. The GRU component comprised two layers of recurrent units, each with 64 hidden units, and uses the tanh activation function⁴ , which provided stability in gradient propagation, reduced the risk of exploding or vanishing gradients, especially when processing long sequences. The choice of this architecture ensured that the model could capture both spatial and temporal features in network traffic data while remaining computationally feasible for real-time deployment.

The deployment of the trained SAE-GRU model for real-time intrusion detection in the emulated network was achieved using Python-based machine learning libraries (i.e., TensorFlow), which supported efficient implementations of deep learning architectures. The model was integrated into the existing network monitoring system through RESTful APIs to enable real-time data stream input and analysis. Real-time inference was optimized by batching incoming network packets and processed them through the SAE for compression followed by GRU-based temporal analysis. The system operated on a cluster of edge computing nodes equipped with moderate GPU support, which distributed the computational workload of IoT traffic processing across multiple nodes. This hardware configuration ensured that the model fulfilled the strict low-latency requirements of real-time anomaly detection.

The mechanism for anomaly detection relied on analyzing incoming data streams and comparing them to normal behavior patterns learned during training. Once a sequence of compressed feature vectors passes through the GRU, the model outputs a score representing the likelihood of the sequence being normal or malicious. Herewith, the Thresholding method was applied to classify network behavior. A decision threshold $T$ is empirically set by analyzing the Receiver Operating Characteristic (ROC) curve during validation. The value of $T$ is chosen to balance the trade-off between false positives and false negatives, minimizing the total classification error. If the output score $s\left(x\right)$ exceeds $T$, the behavior is classified as malicious. This is represented in Eq. (11):

(11) $$Class=\{\begin{array}{cc}Malicious,\hfill & ifs\left(x\right)>T\\ Normal,\hfill & ifs\left(x\right)\le T.\end{array}$$

The model also incorporated a mechanism to handle false positives and false negatives by maintaining an alert threshold buffer, which reduced the likelihood of triggering false alarms due to benign fluctuations in network traffic. Upon detection of suspicious patterns indicative of botnet activity, the SAE-GRU model automatically triggered a sophisticated alert mechanism that was designed to efficiently inform network administrators and initiate response protocols. The alert mechanism was integrated directly into the IDS in order to ensure swift and effective responses.

Herewith, when the SAE-GRU model identifies an anomaly that exceeds the predefined decision threshold $T$, it triggers an alert generation process. This process involves the formation of an alert packet that includes detailed information about the detected anomaly, such as the time of detection, affected network segments, and a risk assessment score. This packet is then communicated to the IDS through a secure communication channel, ensuring that the information is relayed promptly and securely to the network administrators.

The implemented alert mechanism included an automated logging system that records every detected event into a centralized log database. This database is structured to store comprehensive details of all alerts, facilitating subsequent analysis and forensic investigations. The stored data includes timestamps, sensor IDs, the type of detected anomalies, severity levels, and the actions taken in response to the alerts. This logging is crucial for tracking the effectiveness of the detection system, auditing system responses, and refining detection strategies over time.

The applied IDS was programmed to parse incoming alerts and categorize them based on severity. Depending on the severity and the specific characteristics of the detected anomaly, predefined mitigation strategies are automatically initiated. These strategies include, but are not limited to, autonomic reconfiguring firewalls to block malicious traffic, segmenting parts of the network to isolate compromised devices, and deploying additional monitoring to the affected areas/zones. The system employs a rule-based decision algorithm, as outlined in Table 6, to determine the appropriate response based on the risk assessment score and the type of anomaly detected.

With this, each action was logged with a corresponding entry in the incident management system, which included timestamps, the nature of the response, and status updates on the resolution of the issue. This system not only ensured immediate attention to potential threats but also allowed the designated network administrators to review and adjust the automated responses based on effectiveness and evolving network security requirements.

Model optimization

In our emulated assessment, optimizing the SAE-GRU model was crucial to ensure efficient performance while maintaining detection accuracy. One of the primary optimization techniques applied was model pruning. Pruning involves systematically removing less important neurons or connections from the model, which reduced the overall complexity without significantly compromising its ability to detect anomalies. By identifying and eliminating parameters that contributed minimally to the output during the training phase, we reduced both the storage requirements and inference time. This approach was particularly effective for the GRU component, where certain recurrent connections were pruned based on their contribution to the loss function.

Weight quantization was another technique applied to reduce the computational load. This method involves converting high-precision floating-point weights into lower-precision formats, such as 8-bit integers. While this results in a minimal loss of precision, it significantly decreases the memory footprint and increases the speed of inference on edge devices that often lack powerful processing units. In this consideration, the ‘Quantization-aware training⁵ ’ was used to account for the reduced precision during the training phase which guaranteed that the model remained robust despite the compressed representation of weights.

To further enhance efficiency, we implemented the Sparse matrix multiplication algorithm (i.e., as exhibited in Fig. 2) designed to minimize computational overhead. Specifically, sparse matrix operations were employed during inference, allowing the model to bypass unnecessary computations associated with zero-valued parameters after pruning. This was complemented by batch processing, where multiple incoming data streams were processed simultaneously to utilize the parallelism to reduce latency. For temporal data processing in the GRU, we employed truncated backpropagation through time (TBPTT) to limit the number of sequential time steps during training. This reduced the computational demand by restricting the depth of the network’s temporal memory to make it suitable for real-time applications where long sequences would otherwise introduce delays.

To further elaborate on the applied optimization techniques, pruning in the proposed model was not executed as a one-time static reduction but dynamically adjusted during training using a gradient sensitivity score. Parameters with persistently low gradient contributions across epochs were identified and removed, especially within GRU gates where inactive pathways were filtered out to enhance information flow. For weight quantization, the approach involved a mixed-precision strategy where core arithmetic layers operated on 8-bit integers while retaining 16-bit accumulators in sensitive layers to preserve gradient fidelity. This reduced computational overhead without undermining convergence behavior. Herewith, the Sparse matrix multiplication was applied with structure-aware indexing where zero-valued weights were skipped using pre-computed mask arrays to permit optimized memory access patterns and cache-friendly execution. Applied technique exploited sparsity resulting from pruning to accelerate matrix-vector operations in the recurrent layers. These optimizations were tailored for execution on edge devices with constrained compute profiles so that the model would become eligible to meet latency constraints under real-world traffic load.

For accurate representation, it is essential to convey that the Sparse matrix multiplication was not merely used as a computational convenience but was structurally embedded into inference routines to minimize multiplications involving zero-valued weights post-pruning. This choice directly reduced processing latency, particularly during peak network loads, where real-time response is critical. Correspondingly, quantization-aware training allowed floating-point weights to be compressed into lower-bit representations while retaining learning capacity through adjusted gradient computations. By simulating inference-time quantization during training, the model achieved reliability, consistency, and robustness in edge deployments. We also tailored the model pruning by using a relevance-based thresholding mechanism, which systematically removed parameters with minimal impact on classification loss, especially within the GRU gates where redundant recurrence was detected. The model employed ‘truncated backpropagation through time’ to handle long sequences efficiently to enable temporal learning with reduced memory consumption. We also applied the batch processing across parallel IoT streams which further exploited hardware concurrency that allowed the model to meet sub-second decision latency.

The cumulative effect of optimization technique ensured that the SAE-GRU model could be deployed in metropolitan IoT environments with limited processing and memory resources, while it is still capable of providing real-time, high-accuracy anomaly/malware/intrusion detection. By balancing computational efficiency and detection performance, the model effectively addressed the unique challenges posed by IoT-based smart city networks, where rapid and lightweight processing is paramount for maintaining network security.

Performance evaluation

As it is evident from the preceding section’s discussion, the evaluation methodology used to assess the performance of the SAE-GRU model was rigorous and multifaceted. It aimed to provide a thorough understanding of its effectiveness in detecting IoT botnet activities. For evaluation, we employed metrics such as accuracy, precision, recall, F1-score, and the area under the receiver operating characteristic curve (AUC). These metrics were selected for their ability to measure the correctness of the model’s predictions while balancing the detection of true positives against the minimization of false positives⁶ . Thus:

1. Precision measured the proportion of true positive detections among all positive predictions, ensuring that false positives (benign traffic misclassified as malicious) are minimized.

2. Recall, on the other hand, measured the proportion of actual attacks that were correctly detected, which is critical for ensuring that no malicious behavior goes undetected.

3. The F1-score provided a harmonic mean between precision and recall which offered a balanced metric in scenarios where both false positives and false negatives are of concern.

4. The area under the ROC curve (AUC) was used to evaluate the model’s ability to distinguish between normal and malicious traffic over a range of thresholds that provided a more nuanced view of the classifier’s performance.

Hence, the accuracy of the model is defined as Eq. (12):

(12) $$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$where $TP$ represents true positives $TN$ true negatives, $FP$ false positives, and $FN$ false negatives. Accuracy is a simple metric to assess overall performance but can be misleading in imbalanced datasets. To address this, precision is defined as Eq. (13):

(13) $$Precision={\displaystyle \frac{TP}{TP+FP}.}$$

Recall, which reflects the proportion of true positives among all actual positives, is expressed as: Eq. (14):

(14) $$Recall={\displaystyle \frac{TP}{TP+FN}.}$$

The F1-score combines precision and recall into a single metric, especially useful in scenarios where both false positives and false negatives carry significant costs. It is defined as:

(15) $$F1-score=2\times {\displaystyle \frac{Precision\times Recall}{Precision+Recall}.}$$

Eventually, the AUC measures the model’s ability to distinguish between classes (malicious vs. normal traffic) across different thresholds, with values closer to ‘1’ indicating better performance.

Hereby, Table 7 presents the performance evaluation of the proposed model using ten-fold cross-validation to demonstrate its effectiveness in detecting IoT botnet activities. The results indicate that the model achieves consistently high accuracy, precision, recall, and F1-score across multiple iterations that is indicative of its robustness in differentiating between normal and malicious network traffic. The low false positive rate confirms the model’s ability to minimize erroneous classifications, which is essential for real-world deployment.

To establish a comprehensive and meaningful evaluation of the proposed SAE-GRU architecture, reference models Hazman et al. (2024) to Shareef et al. (2024) were carefully selected based on their methodological diversity, real-world applicability, and relevance to the evolving demands of intrusion detection in IoT-centric smart city environments. These approaches represent state-of-the-art strategies that utilize advanced deep learning, optimization algorithms, and hybrid techniques to address the complex landscape of IoT security. Models such as LSTM-AE with feature engineering (Hazman et al., 2024) and CANFIS-MDRL (Almasri & Alajlan, 2023) incorporate temporal modeling and adaptive inference, which are directly comparable to the GRU component of our architecture, making them suitable benchmarks for assessing temporal learning efficacy. Similarly, frameworks that use bio-inspired optimization—such as FGOA-kNN (Taher et al., 2023), BBO-ERNN (Manickam et al., 2023), and ZOA-DGAN (Shareef et al., 2024)—emphasize dimensionality reduction and adaptive feature selection, aligning closely with the SAE’s role in our model. These models were evaluated using comparable datasets (e.g., BoT-IoT, NSL-KDD, N-BaIoT, and IoT-23), which supports direct comparison across performance metrics like accuracy, recall, and AUC.

A 360-degree (i.e., all inclusive, as exhibited in Table 7, and Figs. 3 to 8) performance evaluation was conducted using cross-validation technique to ensure the robustness and generalization of the SAE-GRU model. A k-fold cross-validation approach was applied, where the dataset was divided into $k$ subsets, and the model was trained on $k-1$ folds while the remaining fold was used for testing. This process was repeated k times to average the performance across all subsets to provide a comprehensively reliable estimate of the model’s effectiveness. The model was tested on various datasets, including synthetic botnet traffic generated during the emulation and real-world IoT botnet datasets (i.e., described in Table 2). The results were compared against baseline models, such as traditional machine learning-based IDS and deep learning models like LSTM networks, to validate the superior performance of the SAE-GRU model. For instance:

Figure 3 illustrates the accuracy distribution across various machine learning models for intrusion detection which highlights their performance variations. Among the models, SAE-GRU exhibits the highest median accuracy with minimal variance that indicates its robust and consistent performance. Other models demonstrate broader accuracy ranges, reflecting higher sensitivity to data variability or parameter configurations. The SAE-GRU model achieves better outcome due to its integration of Stacked Autoencoders for efficient feature extraction, gated recurrent units for capturing temporal patterns, and robust training strategies that enhance generalization while minimizing noise, overfitting, and computational complexity.

The evaluation of accuracy, precision, recall, and F1-score in Fig. 4 emphasized the robustness of SAE-GRU over alternative methodologies. By focusing on feature compression and temporal pattern recognition, SAE-GRU demonstrated reduced variance across performance metrics which indicates higher generalizability in real-world IoT systems.

Figure 5 highlighted the trade-offs between computational efficiency and scalability to showcase the ability of SAE-GRU to maintain optimal performance in resource-constrained environments through its dimensionality reduction and parallel processing capabilities. The scatter plot revealed that traditional models experienced significant scalability bottlenecks, whereas SAE-GRU effectively balanced throughput and real-time processing demands. Here, it is worth noting that the threshold values for classification were determined through empirical tuning using the receiver operating characteristic curve analysis where the optimal threshold was selected based on the highest true positive rate with the lowest false positive rate. A paired t-test (i.e., comparison of the means of two related datasets to determine if there is a statistically significant difference between them) was conducted to compare the SAE-GRU model against baseline models, which demonstrated a statistically significant improvement in detection accuracy with a p-value below 0.05.

Insights into the sensitivity and specificity of models were revealed by mapping (i.e., as illustrated in Fig. 6) false positive and false negative rates. The SAE-GRU model achieved low false classification rates through its adaptive gating mechanisms that prioritized critical traffic patterns. This approach ensured accurate anomaly detection within the complexity of heterogeneous IoT traffic.

Figure 7 underscored the model’s adaptability across diverse datasets, with the SAE-GRU architecture achieving high AUC values irrespective of dataset-specific traffic characteristics. Its superior performance on datasets like IoT-23 (Garcia, Parmisano & Jose Erquiaga, 2020) and MedBIoT (Guerra-Manzanares et al., 2020) confirmed its resilience to evolving attack vectors and rare traffic anomalies. The selection of IoT-23 and MedBIoT datasets was guided by their comprehensive representation of real-world IoT botnet attacks, diverse attack vectors, and protocol-specific traffic patterns. For example, IoT-23 provides labeled network traffic from various IoT malware families that allow for a structured evaluation of intrusion detection techniques. The dataset includes normal and malicious traffic, which aids in training deep learning models for anomaly detection. Whereas MedBIoT dataset contains a large collection of botnet traffic from multiple IoT devices simulating real-world infection scenarios with different attack intensities. The inclusion of applied datasets strengthened the model’s ability to generalize across varying threat landscapes. Their diverse attack signatures and device heterogeneity made them well-suited for evaluation (i.e., validating the performance of the proposed SAE-GRU model).

Figure 8 illustrated the multi-dimensional evaluation of performance metrics in a consolidated manner, with the radar chart providing a holistic view of the SAE-GRU’s capability to outperform baseline models across all critical dimensions. This analysis demonstrated that SAE-GRU’s integration of stacked autoencoders and GRUs not only reduced noise in high-dimensional data but also captured intricate sequential patterns that were often overlooked by other models. By ensuring high detection accuracy while maintaining efficiency, SAE-GRU emerged as a suitable solution for large-scale IoT networks.

Feature importance analysis

Feature importance analysis was conducted to identify which features contributed most significantly to the model’s decisions. This was achieved through feature ablation studies, where individual features were systematically removed from the dataset, and the model’s performance was re-evaluated. A substantial drop in performance upon the removal of a particular feature indicated its importance in the detection process. Herewith, interpretability technique, SHAP (SHapley Additive exPlanations), values were employed to provide a more granular understanding of feature contributions. SHAP values quantified the impact of each feature on the model’s output by attributing the change in prediction to each feature. This approach provided a deeper understanding of how specific network traffic characteristics influenced the detection of botnet activities. The insights from this analysis revealed that features such as packet size, traffic volume, and protocol usage held the most significant impact on detection accuracy. These findings offered valuable information for refining the intrusion detection system in future iterations.

Comparative evaluation with recent IDS models

As evident from Figs. 3–8, in order to validate and contextualize the practical contributions of the SAE-GRU model, a comparative evaluation was conducted against recently proposed intrusion detection frameworks from relevant literature (e.g., LSTM (Hazman et al., 2024), CANFIS (Almasri & Alajlan, 2023), FGOA_KNN (Taher et al., 2023), ERNN (Manickam et al., 2023), AdaBoost (Hazman et al., 2023), DGAN (Shareef et al., 2024), etc.). The comparison considered fundamental metrics including accuracy, precision, recall, and F1-score across prominent intrusion datasets such as BoT-IoT (Alosaimi & Almutairi, 2023), Edge-IIoT (Nuaimi et al., 2023), and NSL-KDD (Zakariah et al., 2023). Unlike conventional models that employed singular methods such as basic recurrent neural networks or static feature extraction, our approach uniquely integrated stacked autoencoders with GRU layers for robust temporal dependency handling. While recent methodologies primarily relied on generalized pruning or naive quantization, the proposed SAE-GRU model adopted gradient-sensitivity-based pruning coupled with mixed-precision quantization, thus optimizing model compactness and real-time detection capability for edge computing scenarios. Compared to reported accuracy levels between 96% and 98.7% (Hazman et al., 2024; Almasri & Alajlan, 2023; Taher et al., 2023; Manickam et al., 2023; Hazman et al., 2023; Ahmed, Beyioku & Yousefi, 2024; Shareef et al., 2024), proposed SAE-GRU model exhibited strong performance across key evaluation metrics, with high accuracy (98.65%), precision (98.11%), recall (98.43%), F1-score (98.27%), and AUC (0.993) which reflects its effectiveness in identifying IoT botnet activity while maintaining low false positive and false negative rates. Its hybrid design—combining SAEs for dimensionality reduction and GRUs for temporal pattern recognition—enabled consistent classification and improved generalization. Optimization methods contributed to efficient real-time processing and reduced resource consumption. Unfortunately, the model’s dependence on labeled data may limit its adaptability to novel or evolving threats in completely unknown traffic scenarios. Furthermore, its layered structure, though streamlined, still imposes a moderate computational load that can hinder deployment in extremely resource-limited IoT environments.

Recommendations for future studies

Future studies may focus on enhancing computational efficiency, improving adaptability to zero-day attacks, and integrating decentralized architectures to strengthen resilience while evaluating the model’s applicability across industrial IoT and healthcare systems. Investigating how the SAE-GRU model performs under varying network loads, diverse device configurations, and sector-specific security challenges will provide deeper insights into its generalizability and optimization for real-world deployments.