Machine learning for Internet of Things (IoT) device identification: a comparative study

Decision tree

DT is a supervised learning technique that can be used for both classification and regression tasks. It creates a tree structure that captures each decision taken in the classification or regression as a split in the dataset through nodes and branches. Each internal node is a decision rule, each branch is an outcome, and each leaf node is the final class label or prediction (Shafiq et al., 2021). At each step of the process, the best feature is selected based on Gini impurity, entropy, or information gain out of the many features available, which ensures that each the resulting subsets are as pure as possible. One of the main advantages of DT is their interpretability, as they mimic human decision-making processes by providing a clear visual representation of how predictions are made. Additionally, they are computationally efficient and do not require extensive data processing. However, DTs have the major problem of overfitting, which is when an excessively deep trees capture noise rather than general patterns. This is often solved by pruning the trees, combining multiple DTs in the ensemble to increase accuracy and generalization, or setting a maximum depth. Although not without issues, DT is widely used for IoT devices classification, medical diagnosis, fraud detection, and many other areas where simplicity and effectiveness in dealing with struct is required.

Random forest

RF is a supervised learning algorithm that combines multiple trees through the idea of ensemble learning. An RF model uses the bagging (bootstrap aggregating) method, where each tree is fitted on a random sample of the data, and the final outcome is decided by majority voting (for classification) or averaging (for regression). In addition, RF employs feature randomness by using a random subset of features for every split, which reduces overfitting and enhances generalization. These combination techniques help the RF outperform a single DT by making it less sensitive to problems of high variance, noise, or overfitting. An RF also efficiently manages imbalanced datasets as well as numerical and categorical features. However, its main drawback is higher computational complexity, as training multiple trees requires more processing time and memory compared to a single DT. In IoT device recognition, the models use network traffic features as their communication behaviors for classification purposes. RF can be seen as an ideal candidate due to its high accuracy, noise resistance, and mastery of variable network traffic patterns (Mahesh, 2018).

Support vector machine

SVM, is one of the most often used supervised learning techniques, which is mostly utilized for multi-class and binary classification (Shafiq et al., 2021). SVM can be utilized for used for both classification and regression tasks, and it is particularly effective in high-dimensional spaces. For the classification purpose, it divides the feature space into sub-spaces separated by planes known as hyper-planes. During this division, an “optimal hyperplane” is chosen based on two criteria: it should best separate classes and also maximizes the margin, also known as the distance, between the neighboring data points, or support vectors. Basically, a wider margin ensures a more proficient model. More “non-linearly separable data” poses more challenge, however, it also showcases SVM’s strength, through its unique ability to apply different kernel functions, linear, polynomial, and radial basis function (RBF) etc. These serve a purpose of transforming the input space into a higher dimensional one and makes it possible for different correspondence classes to become distinguishable. The impressive feature allows SVM to be powerful in computer networking traffic classification, intrusion detection and recognition of IoT devices where complex decision boundaries are essential. Kernel-based methods often show strong classification results, however they come with a downside. The model can be computationally expensive, especially for larger datasets where solving the algorithm becomes a quadratic optimization problem. Nevertheless, SVM remains a widely used and reliable classifier in IoT security applications due to its ability to handle high-dimensional feature spaces and its robustness to overfitting.

Neural networks

A NN is a mathematical or computer model that simulates the structure and function of a biological neural network. The model is composed of three parts: the input layer, the hidden layer, and the output layer. Each layer consists of interconnected neurons that process and transmit information. The connections between neurons are weighted, and these weights are adjusted during training to optimize the model’s performance. The neural network structure used in this study takes various network traffic features as input and processes them through multiple layers to classify different IoT devices. NNs can be employed for both supervised and unsupervised learning tasks. This study utilized a supervised neural network variant. In a supervised NN, the actual output is known in advance. The predicted output is compared to the actual result, and the network parameters are adjusted based on the error. This process is repeated iteratively to improve the accuracy the model (Mahesh, 2018).

Filter-based method

Filtered-based methods are preprocessing tools that evaluate the relevance of each feature independently by ranking features based on certain statistical criteria. The top-ranked features are ten chosen to be used in the ML classifiers. These types of methods are simple, fast, and model-independent.

• (a)

  Pearson correlation coefficient

Pearson correlation coefficient is a statistical measure that evaluates the linear relationship between two continuous variables. The range of values for PCC is from −1 to +1, where −1 indicates a perfect negative linear correlation, 0 indicates no linear correlation, and +1 indicates a perfect positive linear correlation.

The Pearson correlation coefficient ( $\rho$) is calculated by dividing the covariance of the two variables by the product of their standard deviations. The formula to calculate $\rho$ is shown in Eq. (1).

(1) $$\begin{array}{c}{\rho }_{X,Y}={\displaystyle \frac{cov\left(X,Y\right)}{{\sigma }_X\cdot {\sigma }_Y}}\end{array}$$where $cov$ is the covariance, ${\sigma }_X$ is the standard deviation of $X$ and ${\sigma }_Y$ is the standard deviation of $Y$.

When two or more features have a high absolute PCC value (e.g., |r| > 0.9|r| > 0.9|r| > 0.9), they provide nearly the same information. Keeping both features is unnecessary as one can be removed without significant information loss. PCC aids in dimensionality reduction by identifying and removing highly correlated (redundant) features.

• (b)

  Mutual information

MI is a measurement of interdependence or association of random variables. It measures the amount of information that can be inferred from one variable by observing the values of another. Mathematically, the MI between two discrete random variables X and Y can be defined as:

(2) $$\begin{array}{c}I\left(X,Y\right)=\sum _{x,y}{P}_{XY}\left(x,y\right)\log {\displaystyle \frac{P_{XY}\left(x,y\right)}{P_X\left(x\right)P_Y\left(y\right)}}\end{array}$$where $P_{XY}\left(x,y\right)$ is the joint probability distribution function of X and Y, and $P_X\left(x\right)$ and $P_Y\left(y\right)$ are the marginal probability distribution functions of $X$ and $Y$, respectively. MI quantifies the amount of information one variable provides about the other, with higher mutual information indicating a stronger relationship between the variables. When the two variables are independent, the mutual information is low, reflecting a lack of shared information. Mutual information is computed between each feature and the target variable (device type), capturing the relevance of each feature in identifying the target variable. By selecting the top-k features with the highest mutual information, the dataset’s dimensionality can be reduced while retaining the most informative features for the ML model. While MI measures the mutual dependence between variables which captures both linear and non-linear relationships, PCC measures the linear relationship between variables.

Wrapper-based method

Unlike the filtered-based types where features are chosen independently, in the wrapper-based approach, features are selected based on data properties and their direct performance on the specific classifiers. Since features are selected dependently, this approach directly optimizes the model and is able to provide superior results compared to filter methods.

However, due to their iterative nature, wrapper-based selection methods require significantly more processing time and computational resources than their counterparts, making them less suitable for real-time or resource-constrained applications. While the improved accuracy is beneficial for IoT device identification, users must consider the trade-off between accuracy and computational efficiency when selecting the features.

• (a)

  Binary Genetic Algorithm

The genetic algorithm is a heuristic optimization algorithm inspired by the process of natural selection and genetics. When used for FS, GA optimizes subsets of features to enhance the performance of a classification model by simulating genetic operations such as selection, crossover, and mutation.

Firstly, an initial population of chromosomes, each representing a feature subset, is generated randomly. Each chromosome can be represented as a binary vector where 1 indicates the feature selected and 0 indicates the feature is not selected. Then, each chromosome’s fitness is evaluated using the fitness function which measures the performance of the feature subset in the classification task. Based on fitness values, chromosomes are selected as parents for generating the next generation. The selected parent chromosomes undergo crossover to produce new offspring chromosomes. And offspring chromosomes undergo mutation with a certain probability, where specific genes are randomly flipped. This increases genetic diversity and prevents premature convergence. Then, the new generation of chromosomes replaces the less fit chromosomes from the previous generation. The process is repeated until a stopping criterion is met, such as a predetermined number of iterations or no significant improvement in fitness values.

• (b)

  Particle Swarm Optimization

The PSO algorithm draws inspiration from the social behaviour observed in bird flocking or fish schooling. In this algorithm, each particle corresponds to an individual within the swarm and is characterized by a subset of features, serving as its position in the search space. Initially, a set of particles, representing feature subsets is randomly generated. Subsequently, the velocity and position of each particle are initialized. The fitness of each particle’s position is evaluated using a fitness function, determining the global best position. During each iteration, the velocity and position of each particle are updated based on its personal best position and the global best position until either the maximum number of iterations is reached or there is no significant improvement in the global best fitness value. The velocity update equation is depicted as:

(3) $$\begin{array}{c}{v}_i\left(t+1\right)=\omega \cdot v_i\left(t\right)+c_1\cdot r_1\cdot \left({p_i}^{best}-x_i\left(t\right)\right)\\ +c_2\cdot r_2\cdot \left(g^{best}-x_i\left(t\right)\right)\end{array}$$where, $v_i\left(t\right)$ represents the velocity of particle $i$ at time $t$, $x_i\left(t\right)$ denotes the position of particle $i$, $p_i^{best}$ is the best position experienced by particle $i$, $g^{best}$is the best position experienced by all particles, $\omega$ denotes the inertia weight, $c_1$ and $c_2$ are the learning factors, $r_1$ and $r_2$ are random numbers between 0 and 1. In the binary space, the position of the particle consists of 1s and 0s, representing the selection and non-selection of features. Thus, the velocity undergoes a logistic transformation $s\left(x\right)$ as shown in Eq. (5), to represent the probability of bit $x_{id}$ taking the value 1. The position of the particle is then updated based on Eq. (4).

(4) $$\begin{array}{c}{\mathrm{x}}_{\mathrm{i}}\left(\mathrm{t}+1\right)=\{\begin{array}{c}1,ifrand\left(\right)<s\left({\mathrm{v}}_{\mathrm{i}}\left(\mathrm{t}+1\right)\right)\\ 0,otherwise\hfill \end{array}\end{array}$$

(5) $$\begin{array}{c}\mathrm{s}\left(\mathrm{x}\right)={\displaystyle \frac{1}{1+{\mathrm{e}}^{-\mathrm{x}}}}\end{array}$$

• (c)

  Binary Grey Wolf Optimizer

The Grey Wolf Optimizer (GWO) is a metaheuristic optimization algorithm that is inspired by the hunting behaviour of wolves in the wild (Mirjalili, Mirjalili & Lewis, 2014).

The wolves are categorized as alpha ( $\mathit{\alpha })$, beta ( $\mathit{\beta }$), delta ( $\mathit{\delta }$), and omega ( $\mathit{\omega }$) based on their dominance and leadership. The $\mathit{\alpha }$ is the leader and represents the best solution found, $\mathit{\beta }$ is the best candidate and represents the second-best solution, $\mathit{\delta }$ is on the third level and represents the third best solution, while $\mathit{\omega }$ represents the rest wolves and in the lowest level.

Firstly, the wolves are randomly distributed in the search space and represent candidate solutions, and these positions are updated iteratively to improve the quality of the solutions.

The position of a grey wolf during the hunt is embedded using the mathematical model provided by the following equations.

(6) $$\begin{array}{c}\overrightarrow{X}\left(t+1\right)={\overrightarrow{X}}_p\left(t\right)+\overrightarrow{A}\times \overrightarrow{D}\end{array}$$

(7) $$\begin{array}{c}\overrightarrow{D}=\left|\overrightarrow{C}\times {\overrightarrow{X}}_p\left(t\right)-\overrightarrow{X}\left(t\right)\right|\end{array}$$where $\overrightarrow{X}$ is the position of a grey wolf, $t$ means the number of the current iteration, ${\overrightarrow{X}}_p$ is the position of the prey, $\overrightarrow{D}$ is defined in Eq. (2), $\overrightarrow{A},$ $\overrightarrow{C}$ are generated using the Eqs. (8) and (9).

(8) $$\begin{array}{c}\overrightarrow{A}=2\times \overrightarrow{a}\times {\overrightarrow{r}}_1-\overrightarrow{a}\end{array}$$

(9) $$\begin{array}{c}\overrightarrow{C}=2\times {\overrightarrow{r}}_2\end{array}$$where ${\overrightarrow{r}}_1$ and ${\overrightarrow{r}}_2$ are randomly selected from [0,1], $\overrightarrow{a}$ is linearly reduced from 2 to 0, which is defined as:

(10) $$\begin{array}{c}\overrightarrow{a}=2-2\times {\displaystyle \frac{t}{t_{max}}}\end{array}$$where the maximum number of iterations is expresses as $t_{max}$.

Usually, the α drives the hunting and β and δ may also engage in hunting. The hunting behaviour of the grey wolves, alpha, beta, and delta, is anticipated to have more information about the location of the prey for the purposes of mathematical modelling.

After the first three candidate solutions are successfully completed, the remaining search agents adjust their positions to match the best search positions of the positions of agents. Equation (11) shows the updated position:

(11) $$\begin{array}{c}\overrightarrow{X}\left(t+1\right)={\displaystyle \frac{{\overrightarrow{X}}_1+{\overrightarrow{X}}_2+{\overrightarrow{X}}_3}{3}}\end{array}$$where ${\overrightarrow{X}}_1,{\overrightarrow{X}}_2,{\overrightarrow{X}}_3$ are defined in Eqs. (12), (13), and (14) respectively.

(12) $$\begin{array}{c}{\overrightarrow{X}}_1=\left|{\overrightarrow{X}}_{\alpha }-{\overrightarrow{A}}_1-{\overrightarrow{D}}_{\alpha }\right|\end{array}$$

(13) $$\begin{array}{c}{\overrightarrow{X}}_2=\left|{\overrightarrow{X}}_{\beta }-{\overrightarrow{A}}_2-{\overrightarrow{D}}_{\beta }\right|\end{array}$$

(14) $$\begin{array}{c}{\overrightarrow{X}}_3=\left|{\overrightarrow{X}}_{\delta }-{\overrightarrow{A}}_3-{\overrightarrow{D}}_{\delta }\right|\end{array}$$

Here, ${\overrightarrow{X}}_{\alpha },{\overrightarrow{X}}_{\beta },{\overrightarrow{X}}_{\delta }$ are the first three best solutions at iteration t, and ${\overrightarrow{D}}_{\alpha },{\overrightarrow{D}}_{\beta },{\overrightarrow{D}}_{\delta }$ are defined as follows.

(15) $$\begin{array}{c}{\overrightarrow{D}}_{\alpha }=\left|{\overrightarrow{C}}_1\times {\overrightarrow{X}}_{\alpha }-\overrightarrow{X}\right|\end{array}$$

(16) $$\begin{array}{c}{\overrightarrow{D}}_{\beta }=\left|{\overrightarrow{C}}_2\times {\overrightarrow{X}}_{\beta }-\overrightarrow{X}\right|\end{array}$$

(17) $$\begin{array}{c}{\overrightarrow{D}}_{\delta }=\left|{\overrightarrow{C}}_3\times {\overrightarrow{X}}_{\delta }-\overrightarrow{X}\right|\end{array}$$

In this study, a binary variable from the grey wolf optimizer is utilized for the FS problem. To find a practical solution when BGWO is being executed, the formula for computing the new position, as stated in Eq. (11), is altered, as stated in Eq. (18).

(18) $$\begin{array}{c}Sig\left(\overrightarrow{X}\left(t+1\right)\right)={\displaystyle \frac{1}{1+e^{-\overrightarrow{X}\left(t+1\right)}}}\end{array}$$

The possibility of taking the decision variables is indicated by the value of $Sig\left(\overrightarrow{X}\left(t+1\right)\right)$, as seen in Eq. (19), where r is a randomly chosen number between 0 and 1.

(19) $$\begin{array}{c}\overrightarrow{X}\left(t+1\right)=\{\begin{array}{cc}1& ifr<Sig\left(\overrightarrow{X}\left(t+1\right)\right)\\ 0& otherwise\hfill \end{array}\end{array}$$

In the BGWO a fitness function is required to measure how well a potential solution to a problem performs based on a set of criteria. In the context of FS, it evaluates the effectiveness of a feature set in accurately classifying a target instance. Since we want to find the feature set which has the highest classification accuracy and a minimum number of selected features. The fitness function used in BGWO to evaluate the feature set used to identify IoT devices using supervised learning algorithms is shown below:

(20) $$\begin{array}{c}fitnessvalue=\alpha \cdot accuracy+\left(1-\alpha \right)\cdot {\displaystyle \frac{\left|N_o-N\right|}{\left|N_o\right|}\alpha \in \left[0,1\right]}\end{array}$$where $accuracy$ is the classification accuracy defined in Eq. (21), the ${\mathit{N}}_{\mathit{o}}$ is the length of full feature set before selection, $\mathit{N}$ is the length of the selected feature set, $\mathit{\alpha }$ is the weight of classification accuracy relative to the evaluation criteria and is set to 0.99 in this article.

Accuracy of IoT device identification

Given a dataset comprising IoT device network traffic, the goal is to develop a supervised ML algorithm to accurately identify each device based on its unique device behavior (network traffic). The dataset is separated into a training set and a test set. The training set contained labels used to train the model, and the test set is used to test the classification performance of the model. After the ML model is trained by the labeled training set, it will be used to predict the device label for the label-removed test set. The classification performance of the ML model is measured by accuracy, which is simply a ratio of correctly predicted observations to the total observations and is defined as follows.

(21) $$\begin{array}{c}Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}.}\end{array}$$

True/False (T/F) indicates whether the predicted result is right or wrong. Positive/negative (P/N) represents whether the label of the predicted class is correct or incorrect. Thus, true positive (TP) represents the number of correctly classified positive instances, false positive (FP) represents the number of incorrectly classified negative instances, true negative (TN) represents the number of correctly classified negative instances, and false negative (FN) represents the number of incorrectly classified positive instances.

Number of selected features

FS is a critical task in ML, which aims to select the most informative subset of features that can improve the performance of models. The importance of FS lies in the fact that not all features are equally relevant or useful for the given problem. Using all available features may result in overfitting, while using too few features may lead to underfitting.

The number of selected features can range from the minimum to the maximum of the entire device traffic, which is the number of features extracted from the network traffic. Utilizing all the features may cause computational complexity due to the curse of dimensionality, and irrelevant features may influence the classification accuracy. On the other hand, selecting a few features may result in a loss of information and low accuracy. Therefore, selecting the optimal subset of features is essential to achieving optimal model performance.

The FS problem can be defined as follow. Select a new subset of features $f^{\mathrm{\prime }}=\left\{{f^{\mathrm{\prime }}}_{n_1},{f^{\mathrm{\prime }}}_{n_2},{f^{\mathrm{\prime }}}_{n_3},\dots {f^{\mathrm{\prime }}}_{n_j},\dots {f^{\mathrm{\prime }}}_{n_{F0}}\right\}$ from a set of features $f=\left\{f_1,f_2,f_3,\dots ,f_{F_o}\right\}$. Where $F$ is the number of selected features and $F_o$ is the number of features in the full feature set before selection. In this article, $F_{o1}=188$ for the first dataset ( $d_1$), and $F_{o2}=30$ for the second dataset ( $d_2)$. Since a set containing $F_o$ features can generate $2^{F_o}$ distinct subsets, which is $3.92\times {10}^{56}$ subsets for $d_1$ and $1.07\times {10}^9$ subsets for $d_2$. Therefore, considering a very small dataset containing 1 million records, the total number of distinct subsets for $d_1$, and $d_2$ are $3.92\times {10}^{62}$ and $1.07\times {10}^{15}$ respectively. This is time-consuming to find the best solution, and randomly choosing is not likely to capture the underlying relationships within the data. The main goal of FS is to minimize the number of relevant sets for training and test purposes. Therefore, a heuristic algorithm can give us a feasible solution at an acceptable cost.

In this article, we applied two FS types including filter-based and wrapper-based algorithms. Moreover, a metaheuristic optimizer is used and compared with several other FS methods to identify the best feature subset for minimizing the cost and maximizing the accuracy.

Pearson correlation coefficient-based

We determined the Pearson correlation coefficient (PCC) between each feature and used five distinct cutoff points (0.5, 0.6, 0.7, 0.8, and 0.9) to eliminate features. Every feature whose PCC was higher than the cutoff threshold was eliminated. The dimensionality of the dataset decreased after the features with strong correlation coefficients with other variables were eliminated since those variables might represent the features with high correlation coefficients. The average outcome of employing the PCC in datasets 1 and 2 is displayed in Fig. 2. The number of selected features utilizing the associated boundary is indicated by the purple bar, and the accuracy attained by applying the corresponding classifier and selected features is shown by the lines.

Findings using dataset 1, as shown in Fig. 2A, indicate that only the RF achieved the highest accuracy of 97.01% when the cutoff boundary was set to 0.7, resulting in 33 features being retained. The classification accuracy of the other three classifiers, such as DT, NN, and SVM increased as the cutoff boundary was raised. All of the classifiers reached their highest accuracy with a cutoff boundary of 0.9, retaining 67 features: DT achieved 95.25%, NN achieved 94.83%, and SVM achieved 94.59%. The rate of accuracy improvement gradually slowed as the PCC boundary increased, indicating that features with high PCC have a diminishing or even negative impact on accuracy. For instance, when the number of selected features in the RF classifier increased from 33 to 67, the classification accuracy decreased.

Similarly, in dataset 2, as shown in Fig. 2B, only the RF reaches the highest accuracy of 99.61% when the cutoff boundary is equal to 0.7. Other methods exhibit a better performance in terms of classification as the cutoff border and number of features chosen increase. This is because the majority of the attributes in dataset 2 do not correlate with one another, and the dimensionality is not overly large. As shown in Fig. 2B, 19 out of 30 features have a PCC lower than 0.5. The difference in accuracy caused by the different PCC boundaries is not significant.

Mutual information-based

Since the mutual information is calculated between two discrete variables, we initially transformed the continuous variables into discrete ones. Each continuous feature value was replaced by the interval to which it belonged after the range of each continuous feature was equally divided into intervals corresponding to the number of devices in the dataset. Next, we determined the mutual information that exists between the target variable and the characteristics. In order to pick features, we examined five MI bounds (0.01, 0.1, 0.2, 0.3, 0.4, and 0.5) and kept those whose MI values were higher than the border. Figure 3 displays the outcomes of various MI borders.

The performance of all the algorithms improves as the MI boundary increases from 0.5 to 0.6, after which the rate of improvement slows down. The SVM and NN algorithms achieved their highest accuracies of 95.51% and 95.47%, respectively, with an MI boundary of 0.2. In contrast, the tree-based algorithms reached their highest accuracies with an MI boundary of 0.01, achieving 95.42% and 96.89% by DT and RF.

In Dataset 2, DT and RF classifiers demonstrate excellent and consistent performance regardless of the MI boundary selected. Their accuracies reach 98.95% for DT and 99.03% for RF with only six features selected. The accuracies of the SVM and NN classifiers steadily increase as the number of selected features grows. Both classifiers achieve their highest accuracy with an MI boundary of 0.01, with the SVM achieving 95% accuracy and the NN achieving 96.49% accuracy.

BGA-based

Figure 4 presents the results from 10 experiments using BGA as a FS method with different supervised learning algorithms.

The number of features selected by the BGA for the SVM classifier varies significantly across experiments, ranging from 80 to 160, as shown in Fig. 4. The classification accuracy achieved by SVM is from 95.47% to 96.88%. The accuracy improvement with BGA with a maximum difference of 1.52% and minimum difference of −0.41% indicates that in some cases, the accuracy was sacrificed for feature reduction. NN exhibits a relatively stable number of selected features, mostly between 85 and 100. The classification accuracy for NN ranges from 95.17% to 96.48%, and the accuracy improvement differs from 0.21% to 1.74%.

Meanwhile, DT classifier in dataset 1 consistently selects between 81 and 99 features. The accuracy remains high, from 95.65% to 97.33%, with an average accuracy of 96.44%. The difference in accuracy improvement is between 0.40% and 2.49%, indicating a significant benefit from FS. Similarly, RF in dataset 1 selects features within a narrow range of 81 to 102, averaging at 90.3. The classification accuracy is the highest among the classifiers, ranging from 96.98% to 98.13%, with an average accuracy of 97.65%. The accuracy improvement difference is small, between 0% and 1.24%, demonstrating robustness in performance with selected features.

In dataset 2 shown in in Fig. 5, the SVM classifier still shows a wider range in the number of selected features from 17 to 30, with an average of 24.5. The classification accuracy ranges from 95.5% to 96.78%, with an average of 96.49%. The accuracy improvement is not obvious, ranging from −0.22% to 0.22%. The NN for dataset 2 demonstrates FS variability from 13 to 30, averaging at 23.3. The accuracy ranges from 95.63% to 96.67%, with an average of 96.03%. The improvement difference ranges from 0.00% to 2.68%, indicating effective FS enhancing performance.

The DT classifier in dataset 2 maintains a narrow range of selected features, from 7 to 16, with an average of 13.7. It achieves exceptionally high accuracies between 99.29% and 99.73%, with minimal improvement differences ranging from −0.11% to 0.51%. Similarly, RF in dataset 2 consistently selects between 12 and 20 features, achieving accuracies from 99.51% to 99.84%. The improvement difference remains minimal, ranging from −0.06% to 0.38%, indicating stable performance with the selected feature set.

BPSO-based

In dataset 1, the features selected by BPSO for SVM classifier ranges from 83 to 100, with an average of 94.3 as shown in Fig. 6. The classification accuracy varies from 95.57% to 96.68%, averaging at 96.20%. The improvement in accuracy ranges from −0.21% to 1.52%, indicating that while FS can slightly decrease accuracy, it generally provides a moderate improvement.

In dataset 1, the NN using BPSO ranges between 84 and 104 features, with an average of 93.3. The accuracy ranges from 95.98% to 97.31%, with an average of 96.65%. The accuracy improvement difference ranges from 1.04% to 2.59%, showing a significant benefit from FS and consistently enhancing performance.

The DT classifier in dataset 1, employing BPSO, selects features within a narrow range of 82 to 102, averaging 93.6. The classification accuracy ranges from 95.24% to 97.09%, with an average of 96.61%. The accuracy improvement difference ranges from 0.40% to 2.91%, indicating substantial gains from FS.

The RF for dataset 1, using BPSO, consistently selects features between 77 and 102, averaging 89. The classification accuracy is high, ranging from 97.18% to 98.13%, with an average of 97.68%. The accuracy improvement difference is smaller, between 0.21% and 1.24%, demonstrating stable performance with the combination of BPSO and RF.

The NN in dataset 2 (Fig. 7), using BPSO, ranges from 14 to 30, averaging 18.9. The accuracy varies from 95.09% to 96.51%, with an average of 96.01%. The accuracy improvement difference ranges from −0.33% to 2.63%, suggesting that FS with BPSO can significantly enhance performance for NN, though there are some cases of slight decreases.

In dataset 2, DT with BPSO shows a narrow range of selected features, from 10 to 30, averaging 15.7. The accuracy is consistently high, ranging from 99.12% to 99.73%, with an average of 99.39%. The accuracy improvement difference is minimal, from −0.16% to 0.22%.

The RF with BPSO selects features between 9 and 17, with an average of 13.6. The classification accuracy ranges from 99.45% to 99.89%, with an average of 99.71%. The accuracy improvement difference is small, from −0.05% to 0.34%. Since DT and RF already have great performance using the No-FS in Dataset 2, the accuracy improvement of FS is not obvious, but the effect of dimensionality reduction is considerable.

BGWO-based

Figure 8 illustrates the results obtained from employing BGWO across various supervised learning algorithms in dataset 1. The classification accuracy of the SVM algorithm increased from 94.83% to 96.89% using the No-FS feature set to 96.96% to 97.95% by using the BGWO selected feature set. The range of accuracy increment is 0.62–2.17%, which is very considerable. Meanwhile, the 80 to 111 features are selected from the 188 features by BGWO for the SVM classifier.

Similarly, NN in dataset 1 saw notable accuracy improvements. Initially ranging from 93.97% to 96.07%, accuracies improved to 96.58% to 97.95% after using BGWO-selected features, with improvements ranging from 1.61% to 3.23%. Compared with SVM, the number of selected features using NN is less, from 50 to 87.

The range of accuracy using the DT is 96.18–97.92% using the No-FS feature set and 96.45–97.71% using the BGWO, with improvements ranging from 0.11–0.5%. The number of selected features for the decision tree is from 62 to 93, which is between neural networks and SVM. For another tree-based algorithm, since RF combines multiple decision trees, it has the advantages of being less prone to overfitting and better performance in handling high-dimensional feature spaces. The classification accuracy using RF increased from 96.18–97.92% to 97.99–99.38% after using BGWO, which is the highest among all algorithms. BGWO reduced 188 features to 21–37 features for RF, demonstrating a superior reduction in time complexity.

In contrast, dataset 2 showed less pronounced improvements with SVM, where accuracies ranged from 95.73% to 96.83% with the No-FS feature set and 96.01% to 96.99% with selected features (Fig. 9), with minor improvements from −0.16% to 0.39%. The NN varied widely in accuracy due to data sensitivity, ranging from 93.18% to 96.67% initially and improving to 95.64% to 98.52% with BGWO, showcasing accuracy improvements of 0.66% to 2.52% with 13 to 23 features selected out of 30.

Both DT and RF in dataset 2 maintained steady performance, with DT achieving accuracies from 99.16% to 99.67% initially to 99.38% to 99.84% with selected features. RF achieved accuracies from 99.29% to 99.78% initially to 99.62% to 99.95% with selected features, showing minimal accuracy increments from 0.11% to 0.50% for DT and 0.00% to 0.38% for RF. DT selected 13 to 23 features, while RF selected 5 to 15 features, underscoring the efficient FS capabilities of BGWO for them.

Supervised classifiers

In dataset 1 shown in Fig. 10, the RF consistently demonstrates the highest accuracy across all FS methods, particularly excelling with BGWO. It performs exceptionally well on both high-dimensional and low-dimensional datasets, showcasing its robust nature. The other three classifiers do not exhibit significant differences in recognition accuracy.

The DT algorithm shows slightly better recognition accuracy than SVM and NN when using PCC, BGA, and BPSO methods. However, NN did not perform well with other FS algorithms but showed strong performance with the BGWO method. This significant improvement indicates that BGWO’s effective FS greatly enhances the learning capability of NN. SVM displays moderate accuracy and, when combined with different FS methods, consistently selects a larger number of features, confirming its better performance on high-dimensional datasets.

In dataset 2 (Fig. 11), DT and RF exhibit significantly higher accuracy compared to SVM and NN. The DT performs notably better in dataset 2, which contains 30 features, than in dataset 1, which contains 188 features. DT performs better on lower-dimensional data because such datasets can more effectively prevent overfitting in DT.

The RF continues to demonstrate the highest accuracy regardless of the FS method used. Since both DT and RF achieved exceptional performance on the No-FS feature set, the FS algorithms do not significantly improve their recognition accuracy. However, these algorithms effectively reduce the number of features without compromising accuracy. The lower dimensionality of dataset 2 imposes some limitations on the effectiveness of FS algorithms. Additionally, SVM’s preference for high-dimensional datasets means its recognition accuracy is not outstanding in dataset 2, and FS does not significantly improve its performance. Consistent with its performance in dataset 1, SVM performs better on high-dimensional data, which is why the FS algorithms result in fewer feature reductions.

The NN shows similar behavior in dataset 2 as in dataset 1, with a notable improvement in recognition accuracy only when using the BGWO algorithm. This indicates the irreplaceable role of the BGWO method in enhancing NN’s ability to select features for IoT device recognition.

Feature selection methods

Filter-based algorithms select features based on statistical relationships between features or between features and the target variable, whereas wrapper-based algorithms choose features through iterative training and evaluation of the model. In both dataset 1 and dataset 2, wrapper-based methods consistently demonstrated higher performance in improving recognition accuracy compared to filter-based methods.

Filter-based methods often fail to reduce features without compromising accuracy. As illustrated in Fig. 10, the PCC algorithm significantly reduced the number of features in dataset 1 but also noticeably decreased the recognition accuracy for SVM and NN. This suggests that PCC may have removed features that are highly relevant for these classifiers, leading to a drop in performance. Unlike wrapper-based methods, which evaluate feature importance based on model training, PCC relies on correlation coefficients, which do not always capture the most predictive features for ML models. Similarly, the RF algorithm performed slightly worse on the selected features by the MI-based method compared to the No-FS feature set. This indicates that MI might not always be the optimal choice for FS, as it focuses on information gain without considering the interdependencies among multiple features simultaneously. Among filter-based methods, the MI-based method outperformed the PCC-based method for SVM, NN, and DT classifiers in dataset 1. However, PCC proved more effective for the RF classifier.

In contrast, wrapper-based methods managed to reduce the number of features without degrading recognition accuracy. Among these methods, BGWO consistently exhibited superior performance across all classifiers and datasets. Using the same number of search agents and iterations, BGWO selected fewer features while achieving higher accuracy for each classifier. Although BGA and BPSO were slightly less effective than BGWO, they still significantly reduced the number of features and improved recognition accuracy. In dataset 1, BPSO and BGA selected a comparable number of features for each classifier, with BPSO generally providing better accuracy improvements. BPSO enhanced the precision of NN, DT, and RF classifiers more effectively than BGA, which only slightly outperformed BPSO for the SVM algorithm. In a similar vein, these findings align with previous studies that have demonstrated the effectiveness of wrapper-based FS techniques in enhancing classification performance. Studies such as Aksoy & Gunes (2019), Zhang, Gong & Qian (2020), and Du, Wang & Li (2022), reviewed in the literature section, report similar trends where FS methods significantly optimize ML performance while reducing feature dimensionality. For instance, the GA-based FS approach in Mazhar et al. (2021) enabled classifiers to achieve high accuracy while reducing feature sets by more than half, minimizing computational complexity. Similarly, ReliefF in Zhang, Gong & Qian (2020) improved classification accuracy to 95% by selecting the most informative IoT traffic features, and NSGA-III in Du, Wang & Li (2022) enhanced classification accuracy from 94.5% to 95.8%.

In dataset 2, which has fewer features, FS algorithms did not significantly enhance the recognition accuracy of SVM, DT, and RF algorithms. However, it was still observed that wrapper-based methods outperformed filter-based methods, as shown in Fig. 11. BGWO remained the best-performing FS algorithm, not only selecting a more concise feature set to improve recognition efficiency but also enhancing recognition accuracy. The inferior performance of PCC and MI in dataset 2 further confirms that these filter-based methods may not be suitable for scenarios where complex feature interactions play a critical role. Since they do not consider feature dependencies holistically, they may discard relevant features, leading to suboptimal classifier performance. The NN demonstrates more sensitivity to FS compared to other classifiers and shows significant performance improvement with BGWO, similar to its performance in dataset 1. BGWO improved NN by selecting the most relevant features, significantly enhancing its performance. Similarly, BPSO and BGA improved NN’s performance to some extent, whereas PCC and MI did not contribute to any increase in classification accuracy for NN.

Findings from our experiments align with our study, confirming that FS techniques consistently enhance classification performance across various datasets and methodologies. The consistency of our results with prior studies validates the effectiveness of our approach and demonstrates its generalizability to different domains.