Semi-2DCAE: a semi-supervision 2D-CNN AutoEncoder model for feature representation and classification of encrypted traffic

Representation of FlowSpectrum

We first define the set of network flow features as the vector $\overrightarrow{F}=\left\{f_1,f_2,\dots ,f_m\right\}$, here f_i is the eigenvalue, i ∈ [1, m]. For each instance flow, we define this as $\overrightarrow{x}=\left\{x_1,x_2,\dots ,x_m\right\}$, here x_i is the instance flow point value, corresponds to the value of f_i in $\overrightarrow{F}$. We define a set of instance flows as $\mathcal{X}=\left\{\overrightarrow{x_1},\overrightarrow{x_2},\dots ,\overrightarrow{x_n}\right\}$, here $\overrightarrow{x_w}\subseteq \mathcal{X}\subseteq {\mathbb{R}}^m,\mathcal{X}$ is an object in cyberspace, ℝ is a real number space object, w ∈ [1, n]. Also, we define the instance output set as $\mathcal{L}=\left\{t_1,t_2,\dots ,t_k\right\}$, here each t indicates an output type. We create a set $\mathcal{D}=\left\{\left(\overrightarrow{x_1},\overrightarrow{L_1}\right),\left(\overrightarrow{x_2},\overrightarrow{L_2}\right),\dots ,\left(\overrightarrow{x_m},\overrightarrow{L_m}\right)\right\}$, which is the corresponding set of input instances $\overrightarrow{x}$ and output instances $\overrightarrow{L}$, $\overrightarrow{L_w}\in \mathcal{L}$. We denote by ${\mu }_t\left(\overrightarrow{x}\right)$ the probability that the output type of $\overrightarrow{x}$ is t. Expressed in the formula as follows: (1)${\displaystyle {\mu }_t\left(\overrightarrow{x}\right)=P\left\{Y=t\left|\right|X=\overrightarrow{x}\right\}}$ where Y and X are respectively a random variable in $\mathcal{L}$ and $\mathcal{X}$. Finally, we define an 𝔉(𝒳_t) to denote the set of feature values of the network space flow mapping to the real number space (i.e., 𝒳 → ℝ): (2)${\displaystyle \mathfrak{F}\left({\mathcal{X}}_t\right)=\left\{\begin{array}{c}\hfill d\left(\overrightarrow{x_1}\right):{\mu }_t\left(\overrightarrow{x_1}\right),\hfill \\ \hfill d\left(\overrightarrow{x_2}\right):{\mu }_t\left(\overrightarrow{x_2}\right),\hfill \\ \hfill \cdots ,\hfill \\ \hfill d\left(\overrightarrow{x_n}\right):{\mu }_t\left(\overrightarrow{x_n}\right)\hfill \end{array}\right\}}$ where $d\left(\overrightarrow{x_n}\right)$ is the value in the real number space ℝ. We call 𝔉(𝒳_t) the FlowSpectrum. As shown in Fig. 5, we provide a FlowSpectrum mechanism flowchart.

FlowSpectrum applied to traffic classification

The process of generating FlowSpectrum belongs to the decomposition and representation of network flows. As described in Section 1 FlowSpectrum generation is part of the training phase, while flow classification belongs to the classification phase. In the classification phase, the classifier we use is the Bayesian optimal classifier. First, we create the test flows set ${\mathcal{X}}^{{}^{\prime }}=\left\{\overrightarrow{x_1},\overrightarrow{x_2},\dots ,\overrightarrow{x_n}\right\}$, where the probability value of instance $\overrightarrow{x_1}$ belonging to a certain category is set to $P\left(\overrightarrow{x_1}\left|\right|{\mathcal{X}}^{{}^{\prime }}\right)$. Then, the FlowSpectrum of the test flows is as follows: (3)${\displaystyle \mathfrak{F}\left({\mathcal{X}}^{{}^{\prime }}\right)=\left\{\begin{array}{c}\hfill d\left(\overrightarrow{x_1}\right):P\left(\overrightarrow{x_1}\left|\right|{\mathcal{X}}^{{}^{\prime }}\right),\hfill \\ \hfill d\left(\overrightarrow{x_2}\right):P\left(\overrightarrow{x_2}\left|\right|{\mathcal{X}}^{{}^{\prime }}\right),\hfill \\ \hfill \cdots ,\hfill \\ \hfill d\left(\overrightarrow{x_n}\right):P\left(\overrightarrow{x_2}\left|\right|{\mathcal{X}}^{{}^{\prime }}\right)\hfill \end{array}\right\}}$ And then, the similarity probability of the FlowSpectrum is found by clicking the formula as follows: (4)${\displaystyle \mathfrak{F}\left({\mathcal{X}}^{{}^{\prime }}\right)\cdot \mathfrak{F}\left({\mathcal{X}}_t\right)=P\left\{Y=t\left|\right|X\in {\mathcal{X}}^{{}^{\prime }}\right\}}$ where $P\left\{Y=t\left|\right|X\in {\mathcal{X}}^{{}^{\prime }}\right\}$ is the probability that ${\mathcal{X}}^{{}^{\prime }}$ belongs to ${\mathcal{X}}_t$. Finally, the final classification result is found by finding the maximum likelihood estimate $\hat{y}$. This is expressed as follows: (5)${\displaystyle \hat{y}=argmaxP\left\{Y=t\parallel \parallel X\in {\mathcal{X}}^{{}^{\prime }}\right\}=argmax\mathfrak{F}\left({\mathcal{X}}^{{}^{\prime }}\right)\cdot \mathfrak{F}\left({\mathcal{X}}_t\right).}$ when there are uncharacterized spectral values in ${\mathcal{X}}^{{}^{\prime }}$, we need to calculate the minimum distance between the spectral lines to determine the type of unknown spectral lines. We use the calculation method of exponential decay proposed in Yang et al. (2022), as follows: (6)${\displaystyle \overrightarrow{x^{{}^{\prime }}}=arg\underset{\overrightarrow{x^{{}^{\prime }}\in x_t}}{min}∥d\left(\overrightarrow{x}\right)-d\left(\overrightarrow{x^{{}^{\prime }}}\right)∥}$ (7)${\displaystyle {\mu }_t\left(\overrightarrow{x}\right)={\mu }_t\left(\overrightarrow{x^{{}^{\prime }}}\right)e^{-∥d\left(\overrightarrow{x}\right)-d\left(\overrightarrow{x^{{}^{\prime }}}\right)∥}}$ where $\overrightarrow{x^{{}^{\prime }}}$ is the test instance flow.

Encoder

The encoder consists of an input layer, 2D convolutional layer, PRelu layer, and pooling layer, and its function is to extract features from the network flow. The model receives 28x28 byte flow images and trains 2D convolutional kernels in the convolutional layer. The convolutional layer is represented as follows: (8)${\displaystyle H^j=f\left(b^j+\sum _i{w}^{ij}\ast x_{\mathrm{i}}\right)}$ H^j and x_i are the jth output mapping and the ith input mapping respectively. w^ij denotes convolution filter weights. “∗” indicates convolution, b^j is the deviation parameter of the jth mapping. f denotes the activation function. For this convolutional layer, the PRelu activation function is used to prevent overfitting and to fully explore the correlations between features (we will provide a detailed introduction to activation functions in ‘Comparison of feature representation of encrypted traffic’). We then applied a maxpooling layer to reduce redundant data and compress it. Pooling also increases the receptive field, making the pooled data features translation invariant. Overall, we represent the encoder as E, with the input network flow as  $\overrightarrow{x}$ and E_ed defined as the encoded network flow. Then, the encoder is represented in the following formula: (9)${\displaystyle {\mathrm{E}}_{ed}=\mathrm{E}\left[H\left(\overrightarrow{x}\right)\right].}$

Mapper

The mapper consists of a flattening layer, fully connected layer, reshaping layer, and Softmax classifier, and its function is to generate FlowSpectrum by representing the network flow features as spectral lines. After encoding, the high-dimensional data features become multidimensional vector features in the form E_ed. Then, the feature maps are smoothed and mapped to two real-valued numbers using two fully connected layers. We use the notation $\overrightarrow{T}$ to represent the feature that can be characterized and $\overrightarrow{F}$ to represent the feature that cannot be characterized, which correspond to the two real-valued numbers. And then, we input the representable feature $\overrightarrow{T}$ into the Softmax function to get the probability value that the flow belongs to a certain category, and take the category with the largest probability value as the final classification result. The formula of the Softmax function is shown below: (10)${\displaystyle p_i=\frac{e^{T_i}}{\sum _{i=0}^k{e}^{T_i}}=\frac{e^{d\left(x_i\right)}}{\sum _{i=0}^k{e}^{d\left(x_i\right)}}}$ where p_i is the probability that the input sample belongs to category i, $T_i\in \overrightarrow{T}$ is a real number in the range, i is the encrypted flows category index, and k is the total number of encrypted flows categories. Here, we have $d\left(x_i\right)=T_i$, i.e., our FlowSpectrum set $d\left(x\right)=\overrightarrow{T}$. After obtaining the classification results, we use the categorical cross-entropy function to calculate the Loss₁: (11)${\displaystyle {Loss}_1=-\sum \left\{\underset{i}{\overset{n}{X}}{log}_2\left(p_i^n\right)\right\}}$ where n is the sample label index. In addition, we use the representable features $\overrightarrow{T}$ and non-representable features $\overrightarrow{F}$ as the input of the decoder after concatenation and shaping.

Overall, we denote the mapper as $\mathcal{M}$. By continuously training Loss₁ value decreases, we finally get the FlowSpectrum of representable features. Denote ${\mathcal{M}}_{ed}$ as the mapping output and ⊕ as the concatenated shaping operator. Then the mapper is represented by the following equation: (12)${\displaystyle {\mathcal{M}}_{ed}=\overrightarrow{T}\oplus \overrightarrow{F}}$

Decoder

Also known as the reconstructor, after the mapper outputs ${\mathcal{M}}_{ed}$, the decoder is used to reconstruct the original network flows to obtain the corresponding reconstructed value. We use the mean square error function to calculate the Loss₂: (13)${\displaystyle {Loss}_2=\frac{1}{n}\sum _{i=1}^n{∥{\hat{x}}_i-x_i∥}^2}$ where ${\hat{x}}_i$ is the element value of the reconstructed network flows $\hat{x}$.

Overall, we denote the decoder as $\mathcal{D}$. Then in the decoder is represented by the following equation: (14)${\displaystyle \hat{x}=\mathcal{D}\left({\mathcal{M}}_{ed}\right)}$

The loss value of our whole framework design is set to Loss, then that Loss is the sum of the Loss weights of the mapper and decoder.

Hyperparameters

In the Semi-2DCAE model, the size of convolution cores in the convolution layer is 3, the number of convolution cores used in the first and second convolution is 32 and 64, respectively, and the size of the maximum pooling layer is 2. It should be noted that we use L2 regularization in the encoder to prevent over-fitting, and the regularization rate is 0.01. Epoch is set to 300, Batch_size is 64, and the learning rate is 0.0005.

Activate the function

Our model selects the random linear rectification activation function PRelu (Kannari, Shariff & Biradar, 2021) with leakage, whose expression is: (19)${\displaystyle f\left(x\right)=\left\{\begin{array}{cc}\hfill x\hfill & \hfill x>0\hfill \\ \hfill \lambda x\hfill & \hfill x\le 0\hfill \end{array}\right.,\lambda \in \cup \left(L,u\right),L<u,u\in \left[0,1\right)}$ x is the transmission data of neurons, λ is the gradient function of PRelu when the input value is a negative value range, and $\cup \left(L,u\right)$ represents the continuous uniform distribution function probability model. λ is a random variable drawn from $\cup \left(L,u\right)$. Through formula (19), we can find that PRelu can update the weight normally when the neuron output data is greater than zero, while the weight can also be updated slightly when the neuron output data falls within the negative range of zero. As shown in Figs. 7 and 8, during the model training phase (note that during the training phase, we divide the training dataset into training and validation sets in a 2:1 ratio.), we provided the training accuracy and training loss of the model using two different activation functions, PRelu and Relu. From Figs. 7A and 8A, it can be observed that when Relu is used as the activation function for the model, oscillations occur during the training process. This phenomenon arises due to the output of the activation function being zero, preventing the update of weights when data values fall within the negative range, coupled with the presence of a large learning rate. Paradoxically, reducing the learning rate hampers the model’s ability to converge to the global minimum. However, from Figs. 7B and 8B, it is evident that when PRelu is employed as the activation function, our model exhibits greater stability during the training process, leading to an improvement in training accuracy. By choosing PRelu as the activation function, our model effectively mitigates the learning rate issue, ensuring convergence to the global minimum and preventing overfitting.