Sterilization of image steganography using self-supervised convolutional neural network

SS-Net principle

The network models of steganography sterilization methods require paired training data. However, in social networks, it is impossible to capture the stego image corresponding to the cover image, and the network models of these methods cannot be trained when the cover image is not available. Therefore, this paper proposes SS-Net to improve the learning strategy, improve the learning strategy of this model from supervised learning to self-supervised learning, alleviate the phenomenon of the existing steganography sterilization methods that rely excessively on the original cover images, and improve the practicality of this method in practical social networks. The flow of SS-Net is shown in Fig. 1.

From Fig. 1, the SS-Net includes two parts, the first part is purification module, which is able to eliminate the secret messages while preserving the quality of the image. The second part is refinement module, which improves the image generated by purification module, and is capable of secondary filtering of secret messages with low visual degradation to improve the blocking rate of our method in practical applications. Since the refinement module has no training parameters, it does not participate in the network training process. Finally, the sanitized image x_san generated by SS-Net can only extract the wrong secret messages.

Our method aims to eliminate the secret messages without affecting the image quality. The method has two objectives: maintain the visual quality of the sanitized image; and ensure that no valid secret messages can be observed after active defense (i.e., the difference between the secret messages before embedding and the secret messages of the sanitized image extracted after active defense should be large enough). Based on these two objectives, this paper formally defines the sanitized image x_san by Eqs. (1), (2) and (3). (1)${\displaystyle x_{san}=f_{SS-Net}\left(x_{stego}\right)}$ (2)${\displaystyle f_{SS-Net}\left(x_{stego}\right)=f_{ref}\left[f_{pur}\left(x_{stego},\theta \right)\right]}$ (3)${\displaystyle \begin{array}{c}\underset{x_{pur}}{max}{f}_{mes}\left(D_e\left(x_{stego}\right),D_e\left(x_{san}\right)\right)\hfill \\ \begin{array}{c}s.t.f_{pic}\left(x_{stego},x_{san}\right)\le \varepsilon \hfill \end{array}\hfill \end{array}}$

where x_cover and x_stego denote the cover image and the corresponding stego image. f_SS−Net, f_pur and f_ref denote the overall method of this paper, the first part of this method and the second part of this method, respectively, f_mes denotes measure of the difference between secret messages, f_pic denotes measure of the visual distance between the two images. ɛ represents an acceptable threshold, and θ denotes a vector of parameters that can be learned by the network model. Equations (1), (2) and (3) is to measure the distance between sanitized and stego images in two domains: message difference and visual difference.

Self-supervised learning SS-Net

Since steganography starts with hiding communication facts, the stego image is highly correlated with the cover image. In this paper, the steganography embedding model and extraction model are defined as E_n and D_e. The embedding and extraction process can be represented as Eqs. (5) and (8) respectively. Under the additive model, the stego image distribution after embedding the secret messages is shown in Eqs. (4), (6) and (7). (4)${\displaystyle x_{stego}=E_n\left(x_{cover},m\right)}$ (5)${\displaystyle E_n\left(x_{cover},m\right)=x_{cover}+\sum _{op{e}_i\in O}op{e}_i}$ (6)${\displaystyle \varphi \left[E_n\left(x_{cover},m\right)\right]=\varphi \left(x_{cover}\right)+\sum _{op{e}_i\in O}{\delta }_{op{e}_i}}$ (7)${\displaystyle {\delta }_{op{e}_i}=\varphi \left(x_{cover}+op{e}_i\right)-\varphi \left(x_{cover}\right)}$ (8)${\displaystyle m^{\prime }=D_e\left(x_{stego}\right)}$

where m and m′ denote the embedded secret messages and the extracted secret messages, respectively. $op{e}_i\in O,\begin{array}{c}\hfill \end{array}i=1,\dots ,n$ denotes the basic operation of the secret messages embedding process. O is a combination of operations to complete an steganography algorithm. $\varphi \left(\cdot \right)$ denotes the operation of calculating the statistical distribution. δ_opei denotes the effect of one embedding basic operation on the distribution of the cover image. If stego images generated by different steganography algorithms is used to train the model with the corresponding cover image, the model will perform poorly on the real stego image under the unknown steganography. Equation (6) describes the cover distribution after steganography, which is an additive model that simplifies the study (Chonev & Ker, 2011). As shown in Eq. (6), each step of operation within secret messages embedding model E_n has no interactions with each other, and the effects on the cover image distribution are independently additive, such that secret messages are given independent signal for pixels in the cover image, which is able to provide basic assumptions for the training SS-Net. The assumptions are that the secret messages are not related to the cover image. The interpretation of self-supervised learning framework on steganography destruction refers to the use of self-supervised learning methods to detect, identify or restore images or data subjected to steganography attacks. Steganography is a technique for embedding hidden information into overlaying media, such as embedding secret messages into images, audio, or text, without causing a change in human perception. Self-supervised learning frameworks can identify or restore changes caused by these steganographic attacks by learning hidden patterns or anomalies in the data. In self-supervised learning frameworks, a self-supervised task is often designed to simulate the effects of steganography, such as generating labels by adding hidden messages to an image or making changes to the image. The model learns a mapping function that maps the original data to the stego-damaged data to recognize or restore the stego-attacked image. Therefore, this paper proposes a self-supervised approach to train SS-Net, which is a variant of traditional CNN where the center pixel is not visible in the sensory field to predict the corresponding output pixel. We discard the training method of mapping the input of the stego image with secret messages to the clean cover image, and are able to train without the ground truth data of the cover image, which improves the effectiveness of method in social networks.

The self-supervised approach to training SS-Net allows to train directly on the stego image, from which two parts of the training sample, namely the input value and the target value, are derived. In order to clearly demonstrate the use of self-supervised approach to train this network, this paper compares the ordinary network of steganography sterilization with the SS-Net and gives a visualized example as shown in Fig. 2.

In Fig. 2A, a patch is simply extracted as an input and its center pixel is used as a target, and the prediction of individual pixels relies on a square neighborhood of the input pixel called the pixel’s receptive domain. The model directly maps the values at the center of the input patches to the output values. In Fig. 2B, since SS-Net requires that the center pixel is not visible in the receptive field, SS-Net sets the center of input patch as a blind spot. The only one pixel is removed. Rely on other receptive fields of the input pixels instead of the blind spots, thus avoiding learning consistency. It makes it possible to learn to delete pixel-independent secret messages. SS-Net allows the extraction of input blocks and target values from visual training images. This paper can train it by minimizing the empirical risk by Eq. (9). (9)${\displaystyle \underset{\theta }{\mathrm{arg min}}{∥f_{SS-Net}\left(x_{stego},\theta \right)-x_{stego}∥}_1.}$

Purification module

Purification module consists of pixel-shuffle down-sampling (Zhou et al., 2019) and purification network based on blind spot network (Krull, Buchholz & Jug, 2018). Pixel-shuffle down-sampling employs the idea of divide-and-conquer and is able to restrict the front-end and back-end of the purification network, making purification network suitable for steganography sterilization. Pixel-shuffle down-sampling rearranges the image pixel values according to the sampling factor β, decomposes the spatial correlation of the image, and is used to isolate the correlation between the image and secret messages, so as to make the pixel points where the secret messages originally exist to be misaligned, and thus removes the high-frequency components that are rich in information, and then eliminates the obvious secret information. The size of β can affect the spatial correlation of the image, and when β is small enough, it can retain the texture structure in the image, but it cannot achieve the purpose of eliminating the secret information. When β is large enough to cause spectral aliasing in the image, the output is a low-frequency signal that does not actually exist, resulting in significant degradation of the image. We set β to 2 in order to balance the two above. The process of pixel-shuffle down-sampling is shown in Fig. 3. Pixel-shuffle down-sampling decomposes the image patch into 2*2 sub-patches, and the four sub-patches are recombined by tiling from Fig. 3.

Blind spot network outputs a result at a point without being able to see the information at that point but only the information at its surrounding points. Purification network is unable to deduce secret messages at the blind spot from the information of the surrounding pixel points, but will only predict the image signals associated with the surrounding structure. Adding the above strategy to a standard CNN network will inevitably hinder efficient training of the network. Therefore, we propose a purification network that integrates centrally masked convolutions and dilated convolution residual blocks to alleviate the above problem. Meanwhile, this paper employs a two-branch blind spot network, which is able to jointly restore more detailed visual information of images through a tightly connected structure. The overall architecture of the Purification module is shown in Fig. 4.

We utilize the x_PD generated by the pixel-shuffle down-sampling strategy as inputs for self-supervised training of the purification network, and feed them one by one into the network to learn the image mapping, which influences each pixel in the output of the network to predict the x_pur through the special receptive field of the input pixel. In Fig. 4, firstly, after one layer of convolution, the feature map with the same size as the input image is obtained. Secondly, after Type 1 branch and Type 2 branch respectively, the feature maps obtained from the two branches are spliced by columns without adding new dimensions. The two branches are able to work together to eliminate potential secret messages in the image through a tightly connected structure and dual saliency mechanism, while learning more detailed visual messages and generating efficient feature representations. Finally, convolution layers are passed with the aim of outputting three-channel RGB image. Branch is started by the centrally masked convolution layer. Next, after two sets of convolutional layers. Then, after nine sets of dilated convolution residual blocks and convolution layer. Purification network configuration details are known from Table 1. In Table 1, n represents the number of convolutional kernels, h represents the height of the convolutional kernel, w represents the width of the convolutional kernel, p represents the number of pixels to fill the edges, s represents the stride of the convolutional kernel, and d represents the dilation rate. Centrally masked convolution is introduced to return the corresponding weight of the center pixel to 0, so that the center element is masked on the patch of the original tensor and the connection with the neighboring pixels is restricted, so as to achieve the purpose of eliminating secret messages in the center pixel. The centrally masked convolution is defined as follows: (10)${\displaystyle f_k^{\left(i\right)}=f_k^{\left(i-1\right)}\ast \left(w_k^{\left(i-1\right)}\circ mask\right)}$

where $f_k^{\left(i\right)}$ denotes the feature mapping of the k channels of the i-th layer, $f_k^{\left(i-1\right)}$ denotes the feature mapping of the k channels of the i-1 th layer, $w_k^{\left(i-1\right)}$ denotes the k convolution kernels, ∗ and ∘ denote the convolution operation and the elemental product, respectively, and mask denotes the binary mask which is the same as the size of the convolution kernel.

Dilated convolution residual blocks are able to obtain a larger receptive field, which makes the contact to the image range large, and contains features that do not tend to be localization and detail. Purification network will not learn potential secret messages, and thus avoid mapping the secret messages to the network output layer. At the same time, the dilated convolution residual blocks can capture multi-scale image information, and maintain the output feature map resolution, thus reducing the image quality degradation problem. Dilated convolution residual block is composed of a set of network layers, including the main path and skip connection. The main path consists of dilated convolution layer, convolutional layer and activation function for learning feature representation. The skip connection directly bypasses the main path by adding the input information to the output. This ensures that the input information is more easily propagated to subsequent layers and helps avoid the problem of vanishing gradients. Dilated convolution is able to more aggressively merge spatial information across the input with fewer convolutional layers by expanding the spacing between the kernel points. Dilated convolution with dilation rate of 2 adds zero between the convolution kernel elements to expand the sensory field, and dilated convolution with dilation rate of 3 adds double zero between the convolution kernel elements to expand the sensory field. In purification network, the dilated convolution 1 residual blocks use dilation rate of 2, and the dilated convolution 2 residual blocks use dilation rate of 3.

Refinement module

Purification module is able to eliminate embedded secret messages. However, there are visual artifacts in the local details of the generated images. In order to alleviate the above problems, refinement module aims to eliminate secret messages while including improving the image texture details and boundaries, smoothing the flat regions and avoiding the generation of artifacts. Refinement module is not involved in training, which greatly improves the training time of our method. The process of refinement module is shown in Fig. 5.

As can be seen from Fig. 5, x_stego generates x_pur through the purification module. x_pur isused as the input of refinement module, and the output of refinement module is x_san. The refinement module mainly consists of three parts: the first part fills each sub-image pair of x_pur with pixel blocks with secret messages respectively to obtain x_sep that separates secret messages channel. The second part performs pixel-shuffle down-sampling to obtain x_PD, and the purification module is used to eliminate secret messages from each re-filled image x_PD to obtain x_pur′. The third part averages x_pur′ to obtain an image texture detail image x_refineof x_stego, and averages x_refine with x_pur′ to generate a purified image x_san.

Implementation details

We using ALASKA v2 (Cogranne, Giboulot & Bas, 2020a) and BOSSbase 1.01 (Bas, Filler & Pevný, 2011) datasets, where the ALASKA v2 dataset consists of cover images and stego images generated by three steganography algorithms, including J-MiPOD (Cogranne, Giboulot & Bas, 2020b), J-UNIWARD (Holub, Fridrich & Denemark, 2014), and (UERD) (Guo et al., 2015). The images were generated by a steganographic algorithm designed by the ALASKA project team and also included samples of real-world images. The BossBase 1.01 dataset contains 10,000 color images covering a wide variety of topics and content, such as plants, landscapes, architecture, and more. As shown in Table 2. The images are generated by specially designed steganography tools, and in addition to automatic generation, the dataset also contains some real-world images to increase the authenticity and representation of the dataset. The image size is 512 ×512 pixels and the depth is 8 bits.

In this paper, 10,000 images are arbitrarily selected from the above dataset. Among them, training 8,000 images, validation 1,000 images and testing 1,000 images. In order to reduce the computational time complexity and achieve efficient training, all images are preprocessed. Preprocessing: first, the 512*512 stego image is cropped into 256*256 small size patches, the overlap between the small size patches is 128, and 9 small size patches will be generated for each image. Then, convert the small size patches form tensor to numpy. Finally, the dimension of feature image (c, h, w) is converted to (h, w, c).

To train an efficient steganography sterilization network, this paper uses PyTorch, trains SS-Net on an Intel(R) Core (TM) i9-10900 CPU @ 2.80 GHz and an NVIDIA GeForce GTX 1660 Ti GPU. Therefore, there is no need to manually define the weights and biases of the network layers. The parameter settings for specific networks are shown in Table 3. In this paper, 100 images are randomly selected from the testing set of two datasets beforehand, and the three most representative steganography algorithms DMAS (Zhang et al., 2018), GMAS (Yu et al., 2020) and LSB are used for steganography.

This paper uses the following well-known indicators to assess the degree of elimination of secret information and the impact on image quality. In this paper, the ability to eliminate secret messages is measured using the Bit Error ratio (BER) (Al-Sultan, Ameen & Abduallah, 2019), which is the ratio of the number of erroneous bits to the total number of bits. A larger BER result indicates that more secret messages are eliminated after steganography sterilization. It is generally accepted that hidden secret messages are eliminated when the BER is greater than 0.2. In this paper, Peak Signal-to-Noise Ratio (PSNR) (Huynh-Thu & Ghanbari, 2008) and Structural Similarity Index (SSIM) (Zhou et al., 2004) are used to evaluate the quality of the image after steganography sterilization. PSNR is an objective metric to assess the distortion of the image. SSIM is an objective metric to compare the objective metrics for comparing the similarity of the quality of the image after active defense steganography sterilization with that of the original cover image. The larger the results of PSNR and SSIM, the better the quality of the image. It is worth noting that the value of each objective metric in the experiment is the average result on the testing set.

Ablation experiment

In order to verify the effect of the depth of this network architecture on steganography sterilization, the number of dilated convolution residual blocks in this paper is selected as 5, 7, 9, and 11, respectively, and 20 DMAS stego images are randomly selected from the testing set, with a quality factor of 95 for the cover images and payload of 0.01, and BER, PSNR, and SSIM of the purified images after this method are computed, which the BER, PSNR and SSIM are calculated to verify the effect of eliminating secret messages and image quality. The experimental results are shown in Table 4.

As can be seen from Table 4, when the number of dilated convolution residual blocks is 9, the BER value of the sanitized image is 0.499, the PSNR value is 43.0848, and the SSIM value is 0.9898. All the above data show that the optimal effect of steganography sterilization can be achieved when the number of dilated convolution residual blocks is selected as 9. Meanwhile, the BER, PSNR and SSIM values of the sanitized image are closer under different number of dilated convolution residual blocks, indicating that the present network architecture has a greater advantage in steganography sterilization itself.

Objective evaluation

In this section, experiments are conducted for steganography algorithms DMAS, GMAS and LSB to verify the effectiveness of our method. The BER of the sanitized images generated by the SS-Net under different payloads and image quality factors is taken as the mean value for the final experimental results, as shown in Tables 5, 6 and 7.

As can be seen from Table 5, under different image quality factors and payloads, the average BER value of sanitized images is 0.487 after our method actively defends against DMAS, which is 243.5% of the baseline value. The baseline value of BER is 0.2 when the sequence of secret messages can not be recovered. The experimental results show that SS-Net can effectively eliminate secret messages and realize steganography sterilization. As can be seen from Table 6, under different image quality factors and payloads, after the steganography sterilization of GMAS by our method, the BER value of the sanitized images is the smallest of 0.4885, which is 244.25% of the baseline value, when the quality factor is 95 and payload is 0.01bpnac, and this value is far more than the unrecoverable value of secret messages. As can be seen from Table 7, under different image quality factors and payloads, the average BER value of sanitized images is 0.349 after our method actively defends against LSB, which is 174.5% of the baseline value. The above data show that under different steganography methods, this method can achieve the purpose of eliminating secret messages and successfully realizes steganography sterilization.

In summary, the value of BER fluctuates between 0.32 and 0.52 under different quality factors, payloads and steganography algorithms. It is verified that the SS-Net can effectively eliminate secret messages in potentially stego images in social networks and realize the purpose of proactive defense against covert communications without considering the specific steganography algorithm, quality factor, and payload used by the steganographer.

Our conduct a series of experiments for robust steganography to verify the effectiveness of SS-Net in maintaining image quality. This experiment uses PSNR and SSIM to evaluate the sanitized images. Our method actively destroys robust steganography DMAS, GMAS and LSB, and the average values of PSNR and SSIM of sanitized images generated by SS-Net are used as the experimental results, as shown in Tables 5, 6 and 7.

As can be seen from Table 5, the PSNR value of sanitized images on DMAS by this method ranges between 43 and 46, and the SSIM value ranges between 0.98 and 1 under different quality factors with different payloads. It shows that the method of this paper can maintain the quality of the image under different quality factors. From Table 6, it can be seen that on different payloads, the average PSNR and SSIM values of the sanitized images reach 43.419 and 0.9871 at a quality factor of 65. At a quality factor of 75, the average PSNR and SSIM values of the sanitized images were 43.353 and 0.9918. At a quality factor of 85, the average PSNR and SSIM values are 44.6189 and 0.9903. At a quality factor of 95, the average PSNR and SSIM values of sanitized images were 45.2682 and 0.9902. As can be seen from Table 7, the PSNR value of sanitized images on LSB by this method ranges between 48 and 49, and the SSIM value ranges between 0.993 and 1 under different quality factors.

Therefore, the PSNR and SSIM of sanitized images under different steganography algorithms meet the quality requirements of images transmitted by social networks, and all the above data show the effectiveness of the method in this paper to maintain the quality of images.

In addition, the different texture images for the results of our display experiment are shown in Table 8. As can be seen from Table 8, it can be concluded that on different texture images, the BER is above 0.52, the value of PSNR is above 44 and the value of SSIM is above 0.994. It can be seen that this method has achieved good results on both high texture and low texture.

What’s more, we conducted a comparative analysis of the influence of different social networks on steganography destruction in the process of image transmission, as shown in Table 9. In the experiment, we compared the changes of images on five social media platforms: Instagram, Facebook, WeChat, Twitter and YouTube. It can be seen that the images after transmission, BER values are above 0.45, PSNR values are above 42 and SSIM values are above 0.96. It can be seen that this method can meet the requirements of different social media networks.

The experimental effect is shown in Fig. 6. In Fig. 6, the first column is the cover image, the second column is the stego image, and the third column is the sanitized image. It can be seen that there is no change in the visual effect of the image after destruction.

Table 8:

Effect of different texture images on LSB algorithm.

	Images with different textures
Metrics
BER	0.59701	0.59633	0.55556	0.52825
PSNR	50.8473	51.3308	44.8439	53.1643
SSIM	0.9957	0.9974	0.9943	0.9964

DOI: 10.7717/peerjcs.2330/table-8

Comparative experiment

In order to compare with this method, typical and latest steganography sterilization methods are selected in this section: the AO-Net method (Zhu et al., 2021), DFT method (Geetha et al., 2021) and the SC-Net (Zhu et al., 2023) method. Among them, the AO-Net method is the first generalized method for robust steganography and watermarking sterilization. All the test images used in this section are randomly selected from the testing set and the images are compressed into a frequency domain image with a quality factor of 95 and resized to 512 ×512. Thirty stego images were generated at 0.01−0.05 payloads, respectively. Experiments are conducted to compare this paper’s method using the robust steganography DMAS and GMAS as examples, respectively. It is worth noting that the present method performs self-supervised training only on the stego images, while the AO-Net and SC-Net training requires both the stego images and the corresponding cover images. We present the evaluation results of sanitized images obtained by different steganography algorithms, as shown in Tables 10, 11 and 12. Each quantitative metric has 1 number under it, indicating the result after robust steganography DMAS and GMAS sterilization.

As can be seen from Table 10, it can be seen that in terms of eliminating secret messages, after using our method to eliminate secret messages embedded in DMAS, the BER of sanitized images is higher than that of DFT when the payload is 0.03, 0.04. In the other payload, the BER values of the sanitized images obtained by our method are all slightly lower than those of the other three methods. In terms of image quality, compared with the AO-Net method, SC-Net method and DFT method, the PSNR value of the sanitized image is increased by 7.1895%, 17.1815% and 5.8240%, and the SSIM value is increased by 1.095%, 7.545% and 10.855%, respectively. It can be seen that compared with other active hidden-removal methods for DMAS, our method can eliminate secret messages with high BER and improve the visual quality of the image.

As can be seen from Table 11, it can be seen that in terms of eliminating secret messages, the BER values of the sanitized images on GMAS by our method are all slightly lower than those of SC-Net, AO-Net and DFT when the payload is 0.01 to 0.05. However, in terms of image quality, compared with AO-Net method, SC-Net method and DFT method, the PSNR of the sanitized image by GMAS is increased by 7.23279%, 17.20979% and 4.86679%, and the SSIM is increased by 1.103%, 7.543% and 12.383%, respectively. It can be seen that our method has a clear advantage over other methods for the active defense of GMAS. It can be seen that our center mask convolution and dilated convolution residual block have obvious advantages in the restoration of image quality compared with the GMAS method, but we still have the disadvantage of slightly lower BER value.

As can be seen from Table 12, it can be seen that in terms of eliminating the secret message, after using our method to eliminate the secret message embedded in LSB, the BER of the sanitized image is higher than that of SC-Net. In terms of image quality, compared with AO-Net method and DFT method, the PSNR value of sanitized image is increased by 3.34258% and 2.08888% respectively. The SSIM values are increased by 1.183% and 5.135%, respectively. It can be seen that compared with other LSB active masking methods, our method can not only improve the visual quality of the image, but also eliminate the secret information with high bit error rate.