MFI-Net: multi-level feature invertible network image concealment technique

Auxiliary information w

If we aim to hide a secret image within a cover image while ensuring the stego image as similar as possible to the cover image, a portion of the original cover image’s information will inevitably be displaced. This occurs because the capacity of an image is finite. The displaced information is referred to as the lost information, denoted as l. In the recovery process, if one attempts to extract the restored image secret-rev only from the stego image without considering l, it becomes an ill-posed problem. Without the guidance of l, millions of secret images can be recovered from the same stego image. Therefore, this article introduces auxiliary information w as one of the inputs in the recovery process to assist in recovering the unique secret image from the stego image. w is sampled from a case-agnostic Gaussian distribution, which follows the same distribution as lost information l, i.e., $w\sim \mathcal{N}({\mu }_0,{\sigma }_0^2)$. During the training process, the network adjust its parameters based on the feedback from the loss function, ensuring that each sample in the distribution can effectively recover the secret information. In the experiments conducted in this article, ${\mu }_0$ is set to 0 and ${\sigma }_0^2$ is set to 1. (w is case-agnostic, this conclusion has been proven in Jing et al., 2021).

Upsampling convolution block

The residual structure is essential in the process of image hiding using invertible neural networks. The residual structure can effectively extract high-level image features such as abstract concept information, spatial relationship features, etc. Additionally, it helps significantly alleviate the performance degradation caused by network depth. However, while the residual structure focuses on the transmission and reconstruction of high-level semantic information, it often neglects the sparsity and subtle variations in low-level features. This can affect tasks such as image inpainting and restoration, where retaining certain low-level features is essential for combining them with high-level features during image reconstruction, thus maintaining higher fidelity and clarity of details. Moreover, a network structured solely with residual blocks is more prone to overfitting.

Therefore, we designed an upsampling convolution block (UCB) to extract low-level features from the image, enhancing the network’s performance. The overall structure of the UCB is shown in Fig. 2 and consists of two convolutional layers with normalization and LeakyReLU activation functions, as well as an upsampling block. The input feature maps are first convolved with a $3\times 3$ kernel using a stride of 1, followed by normalization and LeakyReLU activation to produce activated feature maps. These activated feature maps are then downsampled with a $3\times 3$ convolutional kernel and a stride of 2, preserving the same channel count. The output feature maps undergo normalization and LeakyReLU activation once more. Finally, the upsampling layer enlarges the feature maps by a factor of two, yielding the final output feature maps. For clarity on how this module processes information, its algorithmic description is provided in Algorithm 3.

Here, $x_{ucb}^{in}$ denotes the input to the UCB, while $x_{ucb}^{out}$ denotes its output. $Con{v}_{3\times 3}^{S1}$ represents a convolution with a kernel size of $3\times 3$ and a stride of 1, whereas $Con{v}_{3\times 3}^{S2}$ denotes a convolution with a kernel size of $3\times 3$ and a stride of 2. $BatchNorm$ stands for the batch normalization layer, $LeakyReLU$ refers to the LeakyReLU activation function, and $Upsample$ denotes the upsampling block.

Residual dense block

In this model, we employ a residual dense block with PReLU activation functions to process images. This structure not only extracts high-level features, maintaining excellent performance as the network deepens, but also captures multi-scale features from the images. Notably, we combine the PReLU function with the residual dense block. Unlike traditional rectified linear unit (ReLU), the PReLU function introduces learnable parameters, enhancing the network’s adaptability and robustness. This addition effectively mitigates gradient vanishing and improves the network’s expressiveness and training efficiency. As shown in Fig. 3, the residual dense block used in this article consists of five layers, with PReLU activation functions in the first four layers. The results of the final layer are output directly.

Why choose five layers? The number of layers in the residual dense block plays a crucial role in the model’s overall performance. To determine the optimal number of dense block layers for our model, we analyzed and referenced several studies (He et al., 2016a, 2016b; Li et al., 2019; He et al., 2017; Shin, 2022; Bejani & Ghatee, 2021; Zhu et al., 2022; Liu & Liu, 2022), on the relationship between network depth and performance. These studies suggest that a suitable depth of dense blocks can enhance the model’s learning capacity and extract deep features from the data. However, an excessive number of layers in dense blocks may reduce effectiveness, increase training time, and heighten the risk of overfitting. Therefore, we initially established that the depth of the dense blocks should be kept between four and seven layers. To finalize the layer count, we referenced recent advanced image hiding methods (Jing et al., 2021; Lu et al., 2021; Xu et al., 2022; Yang et al., 2024) that employ residual dense block structures. These studies consistently showed that a five-layer residual structure performs optimally for image hiding and restoration tasks, effectively extracting deep features without creating an overly complex, hard-to-train network prone to overfitting. Consequently, this article adopts a five-layer structure for the residual dense block.

Invertible block

As shown in Fig. 1, within the invertible block, three interactions occur between the cover image and the secret image after wavelet transformation. To describe more clearly how the invertible blocks process information during the forward hiding process, this article provides the corresponding formulas. Specifically, $\odot$ represents element-wise matrix multiplication, $k$ denotes the k-th block, $K1$ and $K2$ represent the residual dense block and the upsampling convolution block, $cat(a,b,c)$ represents concatenating tensors $a$ and $b$ along dimension $c$, $\epsilon$ signifies a 3 × 3 convolution, and $\varphi$ represents the sigmoid function. For the k-th concealing block in the forward hiding process, the inputs are $x_{cover}^k$ and $x_{secret}^k$, and the outputs $x_{cover}^{k+1}$ and $x_{secret}^{k+1}$ are expressed as follows:

(1) $$x_{cover}^{k+1}=x_{cover}^k+\epsilon (cat(K1(x_{secret}^k),K2(x_{secret}^k),1))$$ (2) $$x_{secret}^{k+1}=x_{secret}^k\odot exp(\varphi (K1(x_{cover}^{k+1})))+\epsilon (cat(K1(x_{cover}^{k+1}),K2(x_{cover}^{k+1}),1)).$$

In the reverse recovery process, the flow of information is reversed compared to the forward hiding process. This means that the way in which invertible blocks process information in the reverse recovery process is the opposite of that in the forward hiding process. To clarify how the invertible blocks handle information during the reverse recovery process, this article provides the corresponding formulas. Specifically, $\odot$, $k$, $K1$, $K2$, $cat(a,b,c)$, $\epsilon$, and $\varphi$ retain the same meanings as in the hiding process. For the k-th recovery block in the reverse recovery process, the inputs are $w^{k+1}$ and $x_{stego}^{k+1}$, and the outputs $w^k$ and $x_{stego}^k$ are expressed as follows:

(3) $$w^k=(w^{k+1}-\epsilon (cat(K1(x_{stego}^{k+1}),K2(x_{stego}^{k+1}),1)))\odot exp(-\varphi (K1(x_{stego}^{k+1})))$$ (4) $$x_{stego}^k=x_{stego}^{k+1}-\epsilon (cat(K1(w^k),K2(w^k),1)).$$

Hiding loss and recovery loss

As our model is based on invertible neural networks, it is essential to use reconstruction loss to ensure the integrity and reversibility of the model. The objective of reconstruction loss is to minimize the reconstruction error between the model’s output and input, enabling the model to accurately reconstruct the input data. In this article, we chose the mean absolute error (MAE) loss function due to its robustness and consistency in the value domain. The formula for the hiding loss is as follows:

(5) $$LOS{S}_{Hi}=\sum _{i=1}^{N}F(x_{cover}^i,x_{stego}^i)$$where $LOS{S}_{Hi}$ represents the total hiding loss, N denotes the total number of training samples, and $i$ denotes the i-th training sample. F is used to measure the discrepancy between the cover image $x_{cover}^i$ and the stego image $x_{stego}^i$. In this context, F refers to the MAE loss function. For the recovery loss, we express it as follows:

(6) $$LOS{S}_{Rev}=\sum _{i=1}^{N}F(x_{secret}^i,x_{secret-rev}^i).$$

Here, $LOS{S}_{Rev}$ represents the total recovery loss, N denotes the total number of training samples, and $i$ signifies the i-th training sample, F still denotes the MAE loss, which is used to measure the discrepancy between the genuine secret image $x_{secret}^i$ and the recovered secret image $x_{secret-rev}^i$.

Frequency domain loss

In the image hiding process, selecting the appropriate region for embedding the secret information is crucial. After performing the wavelet transform on the cover image, four subdomains are generated: High-High (HH), High-Low (HL), Low-High (LH), and Low-Low (LL). Among these, the HH domain contains the highest amount of high-frequency information, followed by LH and HL domains, and the LL domain contains the least. The more high-frequency information a subdomain contains, the more detailed and textured the information is. Since the human eye is less sensitive to high-frequency details and textures compared to low-frequency information, secret data hidden in high-frequency regions is less likely to be detected by the human eye than in low-frequency domains. Therefore, subdomains with higher high-frequency information are more suitable for embedding secret information than those with lower high-frequency content. Existing image hiding methods often lack precision in selecting the appropriate regions for hiding secret data, which can result in a deterioration of the quality of the generated images.

To address this issue, this article introduces a frequency domain loss (FDL) to ensure that secret information is prioritized for hiding in subdomains containing more high-frequency information. Specifically, FDL aims to constrain the LL domain of the cover image to be as identical as possible to the LL domain of the stego image, while maintaining a certain level of similarity between the LH and HL domains of both images (with the constraint strength being weaker than that of the LL domain). In this way, secret information is primarily hidden in the HH domain, as this domain is not constrained by the loss function. If the HH domain’s capacity is insufficient to store all the secret information, the excess will be hidden in the LH and HL domains, which are subject to weaker constraints. Only when the capacities of the HH, HL, and LH domains are all insufficient will the remaining secret information be hidden in the LL domain. The formula for the FDL is as follows:

(7) $$LOS{S}_{FD-LL}=\sum _{i=1}^{N}F(\tau (x_{cover}^i{)}_{LL},\tau (x_{stego}^i{)}_{LL})$$

(8) $$LOS{S}_{FD-LH}=\sum _{i=1}^{N}F(\mu (x_{cover}^i{)}_{LH},\mu (x_{stego}^i{)}_{LH})$$

(9) $$LOS{S}_{FD-HL}=\sum _{i=1}^{N}F(\sigma (x_{cover}^i{)}_{HL},\sigma (x_{stego}^i{)}_{HL})$$

(10) $$LOS{S}_F=0.5\ast (LOS{S}_{FD-HL}+LOS{S}_{FD-LH})+LOS{S}_{FD-LL}$$where $LOS{S}_{FD-LL}$ represents the LL loss, $LOS{S}_{FD-LH}$ signifies the LH loss, $LOS{S}_{FD-HL}$ denotes the HL loss and $LOS{S}_F$ represents the total FDL. $\tau$ represents the operation of extracting LL frequency sub-bands after wavelet transformation, $\mu$ represents the operation of extracting LH frequency sub-bands after wavelet transformation, $\sigma$ represents the operation of extracting HL frequency sub-bands after wavelet transformation, and F still denotes the MAE loss function.

Why is the weight for $LOS{S}_{FD-HL}+LOS{S}_{FD-LH}$ set to 0.5? As mentioned above, FDL aims to constrain the LL, HL, and LH domains of the wavelet transform of both the cover and stego images, with the constraint on the LL domain being significantly greater than that on the HL and LH domains. Therefore, the weight of $LOS{S}_{FD-HL}+LOS{S}_{FD-LH}$ should be less than 0.8, meaning the constraint on these domains should be significantly weaker than on $LOS{S}_{FD-LL}$ (where a weight of 1 implies equal constraints on the HH, HL, and LH domains). Otherwise, if the HH domain has insufficient capacity, the overflowed secret information will not be preferentially hidden in the LH and HL domains, leading to a deterioration in the quality of the generated results. In addition, the weight of $LOS{S}_{FD-HL}+LOS{S}_{FD-LH}$ should not be less than 0.3, as this would result in an excessively weak constraint from the loss function, causing some secret information to be hidden in the HL or LH domains even when the HH domain still has sufficient capacity. This would, in turn, lead to a deterioration in the quality of the generated results. In conclusion, we can initially set the weight range for $LOS{S}_{FD-HL}+LOS{S}_{FD-LH}$ to be between 0.3 and 0.7. To determine the final weight, we conducted experiments on 400 natural images from the COCO (2017 Test images) (Lin et al., 2014) dataset, testing weights between 0.3 and 0.7. The results are shown in Table 2, and a weight of 0.5 was found to be the optimal choice.

Total loss

The total loss is the linear sum of the hiding loss, recovery loss, and frequency domain loss, where $\gamma$1, $\gamma$2, and $\gamma$3 represent their respective weights. The overall formula is as follows:

(11) $$LOS{S}_{Total}=\gamma 1\ast LOS{S}_{Hi}+\gamma 2\ast LOS{S}_{Rev}+\gamma 3\ast LOS{S}_F.$$

During the training process, we use the gradient descent method to minimize the total loss, thereby forcing the model to achieve the desired outcomes.

Experimental datasets

This article utilizes the DIV2K (https://data.vision.ee.ethz.ch/cvl/DIV2K/) (Agustsson & Timofte, 2017) training dataset for network training, which comprises 800 natural images. The testing dataset consists of 100 natural images from the DIV2K testing dataset, 400 natural images from the COCO (https://cocodataset.org/#download) (Lin et al., 2014) testing dataset, and 5,500 natural images from the ImageNet (https://www.image-net.org/challenges/LSVRC/index.php) (Russakovsky et al., 2015) testing dataset. All experiments were conducted on an NVIDIA GeForce RTX 3080 GPU with 10 GB of VRAM.

Data preprocessing

Due to the nature of the datasets used in this study (DIV2K, COCO, and ImageNet), which provide raw image data, preprocessing is necessary before model training and testing. Specifically, for the training dataset, all images are normalized to ensure that pixel values are within the range of 0 to 1, thereby enhancing training stability and model performance. Additionally, this study applies data augmentation techniques such as random cropping, horizontal flipping, and rotation to the training data. This approach increases data diversity and enhances the model’s generalization capability. For the testing dataset, we perform center cropping and standardization but do not apply data augmentation techniques like horizontal flipping and rotation. This ensures that the model’s performance in real-world conditions accurately reflects its capabilities.

Experimental parameters

This article resizes images to $224\times 224$ for network training, uses a batch size of 4, sets the initial learning rate to $1\ast {10}^{-3.9}$, reduces the learning rate to $1\ast {10}^{-4.4}$ after five epochs, and then halves it every 260 epochs, with a total of 2,000 epochs. The model is trained using the Adam optimizer (Qin et al., 2013). For other model hyperparameters, the number of invertible blocks M is set to 18, the initial values of $\gamma 1$, $\gamma 2$, $\gamma 3$ are all set to 1, with manual fine-tuned during multiple training iterations.

Evaluation metrics

This article selects Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), average percentage difference (APD), and root mean square error (RMSE) as the primary evaluation metrics. Compared to other metrics, these are more appropriate for image hiding tasks, as they assess image quality from various perspectives, offering a comprehensive view of the effects of image hiding and recovery.

Specifically, SSIM provides a quality assessment that aligns more closely with human visual perception by considering image structure. PSNR, as a widely used and easily computable standard, can quickly reflect the effectiveness of image reconstruction. APD focuses on revealing the relative error between predicted and actual values, effectively reflecting the model’s performance in detail handling. RMSE quantifies the overall error, offering an intuitive quantitative basis for model evaluation. The following is a detailed introduction to these metrics:

PSNR is a widely used metric for measuring image or video quality. It is employed to evaluate the similarity between the original signal, typically an image or video, and the reconstructed signal after compression, distortion, or other processing. The calculation formula is as follows, where MSE represents the mean squared error:

(12) $$PSNR=20lo{g}_{10}(ma{x}_x)-10lo{g}_{10}(MSE)$$

The SSIM is based on principles of human perception. It calculates similarity by comparing various aspects such as structure, luminance, and contrast between the original and reconstructed signals. SSIM takes into account not only the mean squared error but also losses in structural information and perceptual distortions. The formula is represented as follows:

(13) $$SSIM=\frac{(2{\mu }_x{\mu }_y+C_1)(2{\sigma }_{xy}+C_2)}{({\mu }_x^2+{\mu }_y^2+C_1)({\sigma }_x^2+{\sigma }_y^2+C_2)}.$$

APD represents the average percentage difference between predicted and actual values, reflecting the relative error in the predicted values compared to the actual values. The calculation formula is as follows:

(14) $$APD=\frac{1}{n}\ast \sum _{i=1}^N\left|\frac{y_i-{\hat{y}}_i}{y_i}\right|\ast 100.$$

RMSE is based on the units of actual values and is used to measure the overall difference between predicted and actual values. The calculation formula is as follows:

(15) $$RMSE=\sqrt{\frac{1}{n}\ast \sum _{i=1}^N{\left(y_i-{\hat{y}}_i\right)}^2}.$$

For PSNR and SSIM, a higher value indicates better results, whereas for APD and RMSE, a lower value signifies better performance.

Quantitative results

Table 3 compares our results with the methods of four bit-LSB (Baluja, 2017; Weng et al., 2019), practical robust invertible network for image steganography (PRIS) (Yang et al., 2024), joint adjustment image steganography networks (JAIS-Nets) (Zhang et al., 2023b), and HiNet (Jing et al., 2021). As shown in Table 3, it is evident that MFI-Net outperforms all other methods across all four metrics of cover/stego and secret/secret-rev image pairs. In the DIV2K dataset, the PSNR and SSIM of the cover/stego image pair generated by MFI-Net are 3.22 dB and 0.001 higher, respectively, than the second best results. The APD and RMSE are 0.96 and 1.22 lower, respectively, than the second-best results. For secret/secret-rev image pairs, MFI-Net also performs exceptionally well. Additionally, on the COCO (Lin et al., 2014) and ImageNet (Russakovsky et al., 2015) datasets, MFI-Net performs exceptionally well, demonstrating its strong generalization capability.

Statistical analysis

Box plot results. Figure 4 presents the box plots of PSNR results generated by MFI-Net and other different methods on the COCO dataset. The experimental results demonstrate that MFI-Net outperforms all other methods for both image pairs (Cover/Stego and Secret/Secret-rev). Specifically, the PSNR values of MFI-Net’s generated results are the highest and relatively concentrated, significantly outperforming other methods. In contrast, PRIS (Yang et al., 2024) and Hi-Net (Jing et al., 2021) also perform well, with relatively high and concentrated PSNR values. Other methods, however, have lower PSNR values or a wider distribution range, indicating relatively unstable performance.

Analysis of variance. In addition to using box plots to display the distribution of PSNR across different methods, this article further quantifies the differences in steganographic performance between MFI-Net and other methods through analysis of variance (ANOVA), as shown in Fig. 5. It can be observed that, for both cover/stego image pairs and secret/secret-rev image pairs, the −log10(P-value) values between MFI-Net and the various methods are all significantly greater than 1.3 (the −log10 value at p = 0.05). This indicates that the improvement in steganographic performance of MFI-Net is significant and not due to chance, demonstrating MFI-Net’s exceptional performance and strong generalization capability in steganographic tasks.

Qualitative results

Figure 6 presents the quantitative results of each method. The images generated by the four bit-LSB, Baluja (2017), and Weng et al. (2019) methods exhibit varying degrees of visible distortion. Specifically, the four bit-LSB method produces images with significant distortion. Baluja’s (2017) method shows some improvement in quality, but noticeable blurring and artifacts remain. Weng et al.’s (2019) method generates higher-quality images, though slight distortions are still visible. In contrast, the images produced by JAIS-Nets (Zhang et al., 2023b), PRIS (Yang et al., 2024), Hi-Net (Jing et al., 2021), and MFI-Net exhibit almost no visible distortion. To illustrate this more clearly, the corresponding residual images are provided. From these residual images, it is evident that the images generated by MFI-Net are closest to the original images, with the least residual noise, demonstrating the best performance.

Effectiveness of the upsampling convolution block

In Table 4, we present the results of ablating the proposed UCB. It can be observed that, after removing the UCB, the PSNR and SSIM of the cover/stego image pair decreased by 3.68 dB and 0.007, respectively, while the APD and RMSE increased by 0.46 and 0.5. These changes highlight the effectiveness of the UCB module.

For the secret/secret-rev image pair, after removing this module, we observed a decrease of 0.04 dB in PSNR, no change in SSIM, but a decrease of 0.042 and 0.024 in APD and RMSE, respectively. Our investigation suggests that this is a normal occurrence, given the specific nature of invertible neural networks, where the quality of stego images and secret-rev images is relative. A better-hidden image implies greater difficulty in recovery. Most importantly, the improvement in the quality of stego images due to this module far outweighs the subtle effects on secret image recovery.

Effectiveness of frequency domain loss

Table 4 shows the results after removing the FDL. For the cover/setgo image pair, the PSNR and SSIM decreased by 1.42 dB and 0.002, while APD and RMSE increased by 0.04 and 0.23. For the secret/secret-rev image pair, the PSNR and SSIM decreased by 8.72 dB and 0.004, APD and RMSE increased by 0.488 and 0.669. These changes highlight the effectiveness of the proposed FDL in this study.

In addition to separately ablating the UCB and FDL, this article also tested the model with both components ablated. The results show that, compared to the ablated model, the original model generates a cover/stego image pair with PSNR and SSIM values higher by 4.89 dB and 0.016, respectively, while APD and RMSE are lower by 1.14 and 1.42. The generated secret/secret-rev image pair exhibited a higher PSNR and SSIM by 7.53 dB and 0.003, respectively, while APD and RMSE were reduced by 0.39 and 0.521. These results validate the effectiveness and compatibility of the UCB and FDL proposed in this article.

Figure 7 illustrates the results generated by the model after removing different modules along with their corresponding residual maps. It can be observed that for stego images, removing both the UCB and the FDL results in a decrease in image quality, with the UCB having a greater impact. For the recovered secret images, removing the FDL leads to a substantial reduction in image quality, while removing the UCB slightly enhances image quality. However, the improvement in stego image quality due to the UCB outweighs the minimal effect on secret image recovery. Therefore, the UCB is retained in the model.