A new approach for SPN removal: nearest value based mean filter

Definitions and notions

This paper, let $\mathit{f}\left(\mathit{x},\mathit{y}\right)={\left[{\mathit{f}}_{\mathit{i}\mathit{j}}\right]}_{\mathit{m}\mathbf{\times }\mathit{n}}$ be an original image, ${\mathit{f}}^{\mathbf{\prime }}\left(\mathit{x},\mathit{y}\right)={\left[{\mathit{f}}_{\mathit{i}\mathit{j}}^{\mathbf{\prime }}\right]}_{\mathit{m}\mathbf{\times }\mathit{n}}$ restored image, $\mathit{y}\left(\mathit{x},\mathit{y}\right)={\left[{\mathit{y}}_{\mathit{i}\mathit{j}}\right]}_{\mathit{m}\mathbf{\times }\mathit{n}}$ 3^th stage result image, $\mathit{g}\left(\mathit{x},\mathit{y}\right)={\left[{\mathit{g}}_{\mathit{i}\mathit{j}}\right]}_{\mathit{m}\mathbf{\times }\mathit{n}}$ salt and pepper noisy image, and ${\mathit{g}}^{\mathbf{\prime }}\left(\mathit{x},\mathit{y}\right)\equiv \mathit{g}\left(\mathit{x},\mathit{y}\right)\left(\mathit{m}\mathit{o}\mathit{d}255\right)$ hence only pepper noisy image in ${\mathit{g}}^{\mathbf{\prime }}\left(\mathit{x},\mathit{y}\right)={\left[{\mathit{g}}_{\mathit{i}\mathit{j}}^{\mathbf{\prime }}\right]}_{\mathit{m}\mathbf{\times }\mathit{n}}\mathbf{i}\mathbf{n}\mathit{m}\mathbf{\times }\mathit{n}\mathbf{s}\mathbf{i}\mathbf{z}\mathbf{e}\mathbf{s}$.

Definition 1. In the gray level images, the smallest value ( ${\mathit{\gamma }}_{\mathit{m}\mathit{i}\mathit{n}}$) is 0, and the largest value ( ${\mathit{\gamma }}_{\mathit{m}\mathit{a}\mathit{x}}$) is 255. SPN (salt and pepper noisy) and PN (pepper noisy) models with the same noise levels as follows:

$${\mathit{g}}_{\mathit{i}\mathit{j}}=\{\begin{array}{c}{\mathit{\gamma }}_{\mathit{m}\mathit{i}\mathit{n}},withprobabilityp\\ {\mathit{\gamma }}_{\mathit{m}\mathit{a}\mathit{x}},withprobabilityq\\ {\mathit{f}}_{\mathit{i}\mathit{j}},withprobability1-\left(\mathit{p}+\mathit{q}\right)\end{array}\cdots$$

$${\mathit{g}}_{\mathit{i}\mathit{j}}^{\mathbf{\prime }}={\mathit{g}}_{\mathit{i}\mathit{j}}\left(\mathit{m}\mathit{o}\mathit{d}255\right)\cdots$$

(1) $${\mathit{g}}_{\mathit{i}\mathit{j}}^{{}^{\mathbf{\prime }}}=\{\begin{array}{c}{\mathit{\gamma }}_{\mathit{m}\mathit{i}\mathit{n}},withprobabilityp+q\\ {\mathit{f}}_{\mathit{i}\mathit{j}},withprobability1-\left(p+q\right)\end{array}$$

Definition 2. $\mathit{N}$L is noisy levels. Noise level is the ratio of the number of zeros to the total number of pixels in ${\mathit{g}}_{\mathit{i}\mathit{j}}^{{}^{\mathbf{\prime }}}$. ${\mathit{Z}}_{\mathit{i}\mathit{j}}$ binary matrix in $\mathit{m}\mathbf{\times }\mathit{n}$ sizes.

(2) $${\mathit{Z}}_{\mathit{i}\mathit{j}}=\{\begin{array}{c}1,{\mathit{g}}_{\mathit{i}\mathit{j}}^{{}^{\mathbf{\prime }}}=0\\ 0,{\mathit{g}}_{\mathit{i}\mathit{j}}^{{}^{\mathbf{\prime }}}\ne 0\end{array}$$

(3) $$\mathit{N}\mathit{L}={\displaystyle \frac{{\sum }_{\mathit{i}\mathit{j}}{\mathit{Z}}_{\mathit{i}\mathit{j}}}{\mathit{m}\mathbf{\times }\mathit{n}}}$$

Definition 3. ${\mathit{S}}_{\mathit{i}\mathit{j}}\left(\mathit{w}\right)$ be a image window in ${\left[{{\mathit{g}}^{\mathbf{\prime }}}_{\mathit{i}\mathit{j}}\right]}_{\mathit{m}\mathbf{\times }\mathit{n}}$. It is size of $11\mathbf{\times }11$ with central pixel coordinates (i,j). The pixel value with the smallest Euclidean distance or average of pixels with the same smalest Euclidean distance ( $\mathit{n}\mathit{p}\mathit{v}$).

(4) $${\mathit{S}}_{\mathit{i}\mathit{j}}^{\mathit{n}\mathit{p}\mathit{v}}\left(\mathit{w}\right)=\{\begin{array}{c}{\mathit{K}}_{\mathit{i}\mathit{j}}\left(w\right),{\mathit{g}}_{\mathit{i}\mathit{j}}^{{}^{\mathbf{\prime }}}=0\\ {\mathit{g}}_{\mathit{i}\mathit{j}}^{{}^{\mathbf{\prime }}},otherwise\end{array}$$

(5) $${\mathit{\delta }}_{{\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}}=\{\begin{array}{c}\sqrt{{\left(\mathit{i}-{\mathit{i}}^{\mathbf{\ast }}\right)}^2+{\left(\mathit{j}-{\mathit{j}}^{\mathbf{\ast }}\right)}^2},{\mathit{S}}_{\mathit{i}\mathit{j}}\ne 0\\ \mathrm{\varnothing },{\mathit{S}}_{\mathit{i}\mathit{j}}=0\end{array}$$

(6) $${\mathbf{K}}_{\mathbf{i}\mathbf{j}}(\mathbf{w})=\{\begin{array}{c}{{\mathbf{g}}^{\mathrm{\prime }}}_{{\mathbf{i}}^{\ast }{\mathbf{j}}^{\ast }},\mathbf{I}\mathbf{f}\mathbf{t}\mathbf{h}\mathbf{e}\mathbf{r}\mathbf{e}\mathbf{i}\mathbf{s}\mathbf{o}\mathbf{n}\mathbf{l}\mathbf{y}\mathbf{o}\mathbf{n}\mathbf{e}\mathbf{p}\mathbf{i}\mathbf{x}\mathbf{e}\mathbf{l}\mathbf{w}\mathbf{i}\mathbf{t}\mathbf{h}\mathbf{d}\mathbf{i}\mathbf{s}\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{c}\mathbf{e}\delta {}_{\mathbf{i}\mathbf{j}}^{min}\left(\mathbf{w}\right)\hfill \\ {\mathbf{g}}^{\mathrm{\prime }}\mathbf{M}\mathbf{e}\mathbf{a}{\mathbf{n}}_{{\mathbf{i}}^{\ast }{\mathbf{j}}^{\ast }},\mathbf{I}\mathbf{f}\mathbf{t}\mathbf{h}\mathbf{e}\mathbf{r}\mathbf{e}\mathbf{i}\mathbf{s}\mathbf{t}\mathbf{o}\mathbf{o}\mathbf{m}\mathbf{a}\mathbf{n}\mathbf{y}\mathbf{p}\mathbf{i}\mathbf{x}\mathbf{e}\mathbf{l}\mathbf{s}\mathbf{w}\mathbf{i}\mathbf{t}\mathbf{h}\mathbf{d}\mathbf{i}\mathbf{s}\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{c}\mathbf{e}\delta {}_{\mathbf{i}\mathbf{j}}^{min}\left(\mathbf{w}\right)\hfill \\ {{\mathbf{y}}^{\mathbf{M}\mathbf{e}\mathbf{a}\mathbf{n}}}_{\mathbf{i}\mathbf{j}},\mathbf{I}\mathbf{f}\mathbf{t}\mathbf{h}\mathbf{e}\mathbf{r}\mathbf{e}\mathbf{i}\mathbf{s}\mathbf{n}\mathbf{o}\mathbf{p}\mathbf{i}\mathbf{x}\mathbf{e}\mathbf{l}\mathbf{s}\mathbf{w}\mathbf{i}\mathbf{t}\mathbf{h}\mathbf{d}\mathbf{i}\mathbf{s}\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{c}\mathbf{e}\delta {}_{\mathbf{i}\mathbf{j}}^{min}\left(\mathbf{w}\right)\hfill \end{array}$$

Definition 4. ${\mathit{\beta }}_{\mathit{i}\mathit{j}}^{\mathit{M}\mathit{e}\mathit{a}\mathit{n}}\left(\mathit{w}\right)$ be a image window in ${\left[{\mathit{y}}_{\mathit{i}\mathit{j}}\right]}_{\mathit{m}\mathbf{\times }\mathit{n}}$. It is size of $3\mathbf{\times }3$ with central pixel coordinates (i,j). ${\mathit{R}}_{\mathit{i}\mathit{j}}$ is the average of nonzeros ${\mathit{y}}_{{\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}}$ values.

(7) $${\mathit{\beta }}_{\mathit{i}\mathit{j}}^{\mathit{M}\mathit{e}\mathit{a}\mathit{n}}\left(\mathit{w}\right)=\{\begin{array}{c}{\mathit{R}}_{\mathit{i}\mathit{j}}\left(\mathit{w}\right),NL>0.45\\ {\mathit{y}}_{\mathit{i}\mathit{j}},otherwise\end{array}$$

(8) $${\mathit{R}}_{\mathit{i}\mathit{j}}\left(\mathit{w}\right)={\displaystyle \frac{{\sum }_{\left({\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}\right)\in {\mathit{\beta }}_{\mathit{i}\mathit{j}}\left(\mathit{w}\right)}{\mathit{y}}_{{\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}}}{{\sum }_{\left({\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}\right)\in {\mathit{\beta }}_{\mathit{i}\mathit{j}}\left(\mathit{w}\right)}{\mathit{D}}_{{\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}}}}$$

(9) $${\mathit{D}}_{{\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}}=\{\begin{array}{c}1,{\mathit{y}}_{{\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}}\ne 0\\ 0,{\mathit{y}}_{{\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}}=0\end{array}$$

Proposed method

The IAWMF method is the development of the AWMF method of determining the weights. In this method, the weights of the noisy pixels are taken to 0, while the weights of the noisy pixels are associated with the Euclidean distance to the pixel to be filtered. Thus, the weight of the neighbouring pixel value nearly to the pixel to be filtered is provided to be higher. In this method, the weight values are determined by Eq. (10).

(10) $${\mathit{D}}_{{\mathit{i}}^{\mathbf{\ast }}{\mathit{j}}^{\mathbf{\ast }}}={\displaystyle \frac{1}{{\left(\mathit{\epsilon }+\sqrt{{\left(\mathit{i}-{\mathit{i}}^{\mathbf{\ast }}\right)}^2+{\left(\mathit{j}-{\mathit{j}}^{\mathbf{\ast }}\right)}^2}\right)}^4},0<\mathit{\epsilon }\ll 1}$$

It is a positive approach to determine the weights by Euclidean distance in the IAWMF method. However, the association of weight with the fourth power of Euclidean distance causes the weights of long-distance pixels to get too small. Therefore, pixel values at the nearest distance predominantly affect the pixel value to be filtered.

The method suggested in the study is based on the calculation of the pixel value to be filtered in two stages. In the first stage, the nearest noiseless pixel value is assigned to the pixel to be filtered. If there is more than one pixel with the same distance value, their averages are assigned as pixels values. At this stage, the filter size 11 × 11 is used. The reason for this is to guarantee the presence of noiseless pixels in the filter even at high level noise. However, because noise is randomly distributed throughout the image, even a large filter size may sometimes not guarantee noiseless pixels in the frame. In this case, it is assigned as the pixel value to be filtered by taking the average of the previously filtered pixel values.

Images with a low level noise usually have multiple pixel with the lowest Euclidean distance. Thus, the noisy pixel value in the first stage is calculated by taking the average of these pixels. As the noise ratio in the image increases, the number of pixels with the lowest Euclidean distance will decrease; whereas at high levels of noise, there will usually be only one pixel with the lowest Euclidean distance. In this case, this value will be assigned to the noisy pixel.

In the first stage, the closest pixel value is usually assigned without averaging in images with a high noise ratio. Although this value is close to the original value, this prevents the image from reaching sufficient quality in terms of resolution. For this reason, in cases where the noise ratio is above 45%, the second stage is applied after the first stage. At second stage, an average filter at 3 × 3 size is applied. Pixels with the value of 0 are ignored when applying the Average filter. The algorithm and flow chart of the proposed method is given below (Fig. 1).

Quality metrics

In this section, image quality metrics are provided in order to compare denoising filters used for salt and peppers noisy image. Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM) and Image Enhancement Factor (IEF) were used in the study to evaluate the image quality. PSNR and MSE (Enginoğlu, Erkan & Memiş, 2019; Turan, 2021; Olmez, Sengur & Özmen Koca, 2020) defined as:

(11) $$PSNR\left(X,Y\right)=10lo{g}_{10}\left[{\displaystyle \frac{{\left(L-1\right)}^2}{MSE}}\right]$$

(12) $$MSE\left(X,Y\right)={\displaystyle \frac{1}{MN}{\sum }_{i=1}^{i=M}{\sum }_{j=1}^{j=N}{\left[X\left(i,j\right)-Y\left(i,j\right)\right]}^2}$$

SSIM (Turan, 2021; Olmez, Sengur & Özmen Koca, 2020; Wang et al., 2004) defined as:

(13) $$SSIM\left(X,Y\right)={\left[I\left(X,Y\right)\right]}^{\alpha }\ast {\left[c\left(X,Y\right)\right]}^{\beta }\ast {\left[s\left(X,Y\right)\right]}^{\gamma }$$

(14) $$I\left(X,Y\right)={\displaystyle \frac{2{\mu }_X{\mu }_{Y+}{C}_1}{{\mu }_X^2+{\mu }_Y^2+C_1}}$$

(15) $$c\left(X,Y\right)={\displaystyle \frac{2{\sigma }_X{\sigma }_{Y+}{C}_2}{{\sigma }_X^2+{\sigma }_Y^2+C_2}}$$

(16) $$s\left(X,Y\right)={\displaystyle \frac{{\sigma }_{XY}+C_3}{{\sigma }_X{\sigma }_Y+C_3}}$$

If $\alpha =\beta =\gamma =1\mathrm{a}\mathrm{n}\mathrm{d}{C}_3=C_2/2,\Rightarrow$ $SSIM\left(X,Y\right)$ is organized as follows.

(17) $$SSIM\left(X,Y\right)={\displaystyle \frac{\left(2{\mu }_X{\mu }_{Y+}{C}_1\right)\left(2{\sigma }_{XY}+C_2\right)}{\left({\mu }_X^2+{\mu }_Y^2+C_1\right)\left({\sigma }_X^2+{\sigma }_Y^2+C_2\right)}}$$

IEF (Enginoğlu, Erkan & Memiş, 2019; Djurovi¢, 2017) defined as:

(18) $$IEF\left(X,Y,Z\right)={\displaystyle \frac{{\sum }_{i=1}^M{\sum }_{j=1}^N{\left(z_{ij}-x_{ij}\right)}^2}{{\sum }_{i=1}^M{\sum }_{j=1}^N{\left(y_{ij}-x_{ij}\right)}^2}}$$

In the above equations, $X=\left[x_{ij}\right]$ is an original image, $Y=\left[y_{ij}\right]$ is denoised image, $Z=\left[z_{ij}\right]$ is a noisy image.

Image databases

In the study, three different image datasets are used to compare the methods:

1. UC-Berkeley dataset (BSDS)—200 images (Arbelaez, Fowlkes & Martin, 2007);

2. TESTIMAGES dataset—40 images (Asuni & Giachetti, 2015);

3. MATLAB library images—20 images (R2020b; autumn, baby, board, micromarket, car1, coloredChips, fabric, foggyroad, foggysf1, foosball, football, greens, gantrycrane, trailer, hallway, hands1, pears, kobi, lighthouse, onion).

MATLAB Library images can be accessed from a computer with the MATLAB R2020b program installed using the given directory (MATLAB\R2020b\toolbox\images\imdata).

The proposed method and other methods were compared using datasets. Noise was added to each image at the ratios of 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9. For example, 1,800 PSNR, SSIM, and IEF results were obtained for BSDS. The average of all levels of noise was calculated separately for each image quality metric (PSNR, SSIM, IEF). In addition, the general average of all levels of noise has been added to the tables.

In addition, all methods were compared for six images (three from datasets and three from outside of datasets). Thus, it was ensured that the individual performances of all methods on images were observed.

Statistical tests to compare performance between methods

In this section, the significance made of the comparisons between the method developed in this study and the state-of-the-art methods were tested. The significance test was performed with Wilcoxon signed-rank test because the data did not show normal distribution (Hung et al., 2022; Cevahir, 2020). The effect value of significance was determined by the Pearson Correlation Coefficient (r) (Cevahir, 2020). The results obtained are given in Tables 13–15.

The method developed in this study was subjected to the Wilcoxon signed-rank test separately with each method used for comparison. Cases with p < 0.05 as a result of the test indicate that there is a significant difference between the methods compared. The cases where the Z value is negative (−) mean that there is a difference in favor of the developed method. Pearson correlation coefficient (r) expresses the effect value. A Pearson correlation coefficient of 0.1 is considered a small effect, 0.3 is considered a medium effect, and 0.5 is considered a large effect (Cevahir, 2020).

When Table 13 is examined, it can be seen that there is a significant difference in favor of the developed method in all comparisons of PSNR and SSIM between the developed method and the compared methods. In IEF comparisons, it can be seen that there is a significant difference in favor of the method developed in the others, except for one. It is seen that a significant difference creates a small effect in the IAWMF SSIM comparison, a significant difference creates a medium effect in the IAWMF IEF, DAMRmF comparisons. No significant difference was found in the DAMRmF IEF comparison. And a significant difference creates a large effect in all other comparisons.

When Table 14 is examined, it can be seen that there is a significant difference in favor of the developed method in all comparisons of SSIM, and IEF between the developed method and the compared methods. In PSNR comparisons, it can be seen that there is a significant difference in favor of the method developed in the others, except for one. In comparisons of IAWMF SSIM, IAWMF IEF, SFT_lp SSIM and ARmF IEF, it can be seen that a significant difference creates a medium effect. In comparisons of SFT_lp IEF, it can be seen that a significant difference creates a small effect. No significant difference was found in the SFT_lp PSNR comparison. And a significant difference creates a large effect in all other comparisons.

When Table 15 is examined, it can be seen that there is a difference in favor of the method developed according to Z values in all comparisons, except SFT_lp IEF. However, some of them were found to have significant differences. From PSNR comparisons, it can be seen that DAMF has a small effect, AFMF and SFT_lp have a medium effect, ASWMF, MSCF-1, DAMRmF, Bilal’s Method have a large effect. From the SSIM comparisons, it can be seen that the IMF has a small effect, AWMF-DAMF-TSF have a medium effect, and AFMF-ASWMF-MSCF-1-DAMRmF-SFT_lp-Bilal’s Method have a large effect. From the IEF comparisons, it can be seen that AWMF-DAMF-TSF has a small effect, DAMRmF has a medium effect and AFMF-ASWMFMSCF-1-Bilal’s Mehod have a large effect. In the only SFT_lp IEF comparison, a small significant difference is in favor of SFT_lp. The effect rates in other comparisons were not found to be significant.

It has been determined that there are differences in favor of the developed method in all of the comparisons except for three. According to the results obtained in the Berkeley dataset and the significance test results: Significant differences were found in favor of the developed method in all of the significance tests for PSNR and SSIM. Significant differences were found in 11 of 12 tests for IEF results. Only one found a difference in favor of SFT_lp, but this difference was not significant (does not meet the p < 0.05) condition. According to the results obtained in the TESTIMAGES dataset and the significance test results: There were significant differences in favor of the developed method in all of the significance tests for SSIM. Differences were found in favor of the developed method in all of the tests performed for PSNR. Except for one of these differences (does not meet the p < 0.05) condition, 11 are significant. Significant differences were found in favor of the method developed in 11 of the significance tests for IEF and against the method developed in one (SFT_lp). According to the significance test results for MATLAB library images: There were differences in favor of the developed method in all tests for PSNR. Except for five of these differences (does not meet the p < 0.05) condition, seven of them are significant. Differences were found in favor of the developed method in all of the tests performed for SSIM. Except for two of these differences (does not meet the p < 0.05) condition, 10 are significant. There were differences in favor of the developed method in eleven of the tests performed for IEF. All of these differences are significant. A significant difference was found against the developed method in one of them (SFT_lp).

A total of 108 comparisons were made for significance tests. A difference was found in favor of the developed method in 105 of them. A total of 94 of the differences found in favor of the developed method are significant, 11 of them are not. Three differences were found against the developed method. Two of them are significant, one of them is not.

The differences against the developed method are the significance tests with the SFT_lp method. For this reason, there are nine tests in total when SFT_lp and the developed method significance tests are compared one to one. A big difference in favor of the Berkeley PSNR developed method, a big difference in favor of the Berkeley SSIM developed method, and a nonsignificant difference in favor of the Berkeley IEF SFT_lp were determined. Insignificant difference in favor of TESTIMAGES PSNR developed method, moderate difference in favor of TESTIMAGES SSIM developed method, small difference in favor of TESTIMAGES IEF SFT_lp. MATLAB library images have a moderate difference in favor of the PSNR developed method, MATLAB library images have a large difference in favor of the SSIM developed method, MATLAB library images have a small difference in favor of IEF SFT_lp. In total, five of nine comparisons contain significant differences in favor of the developed method. Two of them contain a significant difference in favor of SFT_lp. The other two comparisons do not contain significant difference.

In this case, it can be said that the results obtained with the developed method are better than the compared methods.