A deep learning method for the recognition of solar radio burst spectrum

Channel normalization

It can be seen from Fig. 1A that, due to the influence of the channel effect of observation equipment, the appearance of interference signal noise conceals the real characteristics of the solar radio burst, which makes the dynamic spectra features not obvious enough. To reduce the influence of RFI on the spectrum image, we use the channel normalization method to remove the RFI, and the calculation method is as follows:

(1) $$p^{{}^{\prime }}(x,y)=p(x,y)-{\displaystyle \frac{1}{n}\sum _{y=0}^{n}p(x,y)+{\displaystyle \frac{1}{mn}\sum _{x=0}^m\sum _{y=0}^{n}p(x,y),}}$$where $p(x,y)$ represents the pixel value of the spectrum image on a certain time channel x and frequency channel y, $p^{{}^{\prime }}(x,y)$ is the pixel value after channel normalization. The radio spectrum image after the horizontal stripes are reduced is shown in Fig. 1B. It can be seen that the spectrum image at this time is free of the influence of RFI, and its burst characteristics gradually appear.

Threshold segmentation

The purpose of channel normalization is to reduce the difficulty of dynamic spectra feature recognition caused by instrument noise. However, both burst type and other types of dynamic spectra images have the problem of over-fusion of image backgrounds and features. To further enhance the dynamic spectra features, we use threshold segmentation algorithm to binarize the dynamic spectra. According to the gray value range of spectrum images of from the SBRS, the threshold segmentation algorithm is designed as follows:

(2) $$p^{{}^{\prime }}(x,y)=\{\begin{array}{c}255,p(x,y)>(mod(p)+max(p))/2\hfill \\ 0,p(x,y)\le (mod(p)+max(p))/2\hfill \end{array},$$where $mod(p)$ is the gray value of the maximum value of the spectrum image gray histogram, and $max(p)$ is the maximum value of spectrum image gray. Gray histogram counts the frequency of each gray value in the image, mod (p) can be calculated by first counting the occurrence times of the gray value of the whole spectrum image to obtain the gray histogram, and then finding the gray value of the image with the most occurrence times in the gray histogram, max (p) is the maximum gray value of the whole spectrum image, which can be calculated by sorting all gray values in the image. In particular, they are all computed for the entire dynamic spectrum. The binarization of spectrum images by a threshold segmentation algorithm is to segment the whole image into regions with only 0 and 255 gray levels according to the burst characteristics of the spectrum image and the different gray levels occupied by a quiet background. The processing result is shown in Fig. 1C. It can be seen that the dynamic spectra features are discontinuous bright spots distributed along the frequency channel as a whole.

Morphological closed operation

A morphological operation is an image processing method based on set theory of data morphology for binary image. The most basic morphological operations include dilation and erosion. The mathematical definition of expansion and erosion of image A by structural element B are as follows:

(3) $$A\oplus B=\left\{z|{(B)}_z\cap A\ne \mathrm{\varnothing }\right\}$$

(4) $$A\mathrm{\Theta }B=\left\{z|{(B)}_z\subseteq A\right\}$$

A morphological close operation is used to expand the image first and then corrode it. It can filter the image by filling the concave angle of the image. It can be used to fill the small holes in the image, bridge the small cracks, and keep the total position and shape unchanged (Serra, 1982). The spectrum image after a morphological close operation is shown in Fig. 1D, from which we can see that the close operation repairs the intermittent characteristics of the burst type spectrum image, so that the burst type spectrum image basically contains obvious bright spot vertical stripes. Similarly, the calibration type spectrum image contains large bright and dark areas after morphological close operation; the non-burst type spectrum image is basically comprised of dark areas, but some images also exhibit white noise.

Solar radio spectral data enhancement

In 2016, Chen et al. (2016) established the solar radio spectrum database, and the spectrum image data are from the SBRS. The database collected 4,408 radio spectrum images with a size of 2,520 × 240. These images are divided into three categories: burst for radio spectrum images with burst behavior, non-burst for images without burst behavior, and calibration for images requiring secondary calibration. The distribution of various types of samples in the original dataset is shown in Table 2. It can be seen that the number of non-burst images accounts for 76% of the total number of samples, while the number of the other two types of samples is relatively small. This unbalanced sample distribution will greatly limit the advantages of deep learning, and it will not be able to make full use of all radio data to train the network, which will easily cause the network model to overfit and adversely affect the final experimental results. In previous studies, except for the random index method used by Zheng (2019) to enhance the database samples, other researches have improved the classification accuracy by improving the classification model. However, the random index only adds the existing images to all kinds of samples randomly, which cannot effectively improve the sample richness of the dataset. Therefore, this paper proposes a data enhancement method based on an affine transform and random index to expand the solar radio spectral dataset, improve the sample richness, and solve the problems of balanced sample distribution and sample data in the dataset.

First, the original data are divided into left and right circular polarization parts according to single and double channels. The size of a single image is scaled to 120 × 2,520. At the same time, the data of all kinds of samples are doubled, and the total number of samples is 8,816. Secondly, due to the imbalance of burst and calibration type samples, affine transformation is used for data enhancement. The specific operation is to first flip the original image horizontally, change the temporal characteristics of the spectrum image, and double the number of samples; the original image is flipped vertically to change its frequency characteristics. After two image flipping operations, the number of burst and calibration type samples reaches 3,474 and 2,964 respectively. Finally, to further compensate for the gap in the number of samples with the non-burst type, the random index method is used to increase the number of various samples to 6,670. At this time, the dataset contains 20,010 dynamic spectra, which can basically meet the data requirements of model training. Table 3 shows the sample distribution of the solar radio spectrum dataset after data enhancement.

Evaluation criterion

We use true positive rate (TPR) and false positive rate (FPR) to evaluate the experimental effect, which is calculated as follows:

(5) $$TPR={\displaystyle \frac{TP}{TP+FN}\times 100\mathrm{\%},}$$

(6) $$FPR={\displaystyle \frac{FP}{FP+TN}\times 100\mathrm{\%}.}$$where TP represents a correctly classified positive sample, FN represents a wrongly classified positive sample; TN represents a correctly classified negative sample, and FP represents a wrongly classified negative sample. TPR can be expressed as the proportion of positive samples that are correctly predicted. Its numerator is the number of positive samples that are correctly classified and the denominator is the number of all positive samples. FPR represents the proportion of negative instances that are predicted; its numerator is the number of misclassified negative samples, and its denominator is the number of all negative samples. Generally speaking, if the TPR value of a certain type of sample is larger, it means that the probability of successful identification of this type is greater. If the FPR value is larger, it means that the probability of other class samples being incorrectly identified as this type is greater.

Performance comparisons

Before inputting all the data into the network, we preprocessed the spectrum images in the dataset. First, the image standardization method is used to rescale the gray value to the range of 0–255; second, in order to speed up the network training speed, we limit the size of the image to 120 × 120, Fig. 4 shows the spectrum image changes before and after downsampling, it can be seen from its gray-scale histogram that the down-sampling operation will not significantly change its statistical characteristics; finally, the image is filtered and feature-enhanced using methods such as channel normalization, threshold segmentation, and morphological operations. In addition, we label the non-burst type image in the dataset as [1 0 0], the burst type data as [0 1 0], and the calibration type data as [0 0 1]. At the same time, we divide the dataset into trainingset and testingset according to the ratio of 4:1, including 16,008 training pictures and 4,002 testing pictures.

To reflect the influence of different pre-processing methods and network structures on the recognition accuracy and verify the effectiveness of the method advanced in this paper, two comparative experiments were designed. The adam optimizer was used, the number of iterations was 300, the size of the batch was 128, the learning rate was 0.0001, the time-series length processed by the memory unit was 120, and the number of hidden neurons was maintained at 128. The hardware configuration and processing speed used in the experiment are listed in Table 4. Figure 5 shows the change of model loss and validity accuracy during model training.

Table 5 shows the effect of different convolutional layers on the recognition accuracy of CGRU and CBiGRU. It can be seen that the deepening of convolution layers has a very limited effect on the mining of a spectrum image features after pre-processing. This is because too many convolution layers will cause the network to over fit and ignore the small features of the image, resulting in the reduction of the recognition accuracy of the two models. Although BiGRU adopts a two-way network structure, it is different from natural language processing tasks in that spectrum images do not need to extract contextual features. For time-series images, only historical information must be considered, so the overall recognition accuracy is almost the same as that of the GRU.

Table 6 shows the results of comparison of the experimental results of all the methods from the literature. The following can be found from the table.

1. The calibration type spectrum image has relatively simple characteristics, so basically all models have achieved relatively high recognition accuracy. The TPR value obtained in Zheng (2019) is significantly lower than that of other models. This is because the image pre-processing method is too simple and causes the model to fail to distinguish the image background and features.

2. The recognition accuracy of the burst and non-burst types in Zheng (2019) is significantly higher than that of other models. The reason is that it expands the original sample to solve the problem of sample imbalance, but the classification model it uses has not changed much.

Compared with the classification results in the existing literature, the recognition accuracy of the proposed method is improved by 0.76%, and the optimal result is 98.73%. The TPR value of non-burst type is close to 96.8%, which is slightly lower than the best result. By comparing the experimental methods and results, the following conclusions can be drawn.

1. By reasonably expanding the sample dataset and increasing the effective number of various samples, the balance of neural network training and testing can be improved, and the classification accuracy can be improved.

2. Because the background noise of the original dynamic spectra is relatively strong, it is difficult for the neural network to directly extract the image features. The feature enhancement of the sample before input to the neural network is helpful to improve the final recognition effect.

3. The neural network structure of CGRU can extract the image features. Spatial features can capture time-series features and are more sensitive to solar radio bursts.

4. In the process of feature enhancement in this paper, although the feature of non-burst dynamic spectra is enhanced after closed operation, it also samplifies the background noise in the original spectrum image, which is the main reason that causes non-burst type to be recognized as burst type.