Hydrographic data inspection and disaster monitoring using shipborne radar small range images with electronic navigation chart

Materials

32 shipborne radar images were acquired during the cruises of the teaching-training ship Yukun (Fig. 1) of Dalian Maritime University. Among them, 21 images contained ships, 7 images contained shorelines, and 6 images contained oil spills. The three images shown as Fig. 2 are used to introduce our method. The radar parameters are listed in Table 1. The data acquisition period is 2 s. The monitoring radius in the images is 0.75 nautical miles (NM). The image size was 1,024 × 1,024 pixels and the gray level was 256. The chart of the Port of Dalian is used here, as shown in Fig. 3. The image processing and analysis platform is MATLAB 2014a, and the system development platforms are Visual Studio 2010 and ArcGIS 10.2.

Data preprocessing

Many oil films exist around the ship, as shown in Fig. 2A. Ships and coastlines information are presented in Figs. 2B and 2C. The images needed to be denoised first, especially the co-frequency interference noises, which seriously affect the uniform gray distribution. The data preprocessing scheme is shown in Fig. 4. First, the image was converted to a Cartesian coordinate system. After that, the co-frequency interference noises were segmented by the Laplace operator and Otsu method. They were then suppressed via mean filtering.

The original shipborne radar image was collected in the Cartesian coordinate system. In Formula Eq. (1), the image is transformed into a polar coordinate system for understandable, as shown in Fig. 5. (1)${\displaystyle \left\{\begin{array}{c}\rho =\sqrt{x^2+y^2}\hfill \\ \theta =arctan\left(\frac{y}{x}\right)\hfill \end{array}\right.,}$

where x and y are the abscissa and ordinate of the Cartesian coordinate system, respectively, and ρ, θ are the distance and orientation of Polar coordinate system, respectively.

Many meaningful image features are present in the Cartesian coordinate system. For example, the co-frequency interferences appear as radial noises in the vertical direction (Fig. 6), and they are processed without considering the angle factor. Therefore, we used Formula Eq. (2) to transform the experimental data from the polar coordinate system to the Cartesian coordinate system. (2)${\displaystyle \left\{\begin{array}{c}x=\rho cos\theta \hfill \\ y=\rho sin\theta \hfill \end{array}\right.,}$

The Laplace operator is as follows: (3)${\displaystyle {\nabla }^{2}f=4f\left(x,y\right)-\left[f\left(x+1,j\right)+f\left(x-1,j\right)+f\left(x,\mathrm{y}+1\right)+f\left(x,y-1\right)\right],}$

where x and y are the abscissa and ordinate of the image, respectively. The mean filtering is calculated (n = 3 in our experiment) as (4)${\displaystyle f\left(n,y\right)=\frac{\sum _{i=0}^{n}X\left(n-1,y\right)+X\left(n+1,y\right)}{2n},}$

Figure 2C is used to introduce the preprocessing flow, as shown in Fig. 7. First, the original radar image was transformed from the polar coordinate system to the Cartesian coordinate system (Fig. 7A). Then, the Laplace operator was used to enhance the co-frequency interference and weaken the other information (Fig. 7B). Next, the Otsu threshold was used to segment the co-frequency interference (Fig. 7C). Finally, a mean filter was used to suppress the co-frequency interferences, as shown in Fig. 7D.

Data analysis process

The data analysis process is shown in Fig. 8. First, it is necessary to determine whether to extract the light or dark target firstly. For a bright target detection, the dual-threshold method is used for segmentation after setting the monitoring area. Otherwise, the local threshold method is used to segment the dark target after setting the effective wave area. Then, the identified targets are transformed to the polar coordinate system. After that, the contour of the target is extracted, and the boundary points of the contour are projected onto the geographic coordinate system for data fusion.

Bright target segmentation

In the original shipborne radar image, the brightness of the wave echo area at a distance is high. Therefore, the monitoring area should be set before identifying the ship or coastline. We removed the image information within 0.3 NM (mainly the highlighted wave echo pixels) and retained the image information of the bright target monitoring area, as shown in Fig. 9.

The following dual-threshold method is selected to extract the target, as shown in Fig. 10. (5)${\displaystyle D\left(x\right)=\left\{\begin{array}{c}d\left(x\right)>T\left(gray\right)\hfill \\ A\left(region\right)>T\left(Area\right)\hfill \end{array}\right.,}$

where D (x) is the continuous pixel of the target, d (x) is the gray value of the image pixels, T (gray) is the gray threshold, A(region) is the number of pixels of the continuously highlighted image regions, and T(Area) is the number threshold of A(x). For a radar image of 1,024 pixels ×1,024 pixels with a monitoring range of 0.75 NM, the actual area of 1 pixel is 7.34 m². It is suggested to set the continuous highlight area of less than 20 pixels as speckles and eliminate them here. If the radar range increases and the image size does not change, then T(Area) should be smaller.

Dark target segmentation

Because of the different reflection characteristics of the monitoring targets, the identification of oil spills is more complicated than that of ships and coastlines. The oil film is dark in the sea clutter area, so the effective monitoring of the wave should be determined first.

In the image after preprocessing, the information is deleted at a long distance (0.5 NM away in our experiment). Then, the image gray distribution matrix (Fig. 11B) is generated by convolution of a 20 ×80 window matrix. Next, the threshold (“61.02” in Fig. 11C) is selected in the color scale image of Fig. 11B to determine the effective wave range by visual interpretation and manual selection , as shown in Fig. 11D.

The method of final segmentation is also more complicated than that of bright targets. Oil films can suppress the echo of sea waves and display them as relatively dark areas on waves in the images (Tong et al., 2019; Fingas & Brown, 2018). Therefore, local thresholds and active counter models are often used for segmentation (Liu et al., 2019). Figure 11D is segmented as Fig. 12 by the local threshold method (Xu et al., 2019) as: (6)${\displaystyle T=m\times \left[1+k\left(\frac{v}{R^2}-1\right)\right],}$

where m is the local mean, k is a user-defined parameter that takes negative values, v is the local standard deviation, and R is the dynamic range of the standard deviation. In our experiments, k = 0.25 and R = 128.

Data fusion

The image acquisition method of shipborne radars is head-up, and the ENC in our experiment is north up. To match the radar target with the ENC, the identification result needs to be rotated. An image matrix coordinate system was thus established to reduce the difficulty of image rotation, as shown in Fig. 13.

The image rotation formula is: (7)${\displaystyle \left\{\begin{array}{c}{x}^{\prime }=xcos\left(a\right)-ysin\left(a\right)\hfill \\ y^{\prime }=xsin\left(a\right)+ycos\left(a\right)\hfill \end{array}\right.}$

where x′ and y′ are the abscissa and ordinate after rotation, respectively; x and y are the abscissa and ordinate before rotation, respectively; and a is the angle of rotation.

The identification target is rotated with the angle according to the heading of the ship, and the counter is extracted as shown in Fig. 14.

The longitude and latitude coordinates of the ship can be obtained through GPS. In ArcGIS, the plane coordinate (X_os, Y_os) of the ship in meters can be obtained by projecting it from the WGS_1984 geographic coordinate system to the Beijing_1954 projection coordinate system. Based on the image matrix coordinates (X_Pic, Y_Pic), radar detection distance D, image radius R, and plane coordinates of the ship (X_os, Y_os), the plane coordinates (X_BJ, Y_BJ) of the target contour boundary point in the geodetic coordinate system of Beijing_1954 can be obtained as (8)${\displaystyle \left\{\begin{array}{c}{X}_{BJ}=X_{os}+\left(X_{Pic}∕R\right)D\hfill \\ Y_{BJ}=Y_{os}+\left(Y_{Pic}∕R\right)D\hfill \end{array}\right.}$

Then through ArcGIS, target contour boundary points were converted back orderly to the geographic coordinates of the WGS_1984 coordinate system, as shown in Table 2.

After the geographical coordinates of the target boundary points were obtained, the target polygons were generated into an ENC based on ArcGIS Objects, as shown in Fig. 15. The detailed steps include the following: (a) According to the interfaces of ISpatialReferenceFactory and ISpatialReference, the properties of spatial reference (SpatialReference) and spatial projection (Project) of the target contour boundary point (IPoint) are set. (b) Using the AddPoint method, the boundary points are added to the point set (IPointCollection) to generate the target polygon. (c) The geometry attribute of the target polygon is assigned to the interface of the IElement. (d) Through the interface of IFillShapeElement, method of AddElement, class of GraphicsContainer and class of ActiveView, the target polygon is was added to the development control (axMapControl) of the ENC.

The coastline targets were fused into the ENC, as shown in Fig. 16A, which includes local information before and after fusion. The oil spill targets and ship targets were fused into the ENC, as shown Figs. 16B and 16C, respectively.

Role of data fusion

The target data fusion between radars and the ENCs cannot only verify the hydrographic data but also assist the emergency command of marine disasters and accidents. Bright targets can be used to test the accuracy and timeliness of hydrographic data. For example, Fig. 16A can determine whether the island coastline data of the ENC are correct or whether the location of the lighthouse is correct. The coastline changes due to the influence of tides and storm surges. The island profile of low tide is shown in the ENC (green region in Fig. 16A). The island belongs to a strong reflector, so the radar can only detect its illuminated surface but not the blocked area. The radar detected that the coastline of the irradiating surface was beyond the island profile of the ENC. Thus, the ENC data must be updated in time.

Dark targets can be used for marine disaster monitoring, such as the monitoring of oil spills, as shown in Fig. 16B. Remote sensors have become the main means of marine disaster monitoring technology. With respect to radar applications, airborne and spaceborne radars are maturely used for offshore disaster monitoring (Sankaran, 2019; Hammoud et al., 2019; Gallego et al., 2018). However, spaceborne remote sensing cannot conduct monitoring in real time. Airborne sensors cannot be used in severe weather conditions. The technology of shipborne radar marine disaster monitoring can overcome adverse weather conditions and remotely monitor real-time situations with high resolution. The data fusion of dark targets can provide technical support for the early warning and emergency disposal of marine disasters and accidents.

The ship target fusion of shipborne radars and ENC can provide a more effective guarantee for navigation safety. Our experimental data range is 0.75NM, as shown in Fig. 16C, which only provides a ship target fusion method for radars and the ENC. Real applications need to improve the detection range of radar images.

Different segmentation methods for bright and dark targets

Ship and coastline targets have strong reflectivity, as highlighted in the images. Fixed thresholds can be used to segment them. However, oil film extraction is more difficult. A single threshold cannot identify discrete oil films in different positions. The local adaptive threshold is a better solution. In addition, because the database of the oil spill analysis is composed of waves, the monitoring range is close to the own ship, as shown in Fig. 11D. But, ships and coastlines are monitored for the inspection of hydrographic data. The monitoring area is located far away from the ship (Fig. 9).

Comparison with other bright target segmentation methods for shipborne radar images

The method of Wang et al. (2020) (hereafter referred as Method 2) and our method for extracting bright targets are compared here. Figure 2B and 2C are segmented using Method 2, as shown in Fig. 17.

Compared with Fig. 10A, only the preliminary extraction effect presented in Fig. 17B is similar. Too many false positive targets are present in Figs. 17C and 17D, implying that Method 2 is not suitable for large bright-target detection, such as islands and coastlines. Many false positive targets also exist, as shown Fig. 17A. For the complexity of the original image of the shipborne radar, Method 2 needs to set a smaller monitoring area to obtain good segmentation results for ships, as shown in Fig. 17B. Our method can identify ships and islands correctly, and set a large monitoring area, which has strong applicability in bright-target identification.

Comparison with other dark target segmentation methods for shipborne radar images

The methods of Zhu et al. (2015) (hereafter referred as Method 3), Xu et al. (2018) (hereafter referred as Method 4) and our method for extracting oil spills are compared with Fig. 2A in Fig. 18.

In Method 3, a global threshold method is proposed for oil film identification after gray adjustment, as shown in Fig. 18B. From the perspective of an expert interpretation, the recognition effect presented in Fig. 18B is not as good as that of our method. As the global threshold increases, Method 3 recognizes the false oil films in the area without wave information. In Method 4, a dual-threshold method is proposed to detect oil spills after choosing the wave range, as shown in Fig. 18C. Although a small wave range is selected, as shown in Fig. 18C, the segmentation effect is not good. As the wave range increases, the segmentation result becomes worse. Therefore, the major drawback of Method 4 is that it cannot select a further effective wave monitoring range. Therefore, the local adaptive threshold method is more suitable for the segmentation of large-range oil films and yields better results.

Guarantee of navigation safety using the data fusion of an ENC and shipborne radar

The production of an ENC is time-consuming and has a slow update speed. Before sailing, the captain and pilot will create a general route using the information of the ENC. However, in the actual driving process, owing to the weak current situation of the ENC, the route will be made local adjustments in accurate information acquired by the radar.

The ENC can provide information on sunken ships, reefs, and shoals, among others. Shipborne radars can provide collision avoidance information for all other ships around the ship. After the data fusion of the ENC and shipborne radars, the dynamic information of a wider-range radar monitoring and the static information provided by the ENC can form a more efficient digital platform, which can improve the monitoring ability of pilot, and provide better guarantee for navigation safety.

Lack of AIS data

At present, ENCs and shipborne radars have been fused with AIS system data. In radars and ENCs, the location and attribute information of a ship with AIS can be obtained (Jung, Kim & Song, 2007). However, the installation of AIS equipment is not required for small fishing boats or maintenance boats. Radars have an obvious marking effect on strong reflectors, which can highlight their reflecting surfaces on the screen. Therefore, the role of radar navigation and collision avoidance is essential. In addition, AIS targets use specific symbols in radars and ENCs, and there is no real shape. If the radar installation position of the ship is much higher than that of the target, then the target contour can be identified completely and integrated into the ENC. This characteristic makes up for the deficiency that AIS data cannot obtain the contour of the targets.