Splitting ore from X-ray image based on improved robust concave-point algorithm

Industrial background

The typical structure of ore sorting device based on X-ray is shown in Fig. 1. The sorting process is as follows: first, raw ore is crushed into small stones of uniform size, which are sent to the pseudo-dual-energy X-ray identification equipment through the belt conveyor. Then, the industrial computer needs to determine the grade of ore from the high and low energy images obtained from the X-ray device and pass the coordinate of ore to the valve controller, which uses the coordinates to adjust the direction of the air flow to blow the ore into the correct separator box. In this process, the image segmentation algorithm must be able to get the correct position of the ore from the X-ray image, otherwise the subsequent valve controller will not work correctly.

The pseudo-dual-energy X-ray processing subsystem used in the system is shown in Fig. 2. It contains an X-ray source and a pseudo-dual-energy X-ray detector. The X-ray source emits X-ray beam with a continuous spectrum, and the dual-energy detector is a two-layer structure, the upper layer collects low-energy signals, the middle layer uses a copper sheet to filter out the low-energy part of the ray, and the lower layer collects high-energy signals. The pseudo-dual-energy X-ray system is widely used in industrial inspection due to its simple structure, high precision and cost performance.

At present, the mainstream sorting method based on pseudo-dual-energy X-ray relies on the fact that different grades of ore have different physical characteristics and thus have different absorption ability for X-ray, so that the gray level of the picture obtained from the X-ray detector is different (one example is shown in Fig. 3). When the ore is separated from the original image, through these gray features, combined with the classification algorithm, different grades of ore and gangue can be separated.

The difficulty of ore image segmentation based on x-ray

Image cutting

As shown in Fig. 4, the output image of the X-ray processing subsystem consists of two parts: high-energy part and low-energy part, so it needs to be cut apart through the slicing operation of the matrix. The image obtained after cutting is shown in Fig. 5.

Image binarization

In order to find the ore contour using Suzuki’s method (Suzuki & Abe, 1985), the image obtained in the Image cutting section needs to be binarized. Fixed threshold segmentation, Otsu threshold segmentation, and adaptive threshold segmentation are the most commonly used methods. It is found in practice that for the samples obtained from actual production process, fixed threshold and Otsu threshold segmentation, as a global threshold segmentation method, cannot effectively binarize ore image under the condition of dark background, much noise and interference caused by stone powder, while as a method using local threshold, adaptive threshold segmentation can better adapt to complex situations in different scenes. Therefore, this article adopts adaptive threshold segmentation algorithm to segment non-adhesive images, and carries out preliminary segmentation of adhesive images.

By selecting the low energy image or high energy image in Fig. 5, the binary image can be obtained after being processed with the adaptive threshold segmentation algorithm, and the result is shown in Fig. 6.

Noise filtering

In the actual production process, the binary image of ore has a lot of noise. If the noise is not filtered, the program will treat the noise as tiny ores, which not only increases the computing load of the computer, but also interferes with the process of recognizing normal ores.

We use morphological operation (Comer & Delp, 1999) to remove noise from the image. Firstly, the noise inside the stones were removed by dilation operation, as shown in Fig. 7. Similarly, in order to remove noise outside the ore, erosion operation is used, and the effect is shown in Fig. 8.

Contour extraction

In this article, the method proposed by Suzuki (Ren, Zhang & Zhang, 2019, Suzuki & Abe, 1985) is used to extract contour from binary image, and the extracted contour is shown in Fig. 9.

Concave point detection

It can be observed that the concave point is a class of points with the maximum local curvature on the contour formed by two or more circular objects stacked. For a single smooth elliptic object, the contour curve will not have a large curvature mutation, nor will it form a concave region caused by overlapping of different objects. On the contrary, there must be points whose curvature change suddenly on the contour of the cohesive ores, which is the basis for determining concave points.

As shown in Fig. 10, two vectors named $\mathit{u}$ and $\mathit{v}$ are formed by three consecutive points on the contour (the length of the $\mathit{u}$ and $\mathit{v}$ are greater than the hyperparameter d), then the angle between the two vectors is calculated to obtain the concavity corresponding to the point.

For points on the contour $[p_1,p_2,...,p_k,...,p_n]$, in order to find the concavity corresponding to point $p_k$ on the contour, by using Eq. (1), $m=i$ and $m=j$ can be solved separately (in Eq. (1), $i>k,j<k$, and in this article, d is set to 6).

(1) $$\begin{array}{rl}& \arg \underset{m}{min}|k-m|\\ & s.t.|p_k-p_m|>d\end{array}$$

By substituting the values of $i$ and $j$ into Eqs. (2) and (3), we can derive the values of the vectors $\mathit{u}$ and $\mathit{v}$, and their corresponding coordinates $u_x$, $u_y$, $v_x$ and $v_y$.

(2) $$\mathit{u}=(u_x,u_y)=p_i-p_k$$

(3) $$\mathit{v}=(v_x,v_y)=p_j-p_k$$

To calculate the angle between the vectors $\mathit{u}$ and $\mathit{v}$, the function $\theta (x,y)$ defined in Eq. (4) is used to calculate the angle between the x-axis and the vector $\mathit{u}$, and the same is true for $\mathit{v}$, so that we can obtain the values of $\theta (u_x,u_y)$ and $\theta (v_x,v_y)$. Let ${\theta }_1$ be the difference between $\theta (u_x,u_y)$ and $\theta (v_x,v_y)$. Note that $\theta (x,y)$ in Eq. (4) is 0 in the positive direction of the x-axis, and gradually increases in the counterclockwise direction, with a max value of $2\pi$.

(4) $$\theta (x,y)=\{\begin{array}{cc}arctan(\frac{y}{x})\hfill & x>0,y\ge 0\hfill \\ arctan(\frac{y}{x})+\pi \hfill & x<0\hfill \\ arctan(\frac{y}{x})+2\pi \hfill & y<0,x>0\hfill \\ \frac{\pi }{2}\hfill & x=0,y>0\hfill \\ \frac{3\pi }{2}\hfill & x=0,y<0\hfill \end{array}$$

(5) $${\theta }_1=\theta (u_x,u_y)-\theta (v_x,v_y)$$

As ${\theta }_1\in [-2\pi ,2\pi ]$, the result obtained in the previous step need to be processed by using Eq. (6) to obtain the angle $\alpha$.

(6) $$\alpha =\{\begin{array}{cc}{\theta }_1+2\pi \hfill & {\theta }_1\in (-2\pi ,-\pi )\hfill \\ {\theta }_1\hfill & {\theta }_1\in (-\pi ,\pi )\hfill \\ {\theta }_1-2\pi \hfill & {\theta }_1\in (\pi ,2\pi )\hfill \end{array}$$

We can obviously judge the convexity and angle of the local contour from the value of $\alpha$, that is, if $\alpha \in [0,\pi ]$, it is concave, otherwise it is convex. In practical use, in order to reduce interference, relatively flat local contours will be excluded, that is, point on the contour can be regarded as a concave point only if $\alpha \in [0,{\theta }_t]$, where ${\theta }_t<\pi$. After repeated experiments, it is found that selecting ${\theta }_t=\frac{3\pi }{4}$ as threshold in this article can achieve better detection effect. By using the concave detection algorithm described above, the concave points can be detected in the extracted contour (b) in Fig. 9, and these concave points are annotated in red in Fig. 11 (the contour drawn in the Contour extraction section is hidden to highlight the concave points).

Concave point matching

The physical characteristics of the ore often lead to the existence of a large number of concave points on its edge in the absence of adhesion with other ores. For example, because some antimonite exists in the form of crystal, many concave points will be detected when the concave detection algorithm is performed. For this kind of ore, it will lead to over-segmentation if the concave line is used as the dividing line directly. This problem can be solved perfectly by using three-line method proposed in this article. The schematic diagram of this method is shown in Fig. 12.

As can be seen from the Fig. 12, due to the characteristics of the ore itself, concave points can be detected even from the outline of a single ore. Obviously, in such case, the connecting line of the concave points cannot be directly used as the dividing line (marked in green). In order to eliminate the false segmentation caused by such points, auxiliary lines are drawn in the figure (marked in black). The connecting line of its concave points can be used as a dividing line if the auxiliary lines are all inside the contour. The detailed steps of the method are as follows:

Judging the position relation between auxiliary line and contour

In order to judge the position relationship between the auxiliary line and the contour, it is necessary to judge the position relationship between each point of the auxiliary line and the contour. In this article, the PNPoly algorithm (Haines, 1994; Zhang & Zhou, 2022) proposed by W. Randolph Franklin is used to solve the position relationship between the point and the contour (the point to be measured can be on the contour, inside the contour or outside the contour). The idea of the algorithm is shown in Fig. 13.

As can be seen from the figure, the position relationship between the point and the contour can be judged by drawing the horizontal line and finding the number of times that the line crosses the contour.

Complete segmentation process

The overall segmentation process is shown in the Algorithm 2, one of the segmentation examples is shown in Fig. 14.

Analysis of results

In order to better demonstrate the superiority of the algorithm in this article, the algorithm proposed in this article is compared with simple concave point matching algorithm and the watershed algorithm based on distance transform using exactly the same dataset. Several typical images are selected from the results of experiment to illustrate the performance of each algorithm. As shown in Fig. 16, if the overlapping area of two ores is large, they cannot be separated by the watershed algorithm (as shown in the first row). Even if the watershed algorithm can be used to divide the cohesive ore, the dividing line in the segmentation result is not very accurate (as shown in the second row). Due to the physical characteristics of the ore, even a single ore may have a large number of concave points on its contour, in this case, the use of simple concave matching will inevitably lead to over-segmentation (as shown in the third row). In the above cases, the algorithm proposed in this article can work well.

In this article, $P_u$ (under-segmentation rate), $P_o$ (over-segmentation rate) and $P_a$ (accuracy rate) are used as performance indicator to reflect the performance of algorithms when applied in practical industrial applications and they are defined as:

(14) $$P_u=\frac{N_u}{M}\times 100\mathrm{\%}$$

(15) $$P_o=\frac{N_o}{M}\times 100\mathrm{\%}$$

(16) $$P_a=\frac{N_a}{M}\times 100\mathrm{\%}$$

In this formula, $N_u$ is the number of under-segmentation ore, $N_o$ is the number of over-segmentation ore, $N_a$ is the number of ores correctly divided and M is the total number of ores tested ( $M=134$).

As can be seen from the Table 2, compared with other algorithms, the proposed algorithm can significantly reduce the probability of under-segmentation and over-segmentation and improve the accuracy of segmentation.