Detection of linear features including bone and skin areas in ultrasound images of joints

Features for training

As the bone and skin detector uses supervised learning, the training data for algorithms must explicitly indicate for each set of features extracted from the image, whether it is related to bone or skin or to neither of them. This information must be provided a priori as a set of annotations attached to each processed image of a training set. An example of image annotations is shown in Fig. 2B.

The pixels for training features generation selected from bone and skin annotations are called samples. There are three types of samples: (1) positive samples that represent bone, (2) positive samples that represent skin and (3) negative samples that represent neither bone nor skin.

The pixels used for generation of positive bone samples are the same as bone annotation points. Similarly, the pixels used for positive skin samples generation are the same as skin annotation points. By contrast, the pixels used for negative samples generation are selected from areas of an image that are neither bone nor skin regions. The bone and skin regions are defined as areas within a given radius from bone and skin annotation points. Thus, each pixel selected for the negative sample must lie in distance bigger than given radius from any pixel belonging to bone or skin annotation. Figure 3 presents the example of pixels selection that is further used for samples generation.

To obtain the proper characterization of the selected pixels, their neighborhood should also be considered. This is done by taking as a pixel representation the subimage [2⋅w + 1, 1] centered at the pixel, where 2⋅w + 1 is a vertical window around the pixel. The representation vector of a pixel p_i is defined as a vector of pixels K_i = (p_k,l−w, p_k,l−w+1, …, p_k,l+w), where k and l are the image coordinates of the pixel p_i.

Once the pixels for samples are selected and their vertical neighborhood is defined the final samples generation process may start. First of all, a series of new images is obtained by use of different filters, where each new image is an output of particular filter applied to the original image. It includes Gaussian smoothing, first and second derivatives of Gaussian, Laplacian of Gaussian as well as thresholding filters. The Gaussian smoothing, as proposed in Deriche (1990), is done by recursive filtering that approximates an image convolution with the Gaussian kernels. The convolution with first and second derivatives of Gaussian is applied in vertical direction and the Laplacian of a Gaussian filter is obtained by applying the second derivative of a Gaussian in both vertical and horizontal directions. Additionally, all outputs of these filters are processed by the binary thresholding. This operation with the use of some given thresholds produces binary output images by mapping the original pixel values to black or white ones. All filtering operations use the efficient implementation of recursive filters (Deriche, 1993) taken from ITK library. The resulting images obtained by selected filters are presented in Fig. 4.

The set of images obtained by applying n filters to the original image I forms a vector F = (I₁, I₂, …, I_n), which is used for final samples generation. The samples are generated by taking the values of all selected pixels with their vertical neighborhood K_i from each image I_j which is obtained from the jth filter. This generates for each K_i the matrix made from concatenated vectors of values extracted from filtering outputs I_j (where j ∈ [1, n]) for each pixel in the window, which can be denoted as: (1)${\displaystyle V_i=\left(\begin{array}{cccc}\hfill v_{i11}\hfill & \hfill v_{i12}\hfill & \hfill ..\hfill & \hfill v_{i1n}\hfill \\ \hfill v_{i21}\hfill & \hfill v_{i22}\hfill & \hfill ..\hfill & \hfill v_{i2n}\hfill \\ \hfill :\hfill & \hfill :\hfill & \hfill :\hfill & \hfill :\hfill \\ \hfill v_{ik1}\hfill & \hfill v_{ik2}\hfill & \hfill ..\hfill & \hfill v_{ikn}\hfill \end{array}\right).}$

The example of a few different neighborhoods K_i indicating the pixels to be taken from the Laplacian output as part of training samples is presented in Fig. 5.

The final training samples for given image I consist of three matrices. Each such matrix is a concatenation of matrices from Eq. (1) for all samples of given type t:

${\displaystyle X_t=\left(\begin{array}{c}\hfill V_1,V_2,..,V_m\hfill \end{array}\right)}$ (2)${\displaystyle =\left(\begin{array}{cccc}\hfill \left(\begin{array}{cccc}\hfill v_{111}\hfill & \hfill v_{112}\hfill & \hfill ..\hfill & \hfill v_{11n}\hfill \\ \hfill v_{121}\hfill & \hfill v_{122}\hfill & \hfill ..\hfill & \hfill v_{12n}\hfill \\ \hfill :\hfill & \hfill :\hfill & \hfill :\hfill & \hfill :\hfill \\ \hfill v_{1k1}\hfill & \hfill v_{1k2}\hfill & \hfill ..\hfill & \hfill v_{1kn}\hfill \end{array}\right)\hfill & \hfill \left(\begin{array}{cccc}\hfill v_{211}\hfill & \hfill v_{212}\hfill & \hfill ..\hfill & \hfill v_{21n}\hfill \\ \hfill v_{221}\hfill & \hfill v_{222}\hfill & \hfill ..\hfill & \hfill v_{22n}\hfill \\ \hfill :\hfill & \hfill :\hfill & \hfill :\hfill & \hfill :\hfill \\ \hfill v_{2k1}\hfill & \hfill v_{2k2}\hfill & \hfill ..\hfill & \hfill v_{2kn}\hfill \end{array}\right)\hfill & \hfill ..\hfill & \hfill \left(\begin{array}{cccc}\hfill v_{m11}\hfill & \hfill v_{m12}\hfill & \hfill ..\hfill & \hfill v_{m1n}\hfill \\ \hfill v_{m21}\hfill & \hfill v_{m22}\hfill & \hfill ..\hfill & \hfill v_{m2n}\hfill \\ \hfill :\hfill & \hfill :\hfill & \hfill :\hfill & \hfill :\hfill \\ \hfill v_{mk1}\hfill & \hfill v_{mk2}\hfill & \hfill ..\hfill & \hfill v_{mkn}\hfill \end{array}\right)\hfill \end{array}\right)}$

where m is a number of samples of given type, t ∈ {Skin, Bone, None}, k is a window size and n is a number of filters applied to the image I. Each column of matrix Eq. (2) represents the output values of particular filter for one neighborhood K_i of pixel p_i and each row represents the output values from all filters for one index in the neighborhood window K_i across all pixels p_i. Such form of training data is directly passed as feature vectors to learning algorithms in the second phase of the processing described in following sections.

Features for classification

Once the learning algorithms are trained on the feature vectors described in the previous chapter, they require a similar form of data for classification instead of raw pixel values from the original test image. The process of samples generation for classification is the same as for training samples generation. The only difference is that the samples are generated for all pixels p_i of the test image instead of those representing only positive and negative samples. It means that if the test image resolution is M × N, the number of pixels selected for samples generation is M⋅N. The final data for classification are also concatenated in the same way as the training data Eq. (2) using all vertical neighborhoods K_i and output values from all filters F, which gives the final size of the generated matrix as M⋅N⋅n⋅k.