A quantization assisted U-Net study with ICA and deep features fusion for breast cancer identification using ultrasonic data

Data augmentation

The ultrasonic data are used with their labels to process the data. In the dataset, images are not equally sized. Therefore, BUSI data are approximately 512 × 512 × 3 and are resized using the Nearest-Neighbor method. The resizing needs geometric operations to remap the given digital lattice. For example, if the given image is represented as I, then an operation is performed on the image using Eq. (1), where Eq. (2) is representing the f(n):

(1) $$I\left(n\right)=f\left(n^{\mathrm{\prime }}\right)=f(a(n))$$where f(n) is further representing a(n) operation, the f(n) working according to Eq. (2).

(2) $$f\left(n\right)=\left[INT\left(a_1\left(i,j\right)+0.5\right)\right],\left[INT\left(a_2\left(i,j\right)+0.5\right)\right]$$

In Eq. (2), the lattice is handled with two new values as a single value of a₁, which does not entirely represent a newly defined image due to spatial operations, as the new lattice contains a decimal place value. Therefore, two values of a₁ and a₂ are combined to define the new size of the image. The input images are then resized using the explained method or operations. The new input images are shown in Fig. 2.

After resizing, all the given ground truth labels and input images become equal, as shown in Fig. 2, but preprocessing is still needed to feed the input data to U-Net. To remove speckle noise, the input image regions need to be enhanced. Data augmentations are used to handle the overfitting problem in the deep learning models, where more input instances of data samples are created. Therefore, the proposed method uses fast and very flexible image augmentations on the dataset. According to Buslaev et al. (2020), the images are transformed w.r.t. given dimensions using copping and padding, if needed. Almost all of the data preprocessing methods are available; the user chooses operations according to the acquired data. The resized input data are clipped with a shift scale rotation with horizontal flipping. Similarly, we also added contrast stretching, brightness adjustments, sharpening, and contrast-limited adaptive histogram equalization (CLAHE) to used datasets. The overall data are transformed, enhanced, and ready to solve the overfitting problem of deep learning models. The augmented data have a much better resolution and clearance, as shown in Fig. 3.

As shown in Fig. 3, the data and its corresponding labels are also shifted to the same angle and rotation so that images remain correspondent to each pixel label. Thus, the augmented data do not lose any spatial information of the original input images and masks. Furthermore, it also enhances the input samples and removes the speckle noise from the images, if any. However, these preprocessed augmented input data are fed to U-Net for the semantic segmentation of the BUS imagess.

U-Net-based segmentation

U-Net has an encoder-decoder approach to semantically segment the images (Yakubovskiy, 2019). The encoder side of the model down-samples to the given images with their labels, while the up-sampling approach re-codes the images to decode the image back to reach the final convolutional layer. The proposed method uses tensor-flow as a helper library to obtain a model of U-net where the backbone is ‘inceptionresnetv2’, which is used to train using the given augmented data. inceptionresnetv2 is selected for its high accuracy rate compared to other backbones. The overall architect of U-Net is shown in Fig. 4 with some convolutional block visualization. In the inceptionresnetv2 backbone, the model has multiple blocks, including three blocks of convolution, batch normalization, and activation, where pooling is performed using the max and average of various blocks.

The U-Net-based segmentation was tested on two datasets, and the results are shown in Fig. 5. The OASBUD dataset images are noisy, as raw image inputs are not the same as BUSI images. Therefore, using enhancement techniques, the dataset was first processed using a Gaussian smoothing filter and histogram equalization and then passed through a data-augmentation process explained in the first sub-section of the Methodology section.

(3) $$S(i,j)=\sum _{i=1}^n\sum _{j=1}^n\mathrm{e}\mathrm{x}{\mathrm{p}}^{-{\displaystyle \frac{i^2+j^2}{2{\sigma }^2}}}$$

(4) $$S_1={\displaystyle \frac{1}{2\pi {\sigma }^2}\ast S(i,j)}$$

In Eqs. (3) and (4), Gaussian blur is explained where I and j are the rows and columns of any given digital lattice, where σ² represents the standard deviation, which is taken as 0.5 in experimentation. This exponentially powered result is later on multiplied with a denominator of the product of 2π and σ². It returns a final smoothed image in the case of OASBUD images that contain noise. Later on, histogram equalization is applied to the image, which can be calculated in Eqs. (5)–(7).

(5) $$H=T(h)$$

Eq. (5) explains the transformed histogram for a given image, which is our smoothed image in the experiment, so we can make Eq. (6), which shows the equally distributed gray-level intensity bins of the given smoothed input image.

(6) $$S_{1_T}=T(S_1)$$

This transformation occurs with a cumulative histogram subtraction, as shown in Eq. (7).

(7) $$E=|c_i\left(S_{1_T}\right)-c_0(S_1)$$

Eq. (7) explains that the cumulative histogram of the transformed bins is subtracted from the input image grayscale bins, showing the final enhanced image ‘E’, used later for segmentation testing purposes.

As we can observe in Fig. 5, Column-1 contains input images of testing data of the BUSI and OASBUD datasets. The 2nd column contains the input ground-truth masks of both datasets. The 3rd column shows the U-Net predicted lesion masks of input images of Column-1. The last column shows the predicted lesion-based heat-map visualization of the input images. It localizes the lesion part of the input images for better interpretation. These lesions are well-segmented, but some of the images were not extracted as required. Those segmented lesions were re-considered for segmentation using the quantization process.

Quantization based on Otsu’s method

While testing datasets, some of the images were not entirely satisfactory in determining the exact lesion area. First, the lesions obtain a threshold using Otsu’s threshold method and return a 1 × n vector, which can be used to obtain n + 1 discrete values from an image further. These vector-based indexes make quantized images from which the tumor-level-based selection is performed to finalize the input lesion. One of the input images that needs reconsideration for segmentation is the quantization-based selection of the lesion area in Fig. 6.

The final quantized lesions and U-Net-based segmented lesions are then used to extract ICA and denseNet201 deep features to use separately and in serial fusion to classify the breast lesions.

Independent component analysis features

Features Extraction is an important that leads to higher rates or prediciting accuracy, it plays main role in classification. Many recent studies use them in various aspect of field to recognize the given objects such as abstract art painting based features recognition (Li, Raj & Namasudra, 2021). However, Saliency maps are nature inspired, biological plausible features. It calculated as natural statistics in image. Many features are used for recognition or classification tasks but those are not so generalized for visual tasks, also image classification needs visual ability to cope with partial views. Therefore, inspiring from saliency visual statistics (Ramanishka et al., 2017) is used in proposed study. The Independent component analysis (ICA) features are firstly extracted. The sparse filters are created which contain luminance in it, also have effects of chromatic properties. At first, the small patches are extracted with LMS (long, medium, short) color space channel means subtraction. These patches are further used by PCA to reduce its dimensionality, also the high Eigen value containing responses are excluded. The finally normalized feature vector ‘f’ is used by Saliency using natural statistics (SUN) framework to make saliency maps which further taken as responses means and shown in Fig. 7.

The SUN framework uses the bottom-up saliency type to make a map. This saliency is defined as P(f)⁻¹, where the features are represented by f. As the dimensions of the feature vector increase, the ICA becomes more statistically independent. Therefore, the dimension of one-directional distributions can be written as P (F). Both are represented in Eqs. (8) and (9).

(8) $$(f_{}{)}^{-1}$$

(9) $$P(F)={\pi }_{i}P(f_i)$$

In Eq. (9), the f_i representing the features vector is explained above, where P(f_i) represents the generalized Gaussian distributions used as a one-dimensionality conversion method.

Classification

The classification of malignant and benign cases of the used ultrasonic imaging is used with two validation techniques, 5-fold and 10-fold. Different 6-type classical machine learning methods are used and include two variants of support vector machine (SVM), named Cubic-SVM (C-SVM) and Quadratic-SVM (Q-SVM). Moreover, the ensemble bagged-boost (Bag-Boost), RUS-boost (R-Boost), and medium tree (M-Tree) are used. At first, the effect of the saliency-based feature is checked. At the 2nd experiment, the deep, dense features are serially fused to obtain the combined effect of both features to yield more confident results.

Dataset description

Two publically available datasets were employed to evaluate the proposed approach. Dataset-1 uses the BUSI dataset, which contains three types of images—normal, benign, and malignant. The U-Net model is trained on the BUSI training images, which are tested on two datasets. The two testing datasets contain data from the BUSI testing set and OASBUD-enhanced images, respectively. The BUSI dataset was available in images format where OASBUD data was available in raw format. However, the given data is used to create images and their corresponding masks. The multiple morphological operations using its features such as area, circularity, roundness etc. These features are used to get contours of this data. However, the enhancement operation is also performed on OASBUD data using same methods as used in BUSI data, Both dataset descriptions are shown in Table 2.

U-Net experimental setup

The first step of deep-learning-based segmentation is performed using U-Net on two classes, background and tumor. The number of epochs is increased and decreased, which affects the final results. Therefore, 40 epochs are taken as the final optimal number of epochs. However, the learning rate is set to 0.0001. All other training parameters are shown in Table 3.

The loss or error for the evaluation of U-Net is taken as dice loss, and cross-entropy-based loss is taken as total loss. The cross-entropy has been taken previously for the evaluation of semantic networks. However, the dice loss is primarily taken on recent semantic segmentation network evaluations. By considering both as significant losses, the proposed method takes both as summation by naming the new loss as ‘total loss’. The dice coefficient elaborates the area of prediction compared to the entire area that needs to be predicted. In the proposed method, the loss function calculates using dice and focal loss functions. The calculation of both focal and dice loss is shown in Eqs. (11) and (15).

(11) $$DiceLoss=D.L=1-f_1-f_2$$

(12) $$f_1={\int }_{n=1}^N\left({\displaystyle \frac{(L_{FP})(L_{FR})+\epsilon }{L_{FP}+L_{FR}+\epsilon }}\right)$$

(13) $$f_2={\int }_{n=1}^N\left({\displaystyle \frac{L_{BP}\ast L_{BR}+\epsilon }{2-L_{FP}+L_{FR}+\epsilon }}\right)$$

The dice loss or dice coefficient using f₁ and f₂ is calculated, where f₁ uses a product of the predicted L_FP foreground, and the actual L_FR foreground label masks with the division of both predicted and real foreground masks unite. Additionally, the ε weight is added to compensate if the product returns a zero.

(14) $$CrossEntropyLoss=C.E.loss=\{\begin{array}{c}-log{L}_{FP}L=1\\ -log{L}_{BP}L=0\end{array}$$

Cross-entropy loss is a probability measure used for both segmentation and classification, where the more improved form of C.E. loss is Focal Loss (F. Loss), with a modulation factor ${L_{BP}}^{\gamma }$ of the predicted background label mask value, as shown in Eq. (15).

(15) $$FocalLoss=F.Loss=-{L_{BP}}^{\gamma }log{L}_{FP}$$

The D.E loss and F. loss is used for the segmentation performance measurement, where Total Loss (T. Loss) is shown in Eq. (16).

(16) $$TotalLoss=T.Loss=D.L+w_t(F.Loss)$$

The predicted and actual masks having both foreground and background pixel counts with an overlapping area division of union for both are calculated as the mean intersection over union (Mean-IoU) for all testing images, as calculated in Eq. (17).

(17) $$MeanIoU={\int }_{I=1}^N{\displaystyle \frac{OverlappingAreas}{UnionofAreas}}$$

(18) $$MeanF1-score=\left(1+w_t\right){\displaystyle \frac{\left\{Product\left(Precision,Recall\right)\right\}}{\{Union\left(Precision,Recall\right)\}}}$$

Another performance measure of precision- and recall-based product and union, with a product of w_t as a constant value, ensures that the value is in a particular range, as calculated in Eq. (18). The total testing loss is 0.17, with a Mean-IoU of 0.79, and the mean F1-score is 0.88. These prediction results are entirely justified, but in some cases, re-consideration is needed. Therefore, quantization is performed to determine exact tumor lesions for more promising breast cancer classification.

ICA feature-based classification of BUSI data

The extracted features are first given to the extracted 5-fold validation method, which is evaluated in terms of accuracy, precision, recall, f1-score, and kappa statistical index value. The results are shown in Table 4.

The confusion matrix of all methods are combined in one table and shown in Table 5, and a graphical representation of the extracted evaluation measures is shown in Fig. 8. The classification models all reach almost 80%, which is not sufficient. The F1-score, having an effect of both precision and recall, yields approximately 85% for all classifiers. However, the best model using ICA features with a 5-fold validation method is bag-boot in terms of accuracy, and the R-boost is the best among all precision values.

The best recall value can be considered the most accurate model, bag-boost, which shows a 90.16% value. However, the F1-score based on true positives and false negatives is more considerable for any classification evaluation method. Similarly, the Kappa–Cohen index shows a low level of agreement (McHugh, 2012) for all methods, but bag-boost is the most robust of all of them.

The extracted features are again validated on the mainly used 10-fold cross-validation method, which shows almost the same effect in all measures, with slight changes. The results are shown in Table 6.

The confusion matrix of all methods is combined in one table and shown in Table 6. The visual description is shown in Fig. 9. Tables 5 and 7 comparatively show the effect of the same features and classification methods with fewer and more folds. However, the 5-fold results are better than the 10-fold results, which have fewer chances of bias because they have more folds. Therefore, if we see in Table 7, the bag-boost has fewer wrong predictions than other methods.

The visualization in Figs. 8 and 9 also show slight changes in cones in all the applied classification methods. However, the ICA-based classification was not so promising, showing less accuracy, so the state-of-the-art method of the deep-learning-based transfer learning of features is applied. These features are then fused with ICA features to check both effects.

Serial fused results

The deep-transfer-learning-based features can be extracted from many state-of-the-art ImageNet variants that are pre-trained for object classification. The proposed CNN-based transfer learning can also be used to extract deep features. However, denseNet201 has 700+ layers that are dense enough for feature extraction. The extracted deep features are then serially fused with ICA features; the fusion process is already explained by the serial fusion sub-section in the Methodology section. Finally, the fused features are extracted and validated on the aforementioned 5- and 10-fold approaches with the same classification methods.

We can observe from Table 8 that the most accurate method in terms of ICA features changed due to the fusion of both types of features. This is due to the changing range of feature values by the combined effect of both types of features. However, in this case, the best method is Q-SVM, which shows an accuracy of 98.45% with a recall rate of 100%. The F1-score measure with true positives and false negatives is 98.87%, which is also the highest among all classification methods. The confusion matrix of all methods is also improved, as shown in Table 9.

As per Table 9, the Q-SVM, which gives the highest prediction accuracy, has 0 wrong predictions in the benign class, where the malignant class has 10 wrong predictions. It is clear that the benign class has more training and testing instances, which may lead to the higher amount of accurate prediction results for this class. However, the other class results can also be improved by taking more data.

The graphical representation in Fig. 10 shows that the C-SVM, Q-SVM, and bag-boost are more accurate than the other classification methods used. These fused features are again validated on 10-fold methods and shown in Table 10.

Another validation method is used to confirm the results of the proposed method. The 10-fold classification have more training and testing, and again shows the same good results and an improved performance over other classification methods. If we look into accuracy values comparing 5- and 10-fold classification, the C-SVM is improved values compared to the ICA feature results. The other bag-boost is also improved by 1.14%, where the other precision value is also increased to 100%. Results obtain using serial fusion-based 10-fold prediction are presented in Table 11.

The confusion matrix of all methods is reasonable and justified, but the bag-boost has no wrong results in the benign class, and C-SVM and Q-SVM show one wrong prediction. However, the malignant cases are 8 and 9 in C-SVM and Q-SVM, respectively, whereas the bag-boost shows 12 wrong predictions. Therefore, C-SVM is the most accurate in both class predictions compared to the other two models.

The recall value in Fig. 11 has the highest slope among the best three classification methods of the proposed method, whereas the other M-tree and R-boost have less accurate results in the case of fused features. The other dataset used for testing purposes was also tested on the same features and classification methods. The images were not of the same intensity levels but were of the same modality. Therefore, the same fused features were extracted, but yielded higher results for the feature regularization method of lasso regression. Finally, the regression-based classification was performed and is discussed in the next section.

Linear regression based OASBUD data classification

The OASBUD segmented data was used by the same feature extraction methods but failed to yield promising results on the classical machine learning methods. Therefore, lasso regularization with a binomial distribution was used on the extracted features for feature selection and classification purposes, where input feature data were fed as a 5- and 10-fold validation method. The results of both validation techniques are justified, as shown in Table 12.

The noisy and raw data of the signaling data were used with the same set of features, which also reached a 94% accuracy using feature selection using lasso regularization. For cross-check, the two validation techniques were applied as in previous classical ML methods. The recall, precision, or either kappa index all give strong agreement upon the prediction results.

The confusion matrix of both validation methods show that there are three wrong predictions in the case of benign class, whereas there are 3 and 4 wrong predictions in the case of malignant class. The total of 100 instances can also be validated from Table 13. The results are shown in Fig. 12.

The OASBUD dataset was also cross-validated by the ultrasonic dataset with the proposed fused features as a promising method of identifying ultrasonic breast cancer. The proposed method was subsequently compared with state-of-the-art methods to show its superiority.

Comparative analysis

The proposed method was used in different ways to improve accuracy, precision, recall, and F1-score, which was achieved by comparing it with recent studies on the same dataset. The proposed method improved in all ways, including accuracy. These recent studies using various approaches are shown in Table 14.

Table 14 clearly shows that all used classification methods result in improvement as compared to other studies. The compared studies reach a 94.62% accuracy with a 90% precision, a 92.31% recall, and a 91.14% F1-score. The proposed study used five methods that all have an accuracy higher than 96%, and all other measures are also higher than those of the studies compared.