Blood cell image segmentation and classification: a systematic review

WBC segmentation

In medical image analysis, segmenting the region of interest (ROI) is the initial rudimentary step for image processing (Caicedo et al., 2019). Based on the efficient segmentation of blood cell elements like WBCs, RBCs, and platelets, several analyses can be carried out after the segmentation, i.e., feature extraction, classification, etc. These analyses provide valuable information during morphological analysis and disease diagnostic procedures (Liu & Long, 2019). In Nikitaev et al. (2018) introduced a technique for segmenting leukocytes from blood and bone marrow cells. A total of 1,018 cell images were formed out of 50 samples. Background and erythrocytes (RBCs) are separated using luminance histogram analysis of the color component. Object selection (filling), screening artifact, and identification of “sticky” WBCs were achieved by Watersheds and distance transformation techniques. A total of 82% of the nuclei were allocated correctly. They selected three types of artifacts; the first type of artifact was allocated in 8% of the cells, the second type of artifact was 4%, and the third type of artifact was 12% of the cells. In Alomari et al. (2014), Monteiro et al. (2020), Miao & Xiao (2018), proposed algorithms based on marker-controlled watershed and thresholding, morphological-based techniques for systematically segmenting leukocytes and erythrocytes. The algorithms work in two steps: cell nucleus segmentation and blood cell segmentation. The time complexity of the proposed method is 0.32 s, which is less than Wei and Cao’s (0.46 s) method (Wei & Cao, 2016). While it is greater than (Cuevas et al., 2013), whose method complexity was 0.3 s. The proposed method is consistent and coherent in datasets, with the accuracy rates of the WBC segmentation at 97.2% and RBCs at 94.8%. The proposed method may perform negatively on the poor image quality. Shirazi et al. (2016a) proposed a novel technique to enhance and segment leukocytes from microscopic images. Curvelet, transform, and wiener filters were used for image enhancement and noise removal. The snake algorithm and Gram-Schmidt orthogonalization were used for boundary detection and segmentation. Shirazi et al. (2016c), the authors try to overcome the overlapping problems of the blood cells. They got a segmentation accuracy rate of 100%, 96.15%, 92.30%, 92.30%, and 96.15% for basophil, eosinophil, monocyte, lymphocyte, and neutrophil, respectively. They achieved the segmentation by using a Wiener filter and Curvelet transform for image enhancement and noise elimination. They used the ALL-IDB blood cell dataset (Labati, Piuri & Scotti, 2011). Cao, Liu & Song (2018) performed a deep study on the segmentation of leukocytes in peripheral blood by introducing an algorithm SWAM&IVFS (stepwise averaging method) with consideration of RGB color space, HIS color space, and linear combination of G, H, and S component. Results show that the accuracy rate for segmentation was 93.75% with a standard deviation of 0.5984%. A total of 203 reference images were considered, which were sketched by the medical expert. Of these, 10.34% were monocytes, 24.14% lymphocytes, 60.59% neutrophils, 3.94% eosinophils, and 0.99% basophils. In Liu et al. (2015), the authors used the nucleus mark watershed operation and mean shift clustering technique to efficiently segment WBCs in peripheral blood and bone marrow images. The whole method is divided into four steps. In the first step, they obtain the nucleus as the inside seed by using RGB and HIS color space. In the subsequent stage, they generated white blood cells (WBCs) as an external seed by employing mean shift clustering operations, extracting the C channel component in the CMYK model, implementing illumination intensity adjustment, and utilizing image enhancement techniques. The third step relates to cell adhesion by using NMWO, followed by post-processing techniques to obtain the nucleus and WBCs. ALL-IDB1 datasets were used for testing the results. The proposed algorithm outperforms in the case of P, R, and F1 measures with an accuracy rate of 99%, 95.5%, and 97%, respectively. Quiñones et al. (2018) introduce a novel technique for the WBCs segmentation and counting using the HSV saturation component with blob analysis. A total of 12 images were used for the experiments. This model’s accuracy for counting leukocytes was 90.91%, the lowest, and 100%, the highest. While the execution time was 0.032 s slowest and 0.112 s as the highest execution time. Tosta et al. (2015) proposed an unsupervised segmentation method for the nuclear structure in WBCs. They used the threshold Neighborhood Valley-emphasis algorithm to select the region of interest in the image. A total of 367 images were used to test the proposed methodology. The accuracy with Jaccard was 89.89%, while with accuracy matrices, 99.57%. Shahin et al. (2018) used an adoptive neutrosophic similarity score to segment white blood cells. They proposed an algorithm based on the multi-scale similarity measure in the neutrosophic domain. They used a color segmentation framework for nucleus and cytoplasm segmentation. The quantitative results show high accuracy rates of the segmentation performance measurement 96.5% and 97.2% of the proposed method. In Abdulhay et al. (2018), Saleem et al. (2022), Sharma & Buksh (2019) authors proposed leukocyte identification based on the ROI segmentation approach with 98.5% and 95.3% accuracy for detecting leukemia cells using Hybridge CNN and SVM, respectively. Several survey works have been carried out, Aldrin Karunaharan & Prakash (2018), Bagasjvara et al. (2017), Reyes, Rozo & Morales (2015), Rodellar et al. (2018), Shafique & Tehsin (2018b), but most of them target few areas, i.e., comparison of just two techniques for segmentation (Threshold Segmentation Based on Mathematical Morphology (TSMM) and Segmentation Based on Fuzzy Cellular Neural Networks (FCNN)), explore the process of segmentation with different techniques, review on the leukemia detection techniques and detection of acute lymphoblastic leukemia (ALL) cells respectively, but all of them mostly covered traditional approaches not deep learning.

Table 1 is the literature summary regarding developing the WBCs segmentation techniques dataset used for WBC segmentation. It also shows the relevant features and image enhancement techniques commonly used before the WBCs segmentation process.

Table 2 summarizes the literature regarding WBC segmentation performance accuracy and relevant evaluation parameters that are most commonly used for the WBCs segmentation process.

Figure 2, Part B, shows the literature regarding WBC segmentation. It shows the research work, techniques used, and common limitations in the domain of WBC segmentation.

RBC segmentation

In Das, Maiti & Chakraborty (2018), authors put forward a method for the automated detection of nucleated red blood cells (RBCs) in peripheral blood smears. A Special Fuzzy C-mean algorithm was used for image enhancement. They got an accuracy of 99.42% for detecting nucleated RBCs from the blood smear images. A multilevel threshold approach made the selection of nucleated blood cells. Segmentation was achieved by using a marker-controlled watershed algorithm. In Miao & Xiao (2018), Shahzad et al. (2020), the authors used a marker-controlled watershed and a modified SegNet model. They accomplished simultaneous segmentation of red blood cells (RBCs) and white blood cells (WBCs) with accuracies of 94.8% 97.2%, and 97.45% and 93.43%, respectively. In Shahzad et al. (2021), Shirazi et al. (2016b), the author proposed a novel technique for automated diagnosis of several blood diseases like AIDS, iron deficiency, blood disorders, platelets, malaria, leukemia and anemia. The author proposed machine-based blood morphological analysis. The author performed the segmentation of RBCs and WBCs along with their sub-types. They merged different methodologies like feature, color domain, and classifier for cell segmentation. In Imran & Ahmad (2017), Shirazi et al. (2016a) devised a statistical thresholding approach for segmenting red blood cells (RBCs), which was then succeeded by using Fuzzy C-means for accurate segmentation and boundary detection. Texture and geometrical features were used for classification. They tested their results on the ALL-IDB blood cell dataset. They performed single-cell-based RBC classification. A total of 10,000 RBCs were used, of which 8,000 were normal and 2,000 were abnormal. The overall classification accuracy of this technique was 96%. Another work (Al-Hafiz, Al-Megren & Kurdi, 2018) has been performed on RBC segmentation using thresholding value and corner detection with 87.9% accuracy. This method fails to accurately detect edges of images that show overlapped structures of cells. In Rehman et al. (2018), the author introduced an innovative technique for separating Rouleaux red blood cells from thin blood smears by implementing distance transform and local maxima. The proposed method was assessed using three evaluation parameters, namely TPR (96%), ER (4%), and AC (98%).

Furthermore, semantic segmentation was conducted by Chaudhary et al. (2019), Tran et al. (2019) using SegNet architecture and VGG, respectively, for the segmentation and quantification of white blood cells (WBCs) and red blood cells (RBCs) in the ALL-IDB-I dataset. This architecture achieved an average accuracy of 93%. In Aliyu et al. (2019), normal and abnormal RBCs were identified by leveraging Form Factor, Perimeter, and area features, achieving an accuracy of 94%. However, the method was ineffective when dealing with noisy images.

Table 3 shows the literature summary regarding developing segmentation techniques and the dataset used for RBC segmentation. It also shows the relevant features and image enhancement techniques commonly used for RBC segmentation.

Table 4 summarizes the literature regarding RBC segmentation performance accuracy and relevant evaluation parameters that are most commonly used in the literature.

Figure 2, part A summarizes the literature regarding RBC segmentation. It shows the research work, techniques used, and common limitations in the domain of RBC segmentation.

Geometric or morphological features

Morphological features directly interact with visual reflection and narrative made by the pathologist. Usually, these features are easy to interpret, observe, and segment for different diagnostic analyses. The most commonly used descriptor parameters are (Puigví & Alférez, 2018) area, perimeter, diameter, elongation, convexity, nuclear size, cell size, shape, nucleus-cytoplasmic ratio, nuclear eccentricity, roundness, eccentricity, and circularity.

Color features

Color-based image descriptors also play a crucial role in identifying the ROI and edges of the object. It is a physical property commonly used for visual observation. Various models have been used for color feature extraction, i.e., RGB, CMYK, HSV, and Lab and Luv models (Elhassan et al., 2022; Rodellar et al., 2018). These models perform work based on a key tool called a histogram.

Texture features

Despite the color and geometric features, texture descriptors are relatively difficult to observe for visual interpretation. In the digital image processing domain, the texture is defined with the help of pixel tone, density of pixel, uniformity, and their spatial relation with each other (Puigví & Alférez, 2018). There are two main texture descriptor classes: Gray-level co-occurrence matrix and granulometry.

Table 5 shows the quantitative features, relevant accuracy, and the blood database most commonly used for blood cell segmentation and classification.

WBCs classification

After completing ROI segmentation and succeeding feature extraction process, the arithmetical descriptors distinctively identify each blood cell element. Classification is a procedure that labels the ROI with these numerical descriptors among predefined classes. This section will explore several research works on classifying blood cell elements using traditional and deep learning approaches. Table 6 shows the literature summary regarding developing classification techniques and datasets used for WBC classification. It also shows the relevant features and image enhancement techniques commonly used before the WBC classification. In Liang et al. (2018), authors proposed a CNN-RNN framework to classify blood elements. They initially introduced RNN and then combined it with CNN for better results. For classification, the proposed methodology integrates the features extracted by the RNN and local features obtained from the CNN. A transfer learning technique transferred pre-trained weight parameters to the CNN section. They used BCCD (Shenggan, 2018) (small-scale dataset for blood cell detection) and got 12,444 enhanced blood cells by pre-processing the BCCD dataset, out of which 2,487 were test data and 9,957 were training data with 20% and 80% ratio, respectively. This dataset is divided into four different categories, namely, monocyte (2,478), lymphocyte (2,483), neutrophil (2,499), and eosinophil (2,497) for the training purposes, while in the test dataset, the distribution of cells was 620, 623, 624, and 623 as monocyte, lymphocyte, neutrophil and eosinophil respectively. The entire network used RGB images of size 320 × 340 × 3 pixels. Results show that CNN-RNN models (Xception-LSTM, Inception V3-LSTM, RestNet50-LSTM, and Xception- RestNet50-LSTM) outperform with a classification accuracy of 90.79%, 87.45%, 89.38%, and 88.58% as compared to the other CNN models (Inception V3, RestNet50, Xception) 84.08%, 87.62%, and 88.70% respectively. But, regardless of high classification accuracy based on the dataset, these experiments were performed on most single-cell images. The single-cell classification process may take 3.8 s, which is clinically too slow. This method also faces problems where multiple cells overlap and needs to design a task-specific classifier to consider such issues. In Hegde et al. (2018), proposed an image-processing algorithm for classifying WBCs and nuclei based on the nuclei features. TissueQuant and image enhancement methods were used to manage color and illumination variation for nuclei detection. They used 117 images with a magnification rate of 100X acquired with an OLYMPUS CX31 microscope with 1,600 × 1,200 resolution from Leishman-stained peripheral blood smear. Out of 117 images, 14 were basophils, 30 neutrophils, 33 lymphocytes, 23 monocytes, and 22 eosinophils. Shape and texture features of the detected nuclei were used to classify white blood cells. A five-class approach with a neural network and a cell-by-cell approach with a hybrid classifier (SVM & NN) was used to classify WBCs. By comparing both approaches, cell-by-cell approach outperformed with the rate of 1.4%. The accuracy of other WBCs required more analysis of cytoplasm. This method outperformed the detection of lymphocytes and basophils with 100% accuracy by considering two features, i.e., shape and texture. But the accuracy of neutrophils and eosinophils was 93.4% and 94.3%, and the sensitivity rate was 81.8% and 86%, respectively. More study of cytoplasm is required for the accurate detection of other WBCs. In Yadav, Zele & Patil (2018), introduced an automatic image-handling framework for the early detection of blood cancer. From the microscopic image, they examine shape, size, cell nuclei, and blood cell distribution for the determination of cancer. Prewitt and Sobel or Canny were used to remove artifacts and noise. Segmentation was achieved using region-based segmentation, K-means Zack Algorithm, Morphological operation, gradient magnitude, and watershed transform with a supervised machine learning approach for classification. In Jung et al. (2022), Othman, Mohammed & Ali (2017), authors introduced feed-forward back-propagation neural network-based and generative adversarial networks-based techniques for WBCs classification. They got a 96% classification accuracy rate of white blood cells with 16 selected features. However as they increased the number of features up to 69, the accuracy rate decreased by 89.74%. Di Ruberto, Loddo & Putzu (2016) proposed a white blood cell count (WBCC) system to speed up and accurately count cells. WBCs are counted by applying the circular Hough transform, exploiting the grey-level information. This method was tested on ALL-IDB1, ALL-IDB2, Raabin-WBC(Kouzehkanan et al., 2022), and IUMS-IDB datasets (Alomari et al., 2014). The accuracy rate of counting WBCs was 99.2%. For the segmentation of blood cells, they used the SVM technique. Several deep learning-based techniques (Agaian, Madhukar & Chronopoulos, 2018; Cheuque et al., 2022; Liu & Long, 2019; Mishra, Majhi & Sa, 2019; Rawat et al., 2017; Shafique & Tehsin, 2018a; Sharma et al., 2022; Ullah et al., 2023; Vogado et al., 2018; Zhao et al., 2017) have been developed for the detection of leukemic cells in microscopic images. Most of them classified the WBCs into their subtypes with heterogeneous accuracy rates shown in Table 7. Nuclei detection of white blood cells was demonstrated in Hegde et al. (2019) using a novel deep learning framework based on the TissueQuant algorithm with 99% and 96.5% average accuracy and dice coefficient, respectively.

Table 7 shows the summary of previous work regarding WBC classification performance accuracy. 3rd column shows the evaluation parameters used for the evaluation of WBC classification.

Figure 5, Part A summarizes the whole literature regarding RBC classification, while Part B, summarizes the literature regarding WBC classification. It shows the research work, techniques, and common limitations in the WBC and RBC classification domains.

RBCs classification

In Kihm et al. (2018), authors classify RBCs on the basis of their shape. They introduced an outlier tolerant machine learning technique to find out the slipper-shaped and croissant-shaped cells in the blood. They also find the transition point between both stable and unstable phases. Results show that this technique classifies the croissant-shaped cell with an accuracy rate of 85.6% and the slipper-shaped cell with a 91.8% accuracy rate. Shirazi et al. (2017), Imran & Ahmad (2017) introduced a novel technique based on the extreme learning machine (ELM) approach for classifying RBC images. Statistical-based thresholding methods were used for initiating RBCs segmentation, followed by the fuzzy C-means for efficient segmentation and boundary detection. Texture and geometrical features were used for classification. They tested their results on the ALL-IDB blood cell dataset. They performed single-cell-based RBC classification. A total of 10,000 RBCs were used, of which 8,000 were normal and 2,000 were abnormal. The overall classification accuracy of this technique was 96%. Another work (Das, Maiti & Chakraborty, 2018) has been carried out for the classification of nucleated RBCs by using 250 blood smear images. This technique is based on the fuzzy C-mean algorithm with an accuracy rate of 99.42%. Yi, Moon & Javidi (2016) Proposed a technique for automatically classifying RBCs by selecting a linear or non-linear classifier. This method used Gabor-filtered holographic images to measure covariance matrices to classify RBCs. The methodology consists of three basic steps. 1) Numeric re-construction of RBCs from the hologram, 2) marker-controlled watershed transform algorithm was used to segment RBCs, and 3) Gabor wavelet transform was applied for classification along with multivariate statistical test to evaluate the equality of the covariance of matrices. Abood, Karam & Hluot (2017) proposed a technique for classifying RBCs disease using the Fuzzy logic theory. They performed statistical-based analyses like mean, standard deviation, variance, roundness, skewness, and kurtosis, considering size, shape, and color features. The proposed method got an accuracy of 98% for red blood cells. In Acharya & Kumar (2017), the authors presented a computer-aided system designed to diagnose blood disorders, such as Anemia, by classifying red blood cells (RBCs). The K-medoids algorithm was employed to extract white blood cells (WBCs) from the image. To differentiate RBCs from WBCs, granulometric analysis was performed on 1,000 blood smear images obtained from Kasturba Medical College, Manipal, Karnataka, and Atlas of Hematology (2023), Hemato-pathology laboratory (Mathew et al., 2015), and the Internet. The system yielded a 98% accuracy rate for correctly identifying Anemia; however, the execution time increased with the number of RBCs present. Xu et al. (2017) proposed a deep convolutional neural network (CNN) for classifying RBCs in sickle cell anemia. They introduced a high-throughput framework consisting of three stages. First, they developed an automatic hierarchical RBC extraction method for detecting RBCs. Second, they applied a mask-based RBC patch-size normalization method to normalize the segmented single RBCs areas into uniform sizes. Third, they employed deep CNNs to classify the RBCs. A total of eight sickle cell disease patients were selected for collecting 7,000 single RBC images. The proposed method got a 91.01% mean accuracy for RBCs classification under different folds with a mean evolution accuracy of 89.28%. Du et al. (2019), Sampathila, Shet & Basu (2018) proposed a framework for the classification of malaria parasites using RBCs morphological features. The later referred technique uses convolutional neural network for automatic classification blood cell that have ability to integrate with facial detection system.

Table 8 illustrate the literature concerning the development of classification techniques and dataset used for RBC classification. It also shows the features and image enhancement techniques commonly used for RBC classification.

Table 9 shows the summary of previous work regarding RBC classification performance accuracy. A total of 3rd column shows the evaluation parameters used for the evaluation of RBC classification.