Automatic cassava disease recognition using object segmentation and progressive learning

Datasets and protocols

For our study, we utilize an official cassava dataset consisting of 21,367 labeled images, which was specifically curated for the Kaggle competition (https://www.kaggle.com/c/cassava-leaf-disease-classification/data) (Mwebaze et al., 2020). The dataset was gathered through a regular survey conducted in Uganda, Africa. The majority of the cassava images in the dataset were crowdsourced from cassava producers who captured images from their own gardens. These images were then annotated by experts at the National Crops Resources Research Institute (NaCRRI) in collaboration with the Makerere Artificial Intelligence (AI) Lab at Makerere University in Kampala, Uganda. This dataset provides a comprehensive and diverse collection of cassava images, enabling us to train and evaluate our cassava disease recognition system effectively. By utilizing this dataset, we aim to contribute to the advancement of cassava disease recognition technology and facilitate practical applications in the agricultural domain.

The dataset used in our study consists of five categories: Cassava Bacterial Blight (CBB), Cassava Green Mottle (CGM), Cassava Brown Streak Disease (CBSD), Cassava Mosaic Disease (CMD), and healthy leaves without any disease. These categories were assigned based on the judgments of agricultural experts. Figure 1 illustrates some representative images from each of these five categories. The five categories are defined based on their health status and distinctive symptoms. CBB is marked by angular, water-soaked spots that turn brown with yellow borders, accompanied by yellowing leaves, wilting, and branch dieback. CGM affects young leaves with puckering, yellow patterns, mosaic-like green patches, twisted margins, and occasional severe stunting. CBSD exhibits yellow or necrotic vein banding that may coalesce into large yellow patches, without leaf distortion. CMD is distinguished by severe mosaic symptoms, including mottling, discoloration, and malformation of leaves. Healthy leaves lack these pathological features and mostly appear uniform and undistorted.

A summary of the dataset statistics is presented in Table 1. Notably, the dataset exhibits a significant class imbalance. Only approximately 12% of the dataset comprises images of healthy leaves, while the majority of the images represent diseased leaves. Class 3, in particular, contains more samples than all the other classes combined, thereby adding to the complexity of the classification task. The presence of class imbalance poses a challenge in training and evaluating our cassava disease recognition system. However, by addressing this issue and leveraging advanced machine learning techniques, we aim to develop an effective solution that can accurately identify different types of cassava diseases in real-world scenarios.

To simulate the automatic monitoring of cassava leaves in real-life field conditions, volunteers were instructed to capture photos under various conditions, including different weather conditions, illuminations, resolutions, and viewpoints. The challenging conditions in the dataset training images primarily encompass low lighting, which may affect the image either partially or globally; noise, often manifesting as blurriness; and cluttered backgrounds, which may include unrelated objects, large non-relevant areas, human presence, or shadows causing occlusions. Representative examples of these adverse conditions are illustrated in Fig. 2.

Cassava leaf disease recognition approach

In our approach to cassava leaf disease recognition, we first isolate the cassava leaf from complex backgrounds to improve classification accuracy, which would be compromised if performed on the entire image. This isolation begins with traditional clustering techniques, using Gaussian mixture models (GMM), to segment leaf objects, as is an effective pre-processing technique. However, limitations in speed and precision arise with GMM alone. To address these, we incorporate a self-supervised U-Net model, fine-tuned for efficient and precise segmentation. Once segmented, the cassava leaf images undergo feature embedding through a progressive learning algorithm (PLA) to generate a high-dimensional feature vector optimized for disease classification. Finally, we use an ensemble of pre-trained deep learning models to classify the segmented leaf images, achieving a robust framework for accurate and scalable disease detection.

Preprocessing by leaf object segmentation

Before the cassava disease recognition step, it is of vital importance to carry out localization of the cassava leaf object from the cluttered background. Otherwise, if the classifier is applied directly to the whole image, the recognition accuracy of the system will be drastically reduced due to the cluttered background. In the applications of plants, methods have been developed to highlight green vegetation in imagery from widely-used RGB cameras. It has been demonstrated that plant health can be evaluated via the Normalized Difference Vegetation Index (NDVI) (Pettorelli, 2013).

In this work, by applying GMM (Deng, Li & Hong, 2014) based on image feature vectors, we can separate the foreground, i.e. the cassava leaf part, from the cluttered background by solely using traditional image processing methods. Color feature is obtained by the Elliptical Color Index (ECI) (Lee, Golzarian & Kim, 2021) and Color Index Vegetation Extraction (CIVE) (Kataoka et al., 2003), and texture feature is obtained by Gabor filter (Bovik, 1991), respectively. A texture-based feature vector which is 24-dimensional is obtained for each image pixel using a convolution based on Gabor filtering. Given R, G, and B channels of an image, first Gaussian filtering is applied to blur an image, and then vegetation index is calculated, followed by normalization to the range of $\left[0,255\right]$. The ECI and CIVE color indices are defined as follows:

(1) $$ECI={\left(R-1\right)}^2+G^2/0.16$$ (2) $$CIVE=0.441\ast R-0.881\ast G+0.385\ast B+\mathrm{18.787.}$$

To brief the GMM algorithm, the first two sets of segmented regions are obtained which are partitioned by clustering according to the color feature (2-dimension consisting of ECI and CIVE) and the texture feature (24-dimension), respectively. By iteratively maximizing the similarity between the two sets of segmented regions according to the similarity measure, we can obtain the optimized weight vector. Then the color and texture features are combined with the obtained optimal weights and perform the final clustering based on the combined feature vector. We denote $\mathbf{V}=\left\{{\mathbf{v}}_1,{\mathbf{v}}_2,\cdots ,{\mathbf{v}}_N\right\}$ as the set of color or texture feature vectors, then the probability density of a GMM containing K components can be formulated as

(3) $$p(\mathbf{v})=\sum _{j=1}^k{\pi }_j{\varphi }_{{\mathrm{\Sigma }}_j}\left(\mathbf{v}-{\mathbf{c}}_j\right)$$ (4) $${\varphi }_{{\mathrm{\Sigma }}_j}(\mathbf{v}-{\mathbf{c}}_j)=\frac{1}{\sqrt{2^d{\pi }^d\left|{\mathrm{\Sigma }}_j\right|}}\exp \left(-\frac{1}{2}{\left(\mathbf{v}-{\mathbf{c}}_j\right)}^T{\mathrm{\Sigma }}_j^{-1}\left(\mathbf{v}-{\mathbf{c}}_j\right)\right)$$where ${\pi }_j$ is a mixing coefficient of one of the different Gaussian filtering components, which are positive and sum to one, ${\mathbf{c}}_j$ is the centroid vector of each component, ${\mathrm{\Sigma }}_j$ denotes a covariance matrix, and ${\varphi }_{{\mathrm{\Sigma }}_j}(\mathbf{v}-{\mathbf{c}}_j)$ denotes multivariate normal density.

Regarding the estimation of the parameters ${\pi }_j$, ${\mathbf{v}}_j$ and ${\mathrm{\Sigma }}_j$, the expectation maximization (EM) technique is utilized for maximizing the log-likelihood. The EM procedure is carried out iteratively after initialization. Firstly, EM performs E-step, which gives an estimation of the color vector probability density ${\mathbf{v}}_i$ of the component $c_j$ of the $t$-th iteration:

(5) $$p^{(t)}\left(c_j\mid {\mathbf{v}}_i\right)=p\left({\mathbf{v}}_i\mid c_j\right)/\sum _{l=1}^{K}p\left({\mathbf{v}}_i\mid c_l\right)$$ (6) $$s_j^{(t)}=\sum _{i=1}^{N}p\left({\mathbf{v}}_i\mid c_j\right)/\sum _{l=1}^{K}p\left({\mathbf{v}}_i\mid c_l\right)$$

Secondly, the parameters of GMM are then updated during the M-step as follows:

(7) $${\pi }_j^{(t+1)}=\frac{1}{N}{s}_j^{(t)}$$

(8) $${\mathbf{c}}_j^{(t+1)}=\frac{1}{s_j^{(t)}}\sum _{i=1}^N{\mathbf{v}}_i\cdot p^{(t)}\left(c_j\mid {\mathbf{v}}_i\right)$$

(9) $${\mathrm{\Sigma }}_j^{(t+1)}=\frac{1}{s_j^{(t)}}\sum _{i=1}^N{p}^{(t)}\left(c_j\mid {\mathbf{v}}_i\right)\left({\mathbf{v}}_i-{\mathbf{c}}_j^{(t+1)}\right){\left({\mathbf{v}}_i-{\mathbf{c}}_j^{(t+1)}\right)}^T$$

In this work, as the cassava image content can be roughly divided into foreground/background parts, we set $K=2$ as the number of components. Once the cassava image is converted to the two-component GMM representation, we select the image pixels that correspond to the component with higher values in the green channel to be the foreground cassava leaf. Figure 3 depicts the process of the foreground detection, where the original and the extracted cassava images are shown in the left and the right, respectively.

Leaf object segmentation by deep learning

Now the foreground cassava leaf object is obtained by GMM. However, there are some disadvantages: (1) There are some artifacts in object segmentation, e.g., the leaf objects have small holes identified as background object parts. This is due to the lack of noise reduction in traditional object segmentation approaches, and (2) The GMM process is rather slow although it is visually accurate. This is because of the iterative optimization process using traditional feature extraction.

In view of these issues, we propose a novel self-supervised object segmentation approach. Firstly, we apply GMM to infer the pseudo-truth segmentation masks of the cassava images. This is done by randomly cropping $512\times 512$ images from the original images, before extracting features and applying GMM segmentation to generate the vegetation mask. As ECI and CIVE only work correctly on healthy plants, all diseased samples are filtered out, but data augmentation techniques are applied to simulate diseases. Secondly, the vegetation mask of each $512\times 512$ image sample will be kept as a ground truth mask, and fed into a U-Net (Ronneberger, Fischer & Brox, 2015) for training the segmentation model via deep learning.

We specially designed an enhanced U-Net structure which shows visually accurate segmentation with reasonable computation efficiency, which is shown in Fig. 4. The enhanced U-net is essentially comprised of four different convolution modules: (1) Base module; (2) dense module; (3) transition down module; and (4) transition up module, as shown in Fig. 5. The enhanced U-Net has deeper convolutions and seven intermediate loss values during the transition down and transition up stages are summed up to the final stage loss value, enabling the U-Net to achieve more stable training and better accuracy.

In order to reduce the object boundary artifact for better classification, we masked the background with pure green color. Typical segmentation results from the proposed U-Net are illustrated in Fig. 6. The original images, GMM segmented images, and U-Net segmented images are juxtaposed in Fig. 7.

Learning feature embedding

For images masked by foreground cassava leaf objects, the progressive learning algorithm (PLA) (Li et al., 2021, 2022) is exploited to learn the discriminative capability. Figure 8 illustrates the architecture, where the final feature vector is 1,024-dim that is produced by $1\times 1$ convolution of the feature vectors followed by global pooling. These features are optimized through a combination of cross-entropy loss and triplet loss in the back-propagation of the training process. The final 1,024-dim feature embeddings will be generated by feature concatenation, and used for inference.

Triplet loss distinguishes itself from traditional classification loss by producing the feature embedding vectors (Schroff, Kalenichenko & Philbin, 2015; Shi et al., 2016; Hermans, Beyer & Leibe, 2017), and the metric embedding mapping between the original image data and the optimal semantic features is learned: $g\left(\mathit{\theta }\right):{\mathbb{R}}^D\to {\mathbb{R}}^F$, where ${\mathbb{R}}^D$ represents the manifold of the image data, ${\mathbb{R}}^F$ represents the space of semantic features, and $\mathit{x}$ is the input data. We define $\mathit{\theta }$ as the network parameters. In this work, the non-linear function $g\left(\mathit{\theta }\right)$ is implemented through a deep learning network. The metric mapping is defined as $D_{i,j}=D\left(g_{\theta }\left(u\right),g_{\theta }\left(v\right)\right):{\mathbb{R}}^D\times {\mathbb{R}}^D\to R$.

The concept of triplet loss originates from the Large Margin Nearest Neighbor (LMNN) loss (Weinberger, Blitzer & Saul, 2005):

(10) $$\begin{array}{r}{L}_{LMNN}\left(\theta ;X\right)=\left(1-\mu \right)\sum _{y_a=y_b}{D}_{a,b}+\mu \sum _{\underset{y_a=y_b\ne y_n}{a,b,n}}\left[m+D_{a,b}-D_{a,n}\right]\end{array}$$where $a$ and $b$ are samples from an identical class, while $m$ and $\mu$ represent the margin parameter and the weighting factor, respectively. Ideally, based on such loss functions, $g\left(\mathit{\theta }\right)$ is likely to cluster images that are semantically similar onto adjacent feature points in the metric space, and disperse semantically dissimilar images onto distant points in the metric space.

In PLA (Li et al., 2021, 2022), two progressive parameters $p$ and $k$ are introduced to control the process when optimizing the triplet loss. Let $k\in \left[1,\dots ,K\right]$, $p\in \left[1,\dots ,P\right]$, and $\mathrm{L}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{s}{\mathrm{t}}^k(\cdot )$ and $\mathrm{S}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{s}{\mathrm{t}}^p(\cdot )$ represent the value picking operations upon the batch $D_{a,b}$ which ranks the $k$-th largest, and the value from $D_{a,n}$ which ranks the $p$-th smallest, respectively. Consequently, the original triplet loss is generalized as

(11) $${\mathcal{L}}_{GBH}^{k,p}\left(\mathit{\theta };X\right)=\sum _{l=1}^P\sum _{\underset{y_a=y_b=l}{a,b}}\mathrm{l}\mathrm{n}\left(1+e^{m+T_{k,p}\left(a,b,n\right)}\right)$$

(12) $$T_{k,p}\left(a,b,n\right)=\mathrm{L}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{s}{\mathrm{t}}^k\left(D_{a,b}\right)-\underset{y_n\ne y_a}{\mathrm{S}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{s}{\mathrm{t}}^p}\left(D_{a,n}\right).$$

Any discrete hard level can be configured by adjusting the parameters $k$ and $p$ of the above generalized batch hard loss. When $k=1$, only the hardest examples, which might even be outliers, are used for training, resulting in an unstable training process as in batch hard loss. However, when $k>1$, the too-hard or wrong labels are circumvented, and the unstable training situation is alleviated.

Furthermore, the classification loss is integrated to smooth the loss and ameliorate side-effects of the over-difficult training samples. The classification loss is normalized via Softmax:

(13) $${\mathcal{L}}_{softmax}\left(\mathit{\theta };X\right)=-\sum _{i=1}^{PK}\mathrm{l}\mathrm{n}\frac{e^{W_{y_i}^T{\mathit{f}}_i+b_{y_i}}}{{\sum }_{j=1}^M{e}^{W_{y_j}^T{\mathit{f}}_j+b_{y_j}}}$$where M denotes the total classes of the training dataset, and W represents the weights of a $1\times 1$ convolution function.

Consequently, the composite objective function becomes the combination of the cross-entropy classification loss and the generalized triplet loss as follows:

(14) $${\mathcal{L}}^{k,p}\left(\mathit{\theta };X\right)={\mathcal{L}}_{softmax}\left(\mathit{\theta };X\right)+\lambda {\mathcal{L}}_{GBH}^{k,p}\left(\mathit{\theta };X\right)$$where $\lambda$ denotes a predefined trade-off factor to balance the generalized triplet loss and the conventional classification loss. It is determined by a Bayesian optimization procedure in Li et al. (2021).

The whole training procedure is carried out in an end-to-end manner as shown in Algorithm 1. Considering a minimal requirement of memory usage, we set $P=4$, $K=8$ to as the training image batches parameters.

Algorithm 1 :

Cassava classifier training algorithm.

Input: From the training set, construct mini-batches containing $P=4$ randomly chosen classes and $K=8$ randomly chosen images per class.

Output: The optimal CNN weights that will produce 1,024-dim feature embedding.

Initialization: Initialize N sets of random hyper-parameters $\mathbb{W}=\left\{w_1,w_2,\cdots ,w_N\right\}$ where $w_i=\left({\lambda }_i,m_i,k_i,p_i\right)$, ${\lambda }_i\in \left[0,2\right]$, $m_i\in \left[-0.1,0.3\right]$,

$k_i\in \left[1,8\right]$, $p_i\in \left[1,4\right]$, for $i=1,\cdots ,N$.

Repeatably Do

for $i=1$ to N:

Exploration: Back-propagate network for 10 epochs and evaluate $\mathcal{L}$ via Eqs. (11) and (14), followed by the evaluation of Bayesian objective $f\left({\mathit{w}}_i\right)$.

Restoration: network weights are recovered to the values prior to the 10 exploration epochs.

end for

Exploitation: Generate a new improved version of candidate $w^{\mathrm{\prime }}$ according to $f\left(\mathbb{W}\right)$, followed by the Gaussian update process, and add $w^{\mathrm{\prime }}$ to $\mathbb{W}$.

Perform back-propagation to enable network weights updating through 300 epochs once the hyperparameter $\hat{w}$ is updated;

Record the model weights of the lowest $\mathcal{L}$ corresponding to the current hyperparameter $\hat{w}$;

Until the maximum number of epochs ( $M=1,000$) is finished.

DOI: 10.7717/peerj-cs.2721/table-101

Implementation details

The preliminary experiments were conducted on a system equipped with an Intel Core i7-9700K CPU running at 3.6 GHz, 16 GB of RAM, and an NVIDIA GTX 2080Ti GPU with 11 GB of VRAM. The software environment consisted of TensorFlow 2.4 and Python 3.8, running on Ubuntu 20.04. The final score in Kaggle competition was obtained using Kaggle free computing environment of NVIDIA TESLA P100 GPUs. The two environments have consistent results.

All images are resized according to network input shapes, which may vary in different networks with regard to trade-off between efficiency and accuracy. We first resized the image to target shapes, then did a series of image augmentation steps in the training process.

The training dataset, consisting of 21,367 images, is split in a stratified manner into 90% for training and 10% for validation. The testing dataset comprises 15,000 images, which is set by the Kaggle platform. During the training process, five-fold stratified cross-validation is employed to ensure robust evaluation.

The backbone models, ResNet-50 (He et al., 2016) and the EfficientNet series (Tan & Le, 2019), are implemented following the methodologies outlined in their respective original articles. Training details, including hyperparameter settings, model architectures, and training protocols, remain consistent with the original implementations. For further information, please refer to the original publications.

We use Adam (Kingma & Ba, 2014) as the optimizer to learn feature embedding and set ${\beta }_1=0.9$ within 150 epochs and ${\beta }_1=0.5$ for remaining epochs, and ${\beta }_2=0.999$. We followed the learning rate scheduler as proposed by Hermans, Beyer & Leibe (2017):

(15) $$\alpha (\mathrm{e})=\{\begin{array}{cc}{\alpha }_0\hfill & ife\le e_0\hfill \\ {\alpha }_0\times {0.001}^{\frac{e-e_0}{e_1-e_0}}\hfill & if{e}_0\le e\le e_1\hfill \end{array}$$where we set ${\alpha }_0=3\times {10}^{-4}$, $e_0=150$ epochs, and $e_1=300$ epochs.

The image resolutions and classification loss functions for different base networks selected for the ensemble are listed in Table 2. The classification loss has two options: cross entropy loss (Li et al., 2020) and bi-tempered (BiT) logistic loss (Amid et al., 2019). For further details, refer to the implementation code in the Kaggle competition platform (https://doi.org/10.5281/zenodo.14739855).

Data augmentation

Although challenging conditions such as low light, noise, and cluttered backgrounds are already present in the training image dataset, the model trained on this dataset demonstrates resistance to these conditions to some extent.

To further enhance its robustness, we incorporate a range of image augmentation techniques during the training process. These augmentations include random cropping, random flipping, hue saturation value (HSV) adjustment, random brightness-contrast adjustment, normalization (with mean = [0.485, 0.456, 0.406] and standard deviation = [0.229, 0.224, 0.225]), and dropout. Random erasing (Zhong et al., 2017) is also employed, which randomly ( $p=0.5$) masks portions of cassava leaves in the images. These data augmentation strategies aim to simulate a wide variety of real-world conditions, enhancing the model’s ability to focus on disease-relevant features while ignoring extraneous factors. By diversifying the training data, these techniques enhance the model’s generalization capabilities, enabling it to perform reliably across diverse environmental conditions and image quality scenarios.

Specifically, these data augmentation techniques applied during the training process aim to enhance the model’s generalization capabilities: (1) HSV adjustment, Random Brightness-Contrast adjustment: These augmentations simulate lighting variations, including low light conditions and overexposure, to improve the model’s robustness under diverse illumination scenarios. Brightness adjustments also introduces variability in image brightness, mimicking noise caused by lighting inconsistencies. (2) Random erasing: This technique masks parts of cassava leaves, emulating occlusion and missing data, which indirectly aids in generalizing to noise-like conditions and helps the model generalize by masking parts of the background. (3) Normalization: Normalizing image intensities to a standard mean and standard deviation reduces sensitivity to global lighting differences.

Furthermore, integrating restoration techniques, such as illumination restoration based on latent diffusion model (LDM) (He et al., 2023a), could enhance preprocessing steps and improve the model’s performance under extreme degradations. Since maintaining short inference times for efficiency is critical for real-world deployment, this preprocessing step in inference phase is skipped. However, in cases of severe low-light conditions, the diffusion-based image restoration such as He et al. (2023a) could be integrated into the system as a preprocessing step to improve robustness.

Although there is a significantly higher representation of Class 3, the dataset’s imbalance also reflects the natural distribution of cassava leaf disease classes observed in real-world scenarios. This alignment suggests that the private test set is likely to follow a similar class proportion, minimizing concerns about dataset imbalance in practical applications. Nevertheless, steps were taken to address the imbalance in the training dataset to ensure optimal model performance.

To address the class imbalance, additional data augmentation techniques were applied to the underrepresented classes. The augmentation aimed to increase the diversity and quantity of data for these classes, reducing the likelihood of the model overfitting to the dominant class. The implementation of data augmentation is TensorFlow’s ImageDataGenerator package. This augmentation pipeline introduces variability through transformations such as rotations, shifts, flips, zooms, and brightness adjustments, thereby enhancing the training dataset for Classes 0, 1, 2, and 4. By enriching the dataset for these minority classes, the model is better equipped to generalize across all classes, mitigating the impact of imbalance.

Evaluation metrics

To comprehensively evaluate the performance of the proposed method, we utilized several key performance metrics including:

• –

  Precision: Precision quantifies the number of true positives among the predicted positives. It is defined as $\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{TP}{TP+FP}$.

• –

  Recall: Recall measures the proportion of actual positives correctly identified by the model: $\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{TP}{TP+FN}$.

• –

  Specificity: Specificity captures the model’s ability to correctly identify negative instances: $\mathrm{S}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{TN}{TN+FP}$.

• –

  F-Measure: The harmonic mean of precision and recall is the F-Measure, often used for imbalanced datasets: $F1=2\times \frac{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}$.

• –

  G-Mean: G-Mean balances sensitivity and specificity for binary classification: $$\text{G-Mean}=\sqrt{\mathrm{S}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{y}\times \mathrm{S}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}}$$

• –

  Matthews correlation coefficient (MCC): MCC is a balanced measure that accounts for true and false positives and negatives: $$\mathrm{M}\mathrm{C}\mathrm{C}=\frac{TP\times TN-FP\times FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}$$

To ensure the robustness of our results, we conducted a statistical evaluation using three key measurements: mean, root mean squared error (RMSE), and standard deviation (STD) across all runs. These metrics ensure that the performance achieved by the proposed model was not due to chance.

• –

  Mean: The mean value of each evaluation metric across multiple runs is used to estimate the expected performance of the model. – RMSE: RMSE measures the differences between predicted and observed values and is a good indicator of model accuracy $\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{\frac{1}{n}{\sum }_{i=1}^n{(y_i-{\hat{y}}_i)}^2}$.

• –

  STD: Standard deviation provides insight into the variability of model performance across different trials $\mathrm{S}\mathrm{T}\mathrm{D}=\sqrt{\frac{1}{n-1}{\sum }_{i=1}^n{(x_i-\overline{x})}^2}$.

Ablation study

We use ResNet-50 (He et al., 2016) and EfficientNet (Tan & Le, 2019) backbones as the network architecture. In order to validate the effects of our proposed foreground leaf object segmentation approach and progressive feature embedding learning approach, we compare GMM segmentation and PLA separately on different backbone neural networks.

Table 3 summarizes the effectiveness of GMM segmentation using standard classification with cross-entropy loss on cassava leaf dataset, reporting in public accuracy scores in the Kaggle competition. EfficientNet-b4 performs best among the choices of base networks, however, the ensemble of six base networks will outperform any single basenet. With the proposed object segmentation, the concatenated feature embedding using cross-entropy loss achieved accuracy improvement ranging from $0.75\mathrm{\%}$ to $1.32\mathrm{\%}$, respectively. This indicates that GMM segmentation is an effective pre-processing technique that consistently boosts cassava disease classification. The ensemble model benefits from $1.03\mathrm{\%}$ improvement with GMM segmentation.

From Table 4, one can observe that, the proposed progressive learning algorithm consistently improves performance for each base network, ranging from $0.78\mathrm{\%}$ to $1.10\mathrm{\%}$ in terms of accuracy. This indicates that PLA is an effective feature learning technique that consistently boosts cassava disease classification by combining cross-entropy loss and generalized triplet loss. The ensemble model benefits from $0.95\mathrm{\%}$ improvement with PLA.

Table 5 shows the effectiveness of combining GMM and PLA for cassava classification. With cross-entropy loss only, GMM achieves $89.60\mathrm{\%}$ accuracy; With PLA using cross-entropy and generalized triplet loss, the learned feature embedding achieved an accuracy of $89.64\mathrm{\%}$. Since GMM and PLA are independent steps, the combination of GMM and PLA outperforms the accuracy of GMM or PLA alone. The ensemble model using GMM and PLA also outperforms the accuracy using each single base network.

Error case analysis

To better understand the limitations of the proposed model, we conduct an edge/error case analysis by examining misclassified samples. Table 6 presents exemplar images of misclassifications. The table includes ground truth labels as column headers (CBB, CGM, CBSD, CMD, Healthy) and thumbnails of misclassified images, where each image is labeled with its predicted class (italicized). Below, we provide a detailed discussion of the challenges posed by these cases.

Table 6:

Exemplar images of wrong classifications.

CBB	CGM	CBSD	CMD	Healthy

DOI: 10.7717/peerj-cs.2721/table-6

Note:

The top row shows the ground truth labels: CBB, CGM, CBSD, CMD, and Healthy. Each image is labeled with its incorrect prediction (italicized).

• CBB: The first image resembles a mixture of CBB and CBSD characteristics, confusing the model into predicting CBSD. The second image contains a dominant green background, with the actual CBB object being very small, making it challenging for the model to detect. The third image visually resembles a combination of CBB and CGM, resulting in a CGM prediction.

• CGM: In the first image, the real CGM object is overshadowed by surrounding grass, leading to misclassification. The second image predominantly displays green leaves, appearing similar to healthy cassava leaves. The third image is a close-up, with features resembling CBSD, causing confusion in the prediction.

• CBSD: The first example is a noisy-labeled image, which depicts a tuberous root of the cassava plant instead of leaves, significantly confusing the model. The second image is primarily green, resembling healthy cassava leaves. The third image contains various miscellaneous objects, presenting a significant challenge for the model.

• CMD: For the first two images, the real CMD objects are overshadowed by healthy leaves, causing the model to predict a healthy class. The third image appears visually similar to CBSD, leading to incorrect classification.

• Healthy: For the first image, the model struggles to confidently classify the category due to insufficient information related to leaf diseases. The second image, captured from a distance, includes a mix of various leaf types, leading to confusion. The third image bears a strong visual resemblance to CBB, posing a challenge for the model to classify it accurately.

This analysis underscores several key challenges, including noisy labeling, complex and cluttered backgrounds, visual similarities between classes, and the presence of small or occluded objects. These observations suggest potential areas for improvement, such as implementing enhanced pre-processing techniques, employing advanced data augmentation strategies, or integrating multi-scale feature learning to bolster the model’s robustness.

However, when cassava leaf images are properly captured—featuring close-up views with the subject centered and minimal background clutter—the model achieves near-perfect classification accuracy.

Statistical evaluation and discussions

The confusion matrix is shown in Table 7, with 100 samples randomly selected from each category. The matrix highlights that the proposed model effectively identifies both true positives and true negatives while maintaining a low false positive and false negative rate.

We evaluated the proposed model and baseline models on the cassava disease dataset. Table 8 summarizes the results across different evaluation metrics. Our proposed model achieved superior performance in all categories, highlighting its efficacy in cassava disease recognition.

To ensure the reliability of the results, we performed a statistical analysis of the model’s performance across five trials. The evaluation metrics (Precision, Recall, Specificity, F1-Measure, G-Mean, and MCC) were computed for each run, and the Mean, RMSE, and STD were calculated to evaluate the consistency of the model.

As evident in Table 9, The low RMSE and STD values across all metrics demonstrate that the proposed model consistently outperforms baseline methods without significant variation in its results. The statistical analysis further confirms that the achieved performance is not due to random chance.

The proposed cassava disease classification approach with GMM and PLA also outperforms other top-place deep learning methods in the Kaggle competition, as shown in Table 10. Our initial submission received a silver medal (19th place). In late submission, we got the ever best result by submitting a better model with more finetuning on the cassava training dataset.

For the 1st place solution as shown in Table 10, a diverse model ensembling strategy is employed to achieve robust performance. Their approach prioritized diversity over fine-tuning individual models, leveraging various architectures such as ResNet, ResNeXt, Xception, Vision Transformers (ViT and DeiT), and EfficientNet. Key features of this winning approach include model diversity over extensive fine-tuning, advanced augmentation techniques and data fusion, and leveraging domain-specific pre-trained models like CropNet. Comparing their ensemble strategy with our results highlights the value of exploring broader model architectures and innovative ensembling techniques to further enhance system performance.

The 2nd place solution in the competition adopted a relatively straightforward approach, leveraging a pre-trained cassava-specific model from TensorFlow Hub (CropNet) for classification. It also utilized callbacks such as “EarlyStopping” and “ReduceLROnPlateau” to prevent overfitting, typically halting training after approximately 30 epochs. This straightforward approach achieved remarkable success. This outcome highlights the potential of crop domain-specific pre-trained models to deliver competitive performance with minimal fine-tuning.

The 3rd place solution utilized an ensemble of three Vision Transformer (ViT) models, i.e., ViT-Base-Patch16-384, ViT-Base-Patch16-224 (Pattern A), and ViT-Base-Patch16-224 (Pattern B). demonstrating the strength of transformer architectures in image classification tasks. This approach emphasized careful design choices to balance model complexity and generalization, aiming to mitigate overfitting. The results underscore the effectiveness of ViT models for the task and the importance of incorporating multiple architectural variants into an ensemble to capture diverse feature representations.

Our late submission achieved the highest accuracy on both public and private leaderboards, surpassing the 1st-place solution by 0.02% on the public leaderboard and 0.11% on the private leaderboard. These results underscore the efficacy of our approach in addressing the challenges of cassava leaf disease classification.

Unlike many competition entries that relied on ensemble methods or computationally intensive architectures, our model emphasizes both accuracy and practicality. Specifically, the architectural choices, including the use of EfficientNet and ResNet-50 backbones, provide a balance between high performance and computational efficiency. This makes our model more suitable for deployment in real-world agricultural settings, particularly in resource-constrained environments.

Moreover, the training strategies employed, such as advanced data augmentation (e.g., random erasing, HSV adjustment, and random brightness-contrast adjustment), further contributed to the model’s robustness in handling diverse scenarios, such as occlusions, variations in lighting, and background clutter. These design choices distinguish our approach from other solutions, demonstrating the feasibility of applying the proposed algorithm in practical applications.

This comparative evaluation highlights the advantages of our model in achieving competitive results with a focus on real-world applicability, setting a new benchmark for cassava disease detection.

Efficiency evaluation

For PCR and microscopic methods, it appears that the durations can vary based on specific protocols and laboratory setups. For instance, some PCR tests can deliver results in as little as 2 h, while others may take 1 to 3 business days (Arya et al., 2005). Similarly, the turnaround time for microscopic examinations can range from within 1 day to 2–3 business days (University of Missouri Veterinary Medical Diagnostic Laboratory, 2023). Given this variability, the traditional microscopic methods typically need more than 2 h to produce the results, and the PCR methods cost at least 30 min. Both methods achieves nearly 100% accuracy theoretically, but they are time-consuming and costly.

In contrast, our proposed deep learning method takes only 4.663 s to complete the classification procedure. The code is implemented in Python with moderate code optimization. We can conclude that the proposed algorithm outperforms traditional microscopic methods significantly in efficiency, with more than 90% accuracy for the leaf recognition systems.

To comprehensively evaluate the computational cost of our proposed method, we compare it with traditional PCR and microscopic techniques, as well as various deep learning architectures, including ResNet-50, EfficientNet (b3, b4, b5, b6, b7), and an ensemble model. The computational cost is measured in terms of the total number of basic operations (Mul-Add), which directly correlates with actual running time. The running times for ResNet-50 and the EfficientNet series are proportionally estimated based on the Mul-Add values relative to the proposed ensemble method.

The Mul-Add operations can be recomputed using profiling tools such as PyTorch Profiler. The computational cost of an ensemble model depends on the number of models combined and their respective complexities. If an ensemble combines ResNet-50 and EfficientNet variants, the cost can be approximated by summing their individual GFLOPs, accounting for shared computation if implemented efficiently.

Accuracy metrics for the deep learning models are listed in Table 11 along with their Mul-Add and evaluation times, while the typical accuracy of PCR and microscopic methods is referenced from existing literature.

As shown in Table 11, ResNet-50 provides a good trade-off between computational cost and accuracy, with 3.9 GigaOps and 89.69% accuracy. EfficientNet models are more efficient, with b3 achieving similar accuracy to ResNet-50 at less than half the computational cost (1.8 GigaOps). However, as the EfficientNet series scales up (b4 to b7), the computational cost increases significantly, with diminishing returns on accuracy improvements.

The PCR method demonstrates high accuracy (close to 100%) but has significant time constraints, typically requiring at least 30 min to complete the classification process. Similarly, the microscopic method achieves close to 100% accuracy but still requires over 2 h, making it less practical for large-scale or time-sensitive applications.

While the ensemble model achieves the highest accuracy (90.11%), it incurs the highest computational cost (about 75.8 GigaOps) due to the aggregation of all baseline models. This makes it the best trade-off of effectiveness and efficiency in resource-limited environments.