Automatic visual recognition for leaf disease based on enhanced attention mechanism

Methods of crop disease identification

In recent years, crop disease identification methods have undergone significant evolution, largely due to advancements in object recognition detection algorithms and the integration of deep learning techniques. These methods have found successful applications across various domains, including medical image recognition (Huang et al., 2021), pedestrian gait recognition (Dou et al., 2022), and crop disease recognition (Verma et al., 2021). Deep learning, functioning as an end-to-end target recognition and detection algorithm, mimics the network structure and operational mechanisms of the human brain, enabling independent perception and learning of intrinsic features of the target. Several classical algorithms, such as Single Shot MultiBox Detector (SSD) (Liu et al., 2016), Region-based Convolutional Neural Network (R-CNN) (Girshick et al., 2014), Spatial Pyramid Pooling Network (SPP-Net) (He et al., 2015), Region-based Fully Convolutional Network (R-FCN) (Dai et al., 2016), and You Only Look Once (YOLO) (Redmon et al., 2016), have been developed based on deep learning.

Among these algorithms, YOLO stands out for its transformation of the detection problem into a regression problem and its ability to achieve real-time target detection using a convolutional neural network. Consequently, an increasing number of researchers are integrating deep learning with crop disease recognition (Xinming & Hong, 2023; Xue et al., 2023; Morbekar, Parihar & Jadhav, 2020).

For instance, Mohanty, Hughes & Salathé (2016) trained a deep convolutional neural network model using a plant leaf disease dataset, achieving recognition of 26 crop diseases. Notably, the model also recognized disease images not present in the training set. Too et al. (2019) explored the application of a fine-tuned deep learning model for plant disease classification using the PlantVillage dataset, with experimental results demonstrating that the DenseNet network model achieved the best recognition performance.

Turkoglu, Hanbay & Sengur (2022) proposed a model for detecting apple pests and diseases based on a multi-model LSTM convolutional neural network. This hybrid model combined the LSTM network with a pre-trained CNN model, with experimental results showing higher accuracy compared to other models. Nachtigall, Araujo & Nachtigall (2016) introduced a classification method for apple tree diseases based on convolutional neural networks. By leveraging CNN, relevant features of apple tree diseases were learned from the data, with experimental results indicating that the trained convolutional neural network outperformed human experts in identifying apple tree diseases.

In 2021, Shill & Rahman (2021) developed an accurate plant disease detection system using YOLOv3 (Redmon & Farhadi, 2018) and YOLOv4 (Bochkovskiy, Wang & Liao, 2020), respectively, achieving the detection of diseases related to 17 plant leaves. Experimental results demonstrated the system’s high accuracy and applicability. Ganesan & Chinnappan (2022) proposed a rice disease recognition algorithm based on a mixed deep learning model, in which a YOLO classifier replaced the fully connected layer of the ResNet model. Experimental results showed that the recognition algorithm achieved high accuracy. The TC-MRSN model (Wang et al., 2024) excels in diagnosing maize leaf diseases under complex conditions by employing a dual-branch system to effectively capture texture and color features with high precision. The SENet approach (Wen et al., 2024) integrates a multi-scale residual network with Squeeze-and-Excitation mechanisms for precise recognition of mulberry leaf diseases. A method for recognizing pepper leaf diseases (Fu, Guo & Huang, 2024) introduces a lightweight CNN model based on the GGM-VGG16 architecture. Trained on images against a human palm background, it operates as a mobile application for efficient and accurate diagnosis in field conditions. The CoffeeNet model (Nawaz et al., 2024), employing a novel deep learning strategy, addresses the challenge of accurately diagnosing coffee plant leaf diseases. It incorporates spatial-channel attention mechanisms within a ResNet-50-based architecture and utilizes the CenterNet framework for streamlined one-step detection. Furthermore, the GhostNet Triplet YOLOv8s algorithm (Li et al., 2024) enhances maize leaf disease detection, providing a more efficient and accurate solution for real-time agricultural diagnostics. Additionally, an approach (Deari & Ulukaya, 2024) combines Inception v3 for classification with YOLOv5x for precise symptom localization, enhancing early detection and preserving yield.

Although the above methods have made significant progress in terms of recognition accuracy, the reality of plant disease detection requires high detection efficiency under varying light and background conditions. We will enhance the ability to detect plant diseases in these challenging environments.

Identification method of tomato disease

Durmuş, Güneş & Kırcı (2017) utilized deep learning for tomato leaf disease detection using the PlantVillage dataset, conducting tests on the AlexNet and SqueezeNet deep learning network models. Experimental results highlighted SqueezeNet’s lightweight nature, enabling real-time identification of nine tomato diseases. Brahimi, Boukhalfa & Moussaoui (2017) proposed a tomato disease recognition algorithm based on CNN, leveraging CNN’s automatic feature extraction to visualize disease regions in tomato leaves, achieving high accuracy as indicated by experimental results. Rangarajan, Purushothaman & Ramesh (2018) introduced a tomato disease classification algorithm based on pre-trained deep learning network models, specifically AlexNet and VGG16-net, analyzing the effects of image quantity, small batch size weight, and bias learning rate on tomato disease recognition performance.

While these algorithms applied deep convolutional neural networks to tomato leaf disease recognition using the PlantVillage dataset, it is crucial to note that these datasets lack characteristics such as complex backgrounds, variable illumination, and close contrast between disease spots and the background, as illustrated in Fig. 1. Therefore, when applying these deep learning network models to datasets similar to the one shown in Fig. 1 for identifying tomato leaf diseases, detection accuracy is often compromised. However, detection speed can be higher, making it more feasible to achieve the desired recognition effect.

Fuentes et al. (2017) proposed a recognition method for tomato diseases and insect pests based on deep learning, exhibiting robustness in complex environmental conditions and enabling effective recognition of nine tomato diseases, thereby facilitating early prevention of tomato diseases and insect pests. Mohandas, Anjali & Varma (2021) introduced a real-time plant leaf disease recognition algorithm based on YOLOv4 (tiny), capable of recognizing various crop diseases, including those affecting tomatoes, mangoes, strawberries, beans, and potatoes. Experimental results demonstrated that the proposed algorithm can achieve early-stage recognition of plant diseases. Lin et al. (2017) proposed a tomato and apple leaf disease recognition algorithm based on YOLOv4, utilizing EPC to optimize the algorithm’s learning rate, ultimately achieving recognition of eight tomato diseases and insect pests. Additionally, due to its ability to focus on specific regions akin to human vision, attention mechanisms have gradually found applications across various domains, yielding fruitful outcomes (Liu et al., 2022). Nevertheless, there is scarce research leveraging attention mechanisms in conjunction with YOLO for leaf disease classification.

Hard target identification of tomato disease spot by fusion focaler-SIoU

To address the challenges posed by irregularly shaped and difficult-to-classify tomato disease samples, this article introduces Focaler-SIoU, which effectively improves the accuracy of bounding box regression and emphasizes the recognition of hard samples.

YOLOv4 uses the CIoU (Zheng et al., 2020) for bounding box regression by default, which takes into account the similarity between the ground truth boxes and predicted boxes.

(5) $$CIoU=IoU-\frac{{\rho }^2(b,b^{gt})}{c^2}-\beta v,$$

(6) $$\beta =\frac{v}{(1-IoU)+v},v=\frac{4}{{\pi }^2}{\left(\arctan \frac{w^{gt}}{h^{gt}}-\arctan \frac{w}{h}\right)}^2,$$where $w^{gt}$, $h^{gt}$, $w$, and $h$ represent the width and height of the ground truth bounding box and the predicted bounding box, respectively. However, the CIoU does not consider the impact of the angles between bounding boxes. Thus, we introduce SIoU (Gevorgyan, 2022):

(7) $$SIoU=IoU-\frac{(\mathrm{\Delta }+\mathrm{\Omega })}{2}.$$

This further considers the angular compatibility between ground truth boxes and predicted boxes, thereby effectively enhancing the accuracy of bounding box regression.

(8) $$\mathrm{\Lambda }=1-2\cdot {\sin }^2\left(\arcsin (z)-\frac{\pi }{4}\right),$$

(9) $$z=\frac{c^h}{d}=\sin (\phi ),c^h=max(b_{c^y}^{gt},b_{c^y})-min(b_{c^y}^{gt},b_{c^y}),d=\sqrt{{(b_{c^x}^{gt}-b_{c^x})}^2+{(b_{c^y}^{gt}-b_{c^y})}^2},$$where $\mathrm{\Lambda }$ is the angle cost. $b_{c^x}^{gt}$, $b_{c^y}^{gt}$, and $b_{c^x}$, $b_{c^y}$ are the coordinate values of the centers of the ground truth box and the predicted box, respectively. $c^h$ and $d$ denote the height displacement and Euclidean distance between the centers of the ground truth bounding box and the predicted bounding box. By minimizing $z$, the angle $\phi$ between the ground truth box and the predicted box can be minimized.

(10) $$\mathrm{\Delta }=\sum _{t=x,y}(1-e^{-\tau {\rho }_t}),\tau =2-\mathrm{\Lambda },{\rho }_x={\left(\frac{b_{c^x}^{gt}-b_{c^x}}{c^w}\right)}^2,{\rho }_y={\left(\frac{b_{c^y}^{gt}-b_{c^y}}{c^h}\right)}^2,$$where $\mathrm{\Delta }$ is the distance cost, and $c^w$ denotes the width displacement between the centers of the ground truth bounding box and the predicted bounding box. By incorporating the angle cost into the distance cost, when $\phi$ increases, the contribution of the distance cost also increases.

(11) $$\mathrm{\Omega }=\sum _{t=w,h}{(1-e^{-{\mathrm{\Phi }}_t})}^{\theta },{\mathrm{\Phi }}_w=\frac{|w-w^{gt}|}{max(w,w^{gt})},{\mathrm{\Phi }}_h=\frac{|h-h^{gt}|}{max(h,h^{gt})},\theta =4,$$where $\mathrm{\Omega }$ is the shape cost. For different datasets, the value of $\theta$ varies. The variation of $\theta$ affects the rate of exponential decay, influencing the extent to which $\mathrm{\Omega }$ penalizes differences between the ground truth box and the predicted box.

The obtained SIoU loss can be expressed as:

(12) $$L_{SIoU}=1-IoU+\frac{(\mathrm{\Delta }+\mathrm{\Omega })}{2}.$$

Due to the diversity of tomato leaf diseases and the differences in shape, color, and texture between individuals, there is a significant proportion of hard samples encountered during the recognition process. We introduced Focaler-IoU, which focuses more on the bounding box regression of hard samples. Specifically, we reconstruct the IoU loss through linear interval mapping, aiming to perform more accurate bounding box regression.

(13) $$FocalerIoU=\{\begin{array}{ll}0,& \mathrm{i}\mathrm{f}IoU<k\\ {\displaystyle \frac{IoU-k}{g-k},}& \mathrm{i}\mathrm{f}k\ll IoU\ll g\\ 1,& \mathrm{i}\mathrm{f}IoU>g\end{array},\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}[k,g]\in [0,1].$$

By varying the factors $g$ and $k$, we can increase the emphasis on the regression of hard samples. We set $k=0$ and $g=0.95$ in this article. The final Focaler-SIoU loss can be expressed as:

(14) $$L_{Focaler-SIoU}=L_{SIoU}+IoU-FocalerIoU.$$

By incorporating the Focaler-SIoU loss, the proposed model focuses more on hard samples and improves the detection capability for small targets associated with tomato disease spots. This further enhances the model’s learning ability for hard samples.

The influence of the enhanced module DyHead on recognition performance

In subsequent experiments in this article, the base model YOLOv4 (tiny) without the enhanced module DyHead serves as the baseline. The purpose is to compare the performance of the base model with the performance of the models that incorporate different numbers of DyHead modules. DyHead modules with different numbers and embedding dimensions are introduced into the base model to analyze their impact on effective feature extraction and recognition performance of tomato diseases.

As shown in Table 2, we observe that the accuracy of detection increases continuously as the number of stacked DyHead modules and embedding dimensions rise. However, this enhancement comes with an increase in parameters and consequently, detection time. Considering both detection accuracy and speed, we ultimately opt to stack 2 DyHead modules in the base model and set the embedding dimension to 128. The model achieves a detection accuracy of 93.17%, with a slight reduction in speed.

Hence, by introducing an appropriate number of DyHead modules, which combine multi-head attention mechanisms, it becomes possible to effectively extract relevant spatial and scale features, thereby enhancing the model’s feature learning and representation capabilities. This proves crucial in achieving optimal model performance. For all subsequent experiments within this article, we employ 2 DyHead modules with embedding dimensions set to 128.

The effect of Focaler-SIoU loss on recognition performance

The incorporation of Focaler-SIoU into the proposed algorithm represents a notable enhancement in recognition accuracy. As demonstrated in Table 1, the integration of Focaler-SIoU elevates the average detection accuracy of the method to 93.64%. This improvement, amounting to an increase of nearly 0.5% compared to the variant without Focaler-SIoU, is achieved without compromising the algorithm’s average detection speed.

Focaler-SIoU allows the algorithm to effectively train on scenes featuring a higher prevalence of challenging instances of tomato diseases. These “hard samples” typically involve diseases that are more nuanced or less frequently encountered, posing greater difficulty for traditional detection models. By prioritizing these challenging cases through the Focaler-SIoU mechanism, the algorithm can allocate more resources and attention during training, thereby refining its recognition capabilities specifically for these scenarios. The observed increase in accuracy underscores the efficacy of Focaler-SIoU in bolstering the algorithm’s performance in tomato disease recognition tasks. Focaler-SIoU not only improves overall detection rates but also enhances the algorithm’s robustness in handling diverse and challenging conditions commonly encountered in agricultural field settings.

Compared with the base model

As shown in Table 3, the experimental results demonstrate that our proposed algorithm effectively improves the recognition accuracy of tomato diseases while maintaining a balanced recognition speed. Our model achieves 93.64% accuracy, which is a 10.3% improvement compared to the YOLOv4 (tiny)/baseline model. It even surpasses YOLOv7(tiny) at 91.29% and YOLOv5 at 92.44%, with faster detection speed.

More specifically, we compared our proposed model with the YOLOv4 (tiny)/baseline model in terms of AP and F1 score for each type of tomato disease, as shown in Tables 4 and 5. Notably, for tomato leaf flavivirus, the proposed algorithm exhibits the highest increase in AP value, with an improvement of about 17.9% compared to the YOLOv4 (tiny)/baseline model. Additionally, for tomato leaf flavivirus and healthy tomato leaves, the proposed method achieves improvements of 18% and 15% in F1 value, respectively, compared to the YOLOv4 (tiny)/baseline model.

These results validate that our proposed method effectively completes feature extraction for each type of tomato disease, accurately identifies hard tomato samples, and improves the overall recognition effect of the algorithm for each type of tomato disease. The experimental findings confirm the effectiveness of the proposed method in enhancing the recognition accuracy of tomato diseases.