General retrieval network model for multi-class plant leaf diseases based on hashing

Machine learning approaches in plant disease

Putzu, Di Ruberto & Fenu (2016) proposed a mobile application for leaf analysis and automatic plant species identification. This application primarily addresses the identification and segmentation steps, tackling the major issue of uncontrolled lighting conditions and achieved highly accurate results. Li, Gao & Shen (2010) introduced a method for the automatic identification of three wheat diseases by analyzing morphological features extracted from images. This method segments images of wheat powdery mildew, wheat sheath blight and wheat stripe rust to obtain target areas, extracts and optimizes morphological data and combines principal component analysis with discriminant analysis. The method ultimately selects sphericity, roundness, Hu1, Hu2 and equivalent radius as recognition factors, achieving sample identification rates of 96.7%, 93.3% and 86.7% for the three wheat diseases, respectively. Yun et al. (2015) considered the statistical features of leaf images and meteorological data, proposing a crop disease identification method that successfully identified three cucumber diseases with an accuracy of 90%. Le et al. (2020) proposed a novel method that improves the discrimination rate for broadleaf plants by combining features extracted using local binary pattern operators with those from plant leaf contour masks. This method filters noise in plant images using morphological opening and closing operations and classifies crops and weeds using a support vector machine classifier. It achieved a classification accuracy of 98.63% on the “bccr-segset” dataset, significantly outperforming previous methods with an accuracy of 91.85%. Campanile, Di Ruberto & Loddo (2019) developed an open-source ImageJ plugin for image analysis in the biological field, particularly focusing on botany research. This plugin extracts morphological, texture and color features from seed images for classification purposes. Yağ & Altan (2022) developed a hybrid model using FPA, SVM and a simplified CNN for efficient plant disease detection. By selecting optimal features from 2D-DWT transforms of plant images, the model achieves high accuracy with low computational complexity. Deployed on an NVIDIA Jetson Nano, it enables real-time classification of plant diseases, offering a fast and lightweight solution for agricultural applications. Kebapci, Yanikoglu & Unal (2010) proposed a method combining color, shape and texture features for plant image retrieval. They first used the max-flow min-cut (MFMC) graph cut method for interactive segmentation to separate plants from the background. By extracting color, shape and texture features and inputting them into a SVM classifier, effective plant image retrieval was achieved. This research considered not only the local features of plant leaves but also the overall shape, background and flower color of the plants, providing a comprehensive feature description for plant image recognition. Bama et al. (2011) proposed a plant leaf image retrieval method based on shape, color and texture features. They first converted RGB images to the HSV color space and applied Log Gabor wavelets to the saturation channel to extract texture features. They then used the scale-invariant feature transform (SIFT) algorithm to extract key points and performed corner detection for key point elimination. Finally, descriptor ratio matching was used for leaf image retrieval. This method effectively improved the accuracy of plant leaf image retrieval, especially for images with complex backgrounds. Piao et al. (2017) proposed a method combining image descriptors to enhance the performance of crop disease image retrieval systems. By first calculating the similarity between images using a single descriptor and then summing the similarities of multiple descriptors to generate a new similarity score, the system completed image retrieval based on this. Experimental results showed that the use of combined descriptors universally improved system performance, with different optimal descriptor combinations found for different crops. Baquero et al. (2014) proposed an image retrieval strategy based on color structure descriptors and nearest neighbor algorithms for diagnosing greenhouse tomato leaf diseases. This strategy successfully characterized various anomalies such as chlorosis, sooty mold and early blight, aiding non-experts in assessing and classifying tomato disease symptoms. Shrivastava, Singh & Hooda (2016) proposed an efficient soybean leaf disease detection and classification method based on image retrieval. The study introduced two feature descriptors, HIST and WDH and explored various color and texture feature descriptors, such as BIC, CCV, CDH and LBP, for automated soybean disease recognition. The method effectively detected and classified disease regions using background subtraction techniques and validated its robustness in segmenting six soybean diseases.

From the discussion of machine learning methods above, these methods, while simple to apply, require large amounts of training data and heavily rely on manual expertise. Moreover, these techniques are not robust to the wide variability in the size, color and shape of plant diseases. Therefore, newer methods are needed to improve the accuracy of identifying multiple plant leaf diseases.

Deep learning approaches in plant disease

To overcome these limitations of machine learning methods in agriculture, deep learning has been gradually applied in this field  (Fuentes et al., 2017). Shougang et al. (2020) proposed a deconvolution-guided VGG network (DGVGGNet) model to address issues such as shadows, occlusions and varying light intensities in plant leaf disease recognition. This model uses VGGNet for disease classification training, recovers classification results as feature maps using deconvolution and combines upsampling and convolution operations to achieve deconvolution. Finally, it performs binary classification training for each pixel with minimal disease patch supervision, guiding the encoder to focus on true disease areas. Experimental results showed that this model performed exceptionally well in complex environments, with a disease type recognition accuracy of 99.19%, pixel accuracy for lesion segmentation of 94.66% and an average intersection over union (IoU) score of 75.36%. Luo et al. (2023) utilized FPGA to accelerate convolutional neural networks (CNNs) for plant disease recognition, addressing memory and computational constraints on edge devices. They designed a compact seven-layer network, “LiteCNN”, with 176K parameters and 78.47M FLOPs. After optimization and validation on the ZYNQ Z7-Lite 7020 FPGA board, the model achieved an accuracy of 95.71%, showcasing a low-power, high-accuracy and fast solution suitable for real-time plant disease detection. Nain, Mittal & Hanmandlu (2024) explored various color space models to enhance plant disease detection, finding that the HSL-CNN joint model achieved the highest accuracy of 98.97%, precision of 98.61% and F1-score of 99.28%. Verma, Kumar & Singh (2023) proposed a single lightweight CNN model for detecting diseases in maize, rice and wheat, achieving accuracies of 99.74%, 82.67% and 97.5%, respectively. This model outperformed multiple benchmark CNNs with only 387,340 parameters, making it suitable for real-time use in resource-constrained environments. Peyal et al. (2023) developed a lightweight 2D CNN architecture for early detection of diseases in cotton and tomato crops, achieving an average accuracy of 97.36%, outperforming pre-trained models such as VGG16, VGG19 and InceptionV3. This model was implemented in the “Plant Disease Classifier” Android application for efficient, smartphone-assisted plant disease diagnosis. Sharma & Sharma (2024) introduced a semi-supervised and ensemble learning framework to improve plant disease diagnosis, addressing issues like dataset scarcity and annotation costs. The framework achieved 18.03% higher accuracy and a 15% boost in F-score, with a 13.35% improvement in mean average precision, offering practical benefits for sustainable agriculture. Kaya & Ercan (2023) introduced a new deep learning-based plant disease detection method by integrating RGB and segmented images for disease identification. The method employed a multi-head DenseNet architecture and was evaluated on the PlantVillage public dataset, which contains 38 classes and a total of 54,183 images. The model achieved an average accuracy of 98.17%, recall of 98.17%, precision of 98.16% and F1-score of 98.12% in five-fold cross-validation, demonstrating high robustness and low standard deviation, indicating effective differentiation of plant diseases with varying features. Ulutaş & Veysel (2023) developed an integrated deep learning model for early detection of nine tomato leaf diseases, achieving a high classification accuracy of 99.60%. Their approach combined two newly proposed CNN models with well-known architectures, such as MobileNetV3Small and EfficientNetV2L, using particle swarm optimization and grid search for fine-tuning and hyperparameter optimization. This research highlights the efficiency and speed of these models in diagnosing plant diseases, providing valuable tools for preventing infection spread. Guowei et al. (2024) introduced DFN-PSAN, a deep feature fusion network for plant disease classification in natural environments. By combining YOLOv5 with pyramidal squeezed attention (PSA), the model enhances key image regions and improves accuracy. Tested on three datasets, DFN-PSAN achieved over 95.27% accuracy, with PSA reducing parameters by 26%. t-SNE and SHAP were used to improve model interpretability. Dubey & Dilip (2024) developed a machine learning-based method for plant disease detection. Key features are extracted from pre-processed images and an optimized artificial neural network (ANN), enhanced by the adaptive sunflower optimization (ASFO) algorithm, classifies diseases. The model achieved 97.94% accuracy, demonstrating its effectiveness for early plant disease diagnosis. Chen et al. (2020) introduced a novel deep learning architecture, INC-VGGN, for plant disease image recognition, with a validation accuracy of 91.83% on a public dataset. Even under complex background conditions, the model achieved an average accuracy of 92.00% for rice disease image classification. Argüeso et al. (2020) proposed a Few-Shot Learning (FSL) approach for plant leaf classification, addressing classification issues in small datasets. This study used the Inception V3 network for fine-tuning the source domain and transferred the learned knowledge to the target domain, achieving nearly 90% accuracy with very few training samples using Siamese networks and Triplet loss. Experimental results demonstrated that this FSL method significantly improved classification performance on small datasets compared to traditional fine-tuning transfer learning methods. Loddo, Loddo & Di Ruberto (2021) proposed SeedNet, a novel convolutional neural network for seed classification and retrieval tasks. Through deep learning methods, they achieved satisfactory performance in seed retrieval tasks, providing strong support for developing comprehensive seed recognition, classification and retrieval systems. Peng & Wang (2022) introduced a novel image retrieval system for automatic detection, localization and identification of plant leaf diseases. By optimizing the YOLOv5 algorithm, this system improved recognition capabilities for small objects and combined classification with metric learning to flexibly identify new disease types without retraining. Detailed experiments on three public leaf disease datasets demonstrated the system’s effectiveness in quickly determining disease types.

The analysis of existing crop disease identification, classification and retrieval methods is summarized in Table 1. From Table 1, it is evident that deep learning still has gaps in the field of multi-plant disease retrieval.

Current status and limitation

Despite the advancements in plant disease identification, classification and retrieval, as discussed above, there is still a notable gap in the development of models specifically designed for multi-plant disease retrieval. While models have been successfully developed for the recognition and classification of multiple plant diseases, as well as for the retrieval of plant seeds and individual plant diseases, the field lacks robust solutions for multi-plant disease retrieval. This gap highlights the need for a comprehensive approach that can handle the complexity and variability of diseases across different plant species. To address this, our study introduces a novel multi-plant disease retrieval method based on hash learning, which aims to bridge this gap and provide an efficient solution for large-scale plant disease retrieval.

Deep feature extraction

The training of a CNN requires a large number of samples. Moreover, annotating unknown specimens is costly and time-consuming. In order to deal with these obstacles, a pre-trained CNN model for reducing requirements for extensive training samples is introduced in an instance.

In this study, the VGG11 (Simonyan & Zisserman, 2014) model was chosen as the backbone network for the deep feature extraction part, primarily due to its simplicity and strong generalization capabilities. The VGG network, known for its straightforward architecture with eight convolutional layers and three fully connected layers, provides a good balance between depth and computational efficiency, making it highly suitable for a wide range of image retrieval and classification tasks. Moreover, the model’s architecture, characterized by the simplicity of its convolutional layers and fully connected layers. The consistent structure of VGG11 facilitates transfer learning, allowing the model to adapt well to different datasets with minimal fine-tuning. Additionally, its use of smaller convolutional filters (3x3) helps in capturing fine details in the images, which is essential for distinguishing between similar plant diseases.

The specific process is illustrated in Fig. 2. The “maxpool” operation characterizes the size of max pooling, “4096” represents the feature dimension for the fully connected layers and the dropout technique is applied to the fully connected layers 9 and 10. The activation function for all weight layers is the rectified linear unit (ReLU) (Krizhevsky, Sutskever & Hinton, 2017).

It is supposed that there are N training samples, denoting as (1a)${\displaystyle X={\left\{x_i\right\}}_{i=1}^N.}$

The corresponding set of labels can be represented as (1b)${\displaystyle Y={\left\{y_i\right\}}_{i=1}^N}$ where y_i ∈ ℝ^C is the ground-truth vector of sample x_i with only one element of 1 and others are 0, C is the total number of image scene classes. For arbitrary image X_i ∈ X , we can extract its deep features (i.e.,the output of the fully connected layer in Fig. 2.), giving by (1c)${\displaystyle f_i=\Phi \left(x_i;\theta \right),i=1,2,\dots ,N}$ in which Φ is the network function characterizing by θ which denotes all the parameters of the VGG.

Hash-based metric learning and loss function

Paired inputs are utilized by DHCNN for training the network and the process will be illustrated as that of Fig. 2.

Specifically, $\left(x_i,x_j\right)$ represents a pair of images and symbol S_ij is defined for it. Usually, S_ij is 1 if x_i and x_j are from the same class. Otherwise, it is 0. deep features $\left(f_i,f_j\right)$ are obtained by forward propagation in this instance. Subsequently, a hash layer will be connected after the fully connected layer in this model, as shown in Figs. 2 and 3A. The high-dimensional feature vector from the fully connected layer is transformed into hash-like features through a linear transformation and these hash-like features are then mapped to the hash space using the sign function, so that the high-dimensional deep features can be converted into compact K-bit hash codes, which will be followed that (Song, Li & Benediktsson, 2021) (2)${\displaystyle u_t=W_h{f}_t+V_h}$

where u_t is the class hash feature, W_h ∈ ℝ^K×4096 denotes the weight matrix, V_h ∈ ℝ^K×1 represents the bias vector, f_t ∈ ℝ⁴⁰⁹⁶ represents the high-dimensional feature vector obtained after processing through the deep convolutional neural network.

The function $\text{sign}\left(x\right)$ is a sign function used to convert real numbers into binary hash codes. Specifically: (3)${\displaystyle \text{sign}\left(x\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}x\ge 0\hfill \\ -1,\hfill & \mathrm{if}x<0\hfill \end{array}\right.}$ (4)${\displaystyle b_t=\text{sign}\left(u_t\right),t=i,j.}$

In the case where the hash code $B={\left\{b_t\right\}}_{t=1}^N$ is known. The likelihood function of paired labels $S=\left\{s_{ij}\right\}$ can be defined by Song, Li & Benediktsson (2021) (5)${\displaystyle p\left(s_{ij}\mid \mathbf{B}\right)=\left\{\begin{array}{cc}\hfill \phi \left({\omega }_{ij}\right),\hfill & \hfill s_{ij}=1\hfill \\ \hfill 1-\phi \left({\omega }_{ij}\right),\hfill & \hfill s_{ij}=0\hfill \end{array}\right.}$

in which φ(⋅) is the logistic function, $\phi \left(x\right)=\frac{1}{1+e^{-x}}$ and ${\omega }_{ij}=\frac{1}{2}{b}_i^T{b}_j$ . According to above definitions, the loss function is derived by taking the negative log-likelihood of the observed pairs of labels in S. Discretely reformulating the loss function gets (Song, Li & Benediktsson, 2021): (6)${\displaystyle {\mathcal{L}}_1=-\sum _{s_{ij}\in S}\left(s_{ij}{\psi }_{ij}-log\left(1+e^{{\psi }_{ij}}\right)\right)+\beta \sum _{i=1}^N{∥{\mathbf{u}}_i-{\mathbf{b}}_i∥}_2^2}$

where ${\psi }_{ij}=\frac{1}{2}{u}_i^T{u}_j,i,j=1,2,\dots ,N$ and β is the regularization parameter that constrains u_i to converge to b_i . By minimizing ${\mathcal{L}}_1$, the Hamming null distance between similar samples can be optimized to the minimum. Conversely, the Hamming distance between different samples is maximized.

Unlike image retrieval methods based solely on the similarity information between images to learn hash codes, DHCNN incorporates semantic information from each image to enhance the feature representation capability (Li et al., 2018). The specific process will be demonstrated in Fig. 3B. In detail, a fully connected layer with a softmax function after the hashing layer will be inserted for generating a category distribution for each image. Additionally, the cross-entropy loss is used to minimize the error between the predicted labels and the ground truth labels, following that De Boer et al. (2005) (7)${\displaystyle {\mathcal{L}}_2=-\frac{1}{N}\sum _{i=1}^N〈{\mathbf{y}}_i,log\left(\text{softmax}\left({\mathbf{W}}_s{\mathbf{u}}_k+{\mathbf{v}}_s\right)\right)〉}$ where W_s ∈ ℝ^C×K and v_s ∈ ℝ^C×1 denote the weight matrix and bias vector, respectively. Minimizing the loss function ${\mathcal{L}}_2$, the CNN can be adapted to learn the discriminative semantic features of each image.

As mentioned as above, ${\mathcal{L}}_1$ can be utilized to acquire the similarity information between images and ${\mathcal{L}}_2$ is intended to learn the label information of each image. To this end, a new loss function is defined for improving the network performance by considering both similarity information and label information, yielding (8)${\displaystyle {\mathcal{L}}_3=\eta {\mathcal{L}}_1+\left(1-\eta \right){\mathcal{L}}_2}$ where η ∈ [0, 1] is the regularization parameter that balances the label information and similarity information. The label information of each image is only considered if η is 0 and the similarity information between images is required if η is equal to 1. Finally, the objective function is the minimum loss function ${\mathcal{L}}_3$ , which is given by (9)${\displaystyle \mathcal{J}=min{\mathcal{L}}_3=min\left\{\eta {\mathcal{L}}_1+\left(1-\eta \right){\mathcal{L}}_2\right\}.}$

Solving the problem in Eq. (7), the stochastic gradient descent (SGD) algorithm is also considered to learn the parameters.

Completing DHCNN training, the hash codes and class labels of all samples from the database are tried to obtain. For any image x_q , its hash code b_q and class label c_q can be determined by (10)${\displaystyle {\mathbf{b}}_q=\mathrm{sgn}\left({\mathbf{u}}_q\right)=\mathrm{sgn}\left({\mathbf{W}}_h{\mathbf{f}}_q+{\mathbf{v}}_h\right)}$ (11)${\displaystyle c_q=\underset{k=1,2,\dots ,C}{\mathrm{argmax}}{\mathbf{t}}_i^k}$

where ${\mathbf{t}}_i^k$ is the kth component of vector t_i. Eventually, a query image is the input. The DHCNN model can quickly and accurately output a set of similar images after sorting the Hamming distance between the query image and the images in the database as well as the class distribution. The retrieval process is shown in Fig. 4.

Datasets and data augmentation

For retrieval training, a publicly available dataset of plant disease images called Plant-Village (Geetharamani & Pandian, 2019) is selected. This dataset has 39 different categories of plant leaf and background images, generally including 38 categories of plant leaf disease and totaling 54,305 images. There is also a separate category containing 1,143 background images.

Various image augmentation techniques were employed to increase the number of images in the dataset and avoid overfitting. These techniques include image flipping, gamma correction, noise injection, PCA color enhancement, rotation and scale transformation. The strategy adopted for data augmentation was to augment the categories with fewer than 1,000 images to 1,000 images, while leaving the categories with 1,000 or more images unchanged. Subsequently, 1,000 images were selected from each augmented category. For the training and testing phases, a distribution percentage of 80% for training and 20% for testing was adopted. This 80/20 split was chosen based on standard practices in the field, which balance the need for a robust training set with the necessity of having a representative test set to evaluate model performance accurately. The choice ensures that the model can learn effectively from a sufficient number of samples while still being tested on a diverse subset of data. Additionally, multiple trials with different random splits were performed to verify the stability and reliability of the results. The final dataset consists of 7,600 images in the test set and 30,400 images in the training set. Detailed parameters of the dataset are shown in Table 2.

Evaluation metrics

In experiments, five metrics are used to evaluate the performance of retrieval methods. Of those, precision measures the accuracy of positive predictions. It is defined as the ratio of true positives to the sum of true positives and false positives, yielding (12)${\displaystyle \text{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}.}$

Recall, also known as sensitivity or true positive rate (TPR), is the ratio of true positives (TP) to the sum of TP and false negatives (FN). It measures the model’s ability to correctly identify all positive instances, following that (13)${\displaystyle \mathrm{TPR}=\text{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}.}$ False negative rate (FNR) is the ratio of false negatives (FN) to the sum of true positives (TP) and FN. It measures the proportion of actual positive instances that are incorrectly identified as negative by the model. (14)${\displaystyle \mathrm{FNR}=1-\mathrm{TPR}=\frac{\mathrm{FN}}{\mathrm{TP}+\mathrm{FN}}.}$

Mean average precision (MAP) is the mean of the average precision scores for each class in a multi-class classification problem. It evaluates the precision of the model across different recall levels, which is given by (15)${\displaystyle \mathrm{MAP}=\frac{1}{N}\sum _{i=1}^N{\mathrm{AP}}_i}$

where AP_i is the average precision (AP) for class i and N is the number of classes.

The F-score is the harmonic mean of precision and recall. It provides a balance between Precision and Recall, particularly when the class distribution is imbalanced. (16)${\displaystyle F\text{-score}=\frac{2\times \text{Precision}\times \text{Recall}}{\text{Precision}+\text{Recall}}.}$

Area under the curve (AUC) is the area under the receiver operating characteristic (ROC) curve, which plots the TPR against the FPR. It measures the overall ability of the model to distinguish between positive and negative classes. (17)${\displaystyle \mathrm{AUC}={\int }_0^1\mathrm{TPR}\left(x\right)dx}$

where TPR(x) is the TPR as a function of the FPR.

Parameter setting

Adopting the Plant-Village dataset, the DHCNN model is trained and optimized by adjusting different hyperparameters for obtaining best results. Tables 3 and 4 shows the structure and optimized hyperparameters for the proposed DHCNN model.

The epoch is a hyperparameter that defines a single pass through the complete training set during the training of a deep learning model. It refers to the number of times the entire training dataset passes forward and backward through the neural network. In this study, the epoch was set to 200. During experimentation, it was observed that the model’s results began to stabilize around epoch 40. However, the epoch was set to 200 to ensure that the model had ample opportunity to learn from the data and to avoid premature convergence.

The batch size is another crucial hyperparameter that affects computation efficiency, memory usage, gradient estimation accuracy and training stability. In this study, batch sizes of eight, 16, 32, 64, and 128 were experimented with. After thorough investigation, the batch size of 64 was found to provide the best balance between training speed and model performance.

The learning rate is a hyperparameter that controls the step size of the model’s parameter updates during each iteration. It significantly influences both the training speed and the final performance of the model. For this study, an initial learning rate of 0.1 was chosen. To fine-tune the training process, the learning rate was scheduled to decrease by a factor of 0.1 after every 30 epochs. This decay strategy helped maintain the stability of the training process, ensuring that the model converges effectively without overshooting the optimal solution.

Additionally, the hash code length, a critical parameter for the deep hash CNN model, was varied among 16, 32 and 64. The model achieved optimal performance with a hash code length of 64, balancing between retrieval accuracy and computational efficiency.

Search results for single plant disease

The individual retrieval results for five plant types apple, corn, grape, potato and tomato are presented in Table 5.

The convergence of five retrieval models starts at epoch of 50. Besides corn, the accuracy of all models is over 99%. The retrieval of apple plants converges quickly because of distinct differences in features of various disease images and the accuracy approaches 1, which presents the 10% improvement compared to the results achieved by Karthikeyan & Raja (2023). However, corn leaves’ retrieval accuracy is slightly lower than other plants since the characteristic images of corn gray leaf spot and corn rust. Also, northern corn leaf blight are highly similar. Nevertheless, the accuracy still reached 0.984. Owing to the large number of disease types, the retrieval results for tomato plants are relatively lower. However, the average retrieval accuracy still reaches 0.994, which presents the 40% improvement compared to the results achieved by Baquero et al. (2014). The average retrieval accuracy for grape and potato plants reaches 0.997 and 0.995 respectively, which represents the 36% and 46% improvement relative to the results obtained by Piao et al. (2017) and Patil & Kumar (2013). Additionally, the retrieval results for the grape dataset are similar to those obtained by Karthikeyan & Raja (2023).

For establishing models for multiple plants and diseases in practice with high accuracy, the retrieval system results for disease identification in the leaves of multiple plants will be introduced in next sections.

Search results for multiple plant disease

In this section, 38 plant diseases from the Plant-Village dataset will be applied to retrieval experiments for the multi-plant disease. The DHCNN model is adopted to calculate $\mathcal{J}$ loss for validating performance of the model. The training loss values varied with different Epoch are shown in Fig. 5.

In the Epoch-Loss curve, the loss value gradually decreases if the Epoch is increased from 1 to Epoch 40. Such a phenomenon indicates that the model is gradually learning the patterns and features in the dataset. The loss value tends to be stable if the Epoch is larger than 40 and increases gradually. At this point, the model has converged and the training of the retrieval model is completed.

As is shown in Fig. 6, the precision is still higher than 0.995 if the k is small than 800. Here, k denotes the number of top-ranked images retrieved and evaluated. This indicates that the system proposed here with the retrieval ability of extremely high precision and stability. Following the increase of k, the precision will be decreased if it is larger than 800 since most relevant images in the dataset have already been retrieved.

The recall-k curve is unaffected by imbalanced datasets and effectively reflects the changes of true positive rate and false positive rate of retrieval model at different thresholds, showing as Fig. 7. Clearly, as k is increased from 1 to 800, the recall steadily rises and approaches 1. This trend signifies the retrieval system’s effectiveness in identifying more positive samples while keeping false negatives to a minimum. However, when k exceeds 800, the recall value stabilizes. This indicates that beyond this point, the relevant images in the retrieval dataset have already been effectively retrieved, and further increases in k do not significantly impact the recall value.

In order to validate the retrieval performance, a so-called recall-precision curve which characterizes a relationship between recall and precision under different thresholds is shown in Fig. 8. Evidently, the precision on the horizontal axis and recall on the vertical axis illustrates the precision and recall values at different k values. The optimal balance point is found in the top-right corner where both precision and recall tend to be 1. Such a point corresponds to a k value of 800 in the precision-k and recall-k curves. At this point, the model presents the best overall retrieval performance, thus retrieving as many relevant images as possible and minimizing false positives.

To evaluate the retrieval performance, Figure 9 presents the ROC curve, showing the relationship between the TPR (recall) and the FPR at various thresholds. The y-axis represents the TPR, while the x-axis shows the FPR. With an AUC of 0.997, the curve highlights the model’s high retrieval accuracy, effectively capturing relevant results while minimizing false positives. The curve’s near approach to the top-left corner reflects a well-optimized balance between true positive retrieval and reducing false positives across different thresholds.

The variation of MAP with Epoch will be explained in Fig. 10. The MAP gradually increases to 1 as long as the Epoch is also enlarged from 1 to 40. Such a variable behavior indicates that the retrieval performance of the proposed model is also improved gradually and more positive samples with minimal false negatives can be retrieved effectively. The MAP value will be stable with an increase of Epoch if it is larger than 40, which indicates that the model training is essentially completed and the retrieval performance is also stable.

Error analysis

In addition to the overall performance of the DHCNN model, some variations in retrieval accuracy were observed across different plant species datasets. Notably, in the single-species retrieval experiments discussed in ‘Search Results for Single Plant Disease’, DHCNN achieved an exceptionally high mean average precision of approximately 100% for the apple leaf disease dataset, outperforming its performance on the multi-species dataset. This superior result can be attributed to the distinct differences between the diseases in the apple dataset, which made it easier for the model to distinguish between classes, leading to near-perfect accuracy. In contrast, the corn leaf disease dataset posed more challenges, with DHCNN achieving a MAP of 98.4%, lower than its performance on the multi-species dataset. This can be explained by the significant intra-class similarities among the corn disease images, where certain disease symptoms are visually alike, resulting in a slight drop in retrieval accuracy. Similarly, the tomato leaf disease dataset demonstrated slightly lower retrieval accuracy, with a MAP of 99.4%, below the average. This lower performance can be attributed to the large number of disease categories within the tomato dataset. Diseases affecting the same species tend to have visual similarities, making it more challenging to differentiate between them compared to diseases across different plant species. This intra-class similarity, coupled with a greater number of disease categories, contributed to the reduced MAP for tomato disease retrieval. On the other hand, the retrieval results for diseases affecting grapes, potatoes and other plants were consistent with the performance on the multi-species dataset, showing no significant performance drops. These variations highlight the model’s sensitivity to different plant disease datasets. In the following sections, the performance of the DHCNN model will be evaluated from various aspects to provide a more comprehensive assessment of its overall effectiveness.

Ablation study

To evaluate the impact of the hash layer and different combinations of loss functions on the model’s performance, a series of ablation experiments were conducted. The results are summarized in Table 6. These experiments focus on different backbone networks, with or without a hash layer, examining how varying hash code lengths and loss functions affect precision, recall (TPR), and F-score.

First, for the VGG-11 model, it is observed that performance improves consistently as the hash code length increases. When the hash code length is set to 64, using the ${\mathcal{L}}_3$ loss function (a weighted combination of cross-entropy loss and regularization loss), the model achieves the highest F-score of 0.997. This indicates that a longer hash code captures more discriminative information, thereby improving the model’s retrieval performance.

However, when using only the ${\mathcal{L}}_1$ loss (regularization loss), despite having a 64-bit hash code, the model’s performance drops significantly, with an F-score of only 0.129. This suggests that relying solely on regularization loss, which primarily optimizes Hamming distances between samples, is insufficient for effectively distinguishing between different categories, leading to poor retrieval results.

In contrast, when using the ${\mathcal{L}}_2$ loss (cross-entropy loss), the model’s performance improves significantly, achieving an F-score of 0.932. This shows the strength of cross-entropy loss in optimizing label prediction; however, it does not fully exploit the potential of the hash layer, indicating room for further improvement.

Notably, when the hash layer is removed, the F-score of VGG-11 drops to 0.790, confirming the necessity of the hash layer for enhancing retrieval performance.

Additionally, comparative experiments with AlexNet and ResNet18 as different backbone networks further validate the effectiveness of the ${\mathcal{L}}_3$ loss function. With the hash layer included, both AlexNet and ResNet18 achieve high performance, with F-scores of 0.993 and 0.997, respectively.

Overall, the experimental results demonstrate that the combination of the hash layer with the ${\mathcal{L}}_3$ loss function significantly enhances the model’s retrieval capability. In the next step, the performance of the proposed model will be further analyzed in comparison to other mainstream models across different datasets to validate its generalization ability.

Performance evaluation

The performance of two networks, AlexNet and VGG, fused with DHCNN will be compared. Both AlexNet and VGG are well-established models in deep convolutional neural networks. AlexNet offers several advantages, including a simpler network structure, fewer parameters and faster training speed, making it more suitable for straightforward image recognition tasks. On the other hand, VGG, with its deeper network structure, increased parameters, stronger feature representation capabilities and better transferability, is more adept at handling complex image recognition problems, despite the higher computational complexity it brings. To assess the retrieval performance of models using different backbone networks, AlexNet and VGG are employed as the backbone networks. Furthermore, three different hash code lengths, namely 16 bits, 32 bits and 64 bits, are selected to evaluate the experimental effects influenced by them. The MAP values for retrieval using different hash code lengths under the AlexNet and VGG backbone networks are presented in Table 7. The best retrieval performance is attained with the combination of the VGG backbone network and 64-bit hash codes, resulting in an average retrieval accuracy of 0.9974. When compared to the results of models proposed by Patil & Kumar (2017) and Piao et al. (2017), it is evident that the retrieval accuracy achieved in this study is significantly higher, even when dealing with more complex plant and disease categories, underscoring the effectiveness of the approach.

Finally, the retrieval precision, recall and F-score of the DHCNN model were comprehensively compared and analyzed with the latest methods in Table 8 (Karthikeyan & Raja, 2023; Patil & Kumar, 2013; Hussein, Mashohor & Saripan, 2011; Jadoon et al., 2017; Lu et al., 2021). The experimental results indicate that the PLDIR-CM and PSO-LIR models have lower values of precision, recall and F-scores. In contrast, the precision, recall and F-scores of the PNN, SVM and KNN models showed slight improvements. Additionally, the DWT-CBIR, CBIR-CSTF and DTLDN-CBIRA models achieved reasonable values for precision, recall and F-scores. However, the DHCNN technique outperformed the other methods, with the best precision, recall and F-scores being 99.5%, 99.7% and 99.59%, even when dealing with more complex plant and disease categories.

In order to mitigate the influence of a single dataset on the retrieval results of the DHCNN model, PlantDoc and the Plant Disease dataset have been additionally introduced. PlantDoc (Singh et al., 2020) is a dataset for visual plant disease detection. The dataset contains 2,598 data points in total across 13 plant species and up to 17 classes of diseases. Plant disease dataset (Chouhan et al., 2019): 12 economically and environmentally beneficial plants have been selected for this dataset. This dataset consists of 4,503 images, including 2,278 images of healthy leaves and 2,225 images of diseased leaves. In this study, the processing approach for the PlantDoc dataset and the Plant Disease dataset is consistent with the method described in ‘Proposed Methodology’ of this paper. The specific experimental results can be found in Table 9. It is evident that the DHCNN performs consistently well across different datasets.

As depicted in Fig. 11, the AP values under different hash code lengths and backbone networks consistently maintain high precision as k varies from 1 to 800, showcasing a robust and stable retrieval capability. However, the AP results decrease when k exceeds 800. This suggests that most of the relevant images in the retrieval dataset have already been retrieved. Furthermore, the retrieval performance of the model with the VGG backbone network surpasses that of the model with AlexNet. Additionally, increasing the number of hash bits gradually enhances the retrieval performance of the model. This demonstrates the effectiveness of utilizing a deeper network structure and a higher number of hash bits in improving the retrieval accuracy.