A new era in identification of tick genera; artificial intelligence for precision and speed

Motivation

Ticks across various region of the world serve as vectors for numerous pathogens such as bacteria, viruses, protozoa etc. Different species of ticks serve as reservoirs of bacteria which can be transmitted transstadially and transovarially. Among different types of species, the most common species found in Cyprus includes Rhipicephalus and Hyalomma (Tsatsaris et al., 2016). Accurate discrimination of these tick species is crucial for public health, veterinary medicine, and ecological research as they help in risk disease assessment, proper surveillance and emerging threats, public awareness and prevention. Manual classification of tick species is prone to miss-classification or miss-identification and time consuming (Justen et al., 2021; Akbarian et al., 2021). Therefore, there is need for developing automated system that can distinguish between different species.

Despite plethora of studies in the literature on the application of DL or computer-aided detection (CAD) for image classifications, only few studies have attempted to discriminate between Hyalomma and Rhipicephalus. Moreover, majority of the studies only implemented single model, pre-trained models and limited number of datasets. Moreover, there is scarcity of AI/IoT-powered platform for real-time classification of tick species and non-tick images. Consequently, majority of studies only reported models implemented in a computer, which is not freely accessible by patients and healthcare experts. In order to address these issues, this study proposed a framework known as I-TickNet, an AI/IoT-powered platform designed using custom shallow CNN (6-layer), and pre-trained models (Visual Geometry Group 16 (VGG16) and Residual Network 50 (ResNet-50)) for accurate discrimination of two tick species and the subsequent deployment of the models on to a web app for real-time detection. Both customized model and pre-trained models are deployed in order to established the best performing model.

Literature survey

The study by Luo et al. (2022) applied AI-based techniques to classify ticks. The study acquired image dataset from a tick passive surveillance program of the most encountered human-biting ticks, which include Dermacentor variabilis, Ixodes scapularis and Amblyomma americanum. The dataset comprises of 12,000 high-resolution micrographs molecularly confirmed by a genera-specific TaqMan polymerase chain reaction (PCR) assay. The dataset is split into 90% for training (10,800) and validation and 10% for testing (1,200). In order to increase the diversity of training set, data augmentation techniques were implemented via zooming and random rotation. The images are trained and tested using five pre-trained CNN, which include MobileNet V2, Inception v3, DenseNet121, ResNet50 and VGG16. The evaluation of the models and comparative analysis has shown that the Inception V3 model achieved the best result in terms of accuracy with 99.5%.

Xu et al. (2021) developed smartphone app known as TickPhone app for rapid tick identification of three types, which include Amblyomma americanum (line star tick), Ixodes scapularis (deer tick) and Dermacentor variabilis (dog tick). The experimental procedure is set up based on several steps which include collection of image data, optimization of DL models, training and testing and development of smartphone app. The study acquired dataset from the Connecticut Veterinary Medical Diagnostic Laboratory and photos taken using smartphones. In order to enlarge that dataset, the study conducted data augmentation via flipping, shifting, zooming, shearing and rotation which resulted in more than 2,000 images. In order to identify ticks, modified and optimized LeNet is used to extract features followed by classification using Support Vector Machine (SVM). The optimized DL model achieved 85% validation accuracy. Moreover, the study developed TickPhone app and the testing of the app has shown that it can identify 31 tick genera resulting in 95.69% accuracy.

In the study reported by Omodior et al. (2021), the authors proposed the use of CNN for the detection of ticks. The overall methodology revolves around data curation, development of customize CNN, implementation of CNN and pre-trained networks and performance evaluation. The study developed a dataset which comprises 2,130 images of four tick genera, which include Haemaphysalis sp, Dermacentor variabilis, Ixodes scapularis and Amblyomma americanum. The images are further partition into three subset which include 70% for training 20% for validation and 10% for testing. The images are trained and tested using shallow custom-built CNN and pre-trained ResNet50. The comparison between shallow custom built CNN and pre-trained ResNet50 has shown that customized CNN achieved greater accuracy of 80% using subset of unseen dataset.

The development of Inception V3 (TickIDNet) which is DL-based approach for the ternary classification of ticks is reported by Justen et al. (2021). The first step taken in order to achieve accurate classification revolves around the generation of image dataset which comprises of 12,177 images of three common ticks found in the US (Ixodes scapularis, Dermacentor variabilis and Amblyomma americanum). The images are subsequently processed based on quality and 1,273 are discarded while 10,900 images are kept. The processed images are further split into 70% for training (7,625), 20% for validation (2,181) and 10% for testing (1,094). In order to maximize the dataset, several data augmentation techniques were adopted which include rotation, cropping, and flipping which led to the generation of 95, 119 images from the initial 7,625 images. Evaluation of the TickIDNet on two testing set resulted in 87.8% accuracy, 87.73% weighted F1-score, and 0.7895 Kappa agreement score am using a user generated test set and also 91.67% accuracy, 91.55% weighted F1-score, and 0.875 Kappa agreement score using a laboratory test set (300 images with 100 images per each class).

The real-time automated detection or identification of tick genera (binary classification between black-legged (Ixodes scapularis) and over six other non-black-legged genera) using computer vision technique based on CNN is reported by Akbarian et al. (2021). The research is designed based on several steps, including image data collection, pre-processing, training and testing. The first step involves data collection from Public Health Ontario, which comprises of 6,294 images of black-legged (Ixodes scapularis) (41%) and over six other non-black-legged genera (59%). Moreover, the images are split into training and testing. The training set undergoes data augmentation via zooming, flipping and random rotation. The acquired images were trained and tested using 7-layer customized CNN and Inception-ResNet CNN. The last step includes the development of web for real-time identification. Evaluations d comparison between untrained and pre-trained model has shown that InceptionResNet achieved 92.04% accuracy.

The summary of the related work is presented in Table 1.

Code repository

Access to a public repo containing all the links for all the dataset and code used in this study can be found at https://github.com/ameibrahim/ITICK.

Data collection and processing

A comprehensive study of Hyalomma and Rhipicephalus (n = 35) ticks’ genera were participated by an experienced acarologist. 5–7 adult ticks were collected from six different dogs located in Northern Cyprus. The brand of the microscope use is Bresser GmbH (Germany) Biolux Touch (digital LCD microscope) with 4×, 10×, and 40× magnifications. It has integrated digital camera allows users to take photos and record videos directly onto a memory card. Idiosomic (body) ventral and dorsal views and capitulum (head) ventral and dorsal views including the palps and hypostome were all captured.

As a result of this identification process, images were taken from different angles for each tick sample, leading to a total of 9,556 images (4,778) for each class, in order to ensure the two classes are balanced. While for the discrimination of tick and non-tick, 6,972 images are used. Both Hyalomma and Rhipicephalus are grouped in one class (i.e., tick with 3,486 images). While non-ticks comprise of a combination of arachnids and other insects, spiders, beetles, mites, mosquitos and scorpions (3,486 total images).

The dataset undergoes several processing steps which include labelling, cleaning, restructuring etc. The process images are further trained and tested using a customized shallow 5-layer CNN, VGG-16 and ResNet-50. The next step revolves around performance evaluation based on accuracy. The best performing model is selected for the development of IoT-based framework for accurate and real-time classification of ticks. The overall methodology is summarized in Fig. 1.

Description of dataset

Training ML and DL models with sufficient data is crucial for achieving high performance. The two ticks genera used in this study vary from one another. Hyalomma are characterized by their long mouthpart (i.e., longirostrate) and legs with pale rings. In terms of texture, there scutum or conscutum are colored brown. Rhipicephalus on the other hand are characterized by their hexagonal shape. In terms of texture, they vary from yellowish-brown to reddish-brown, with a dark inornate brown scutum (Geevarghese & Mishra, 2011) as shown in Fig. 2.

Data split and data augmentation

Partitioning of dataset for ML and DL application may vary due to several factors such as dataset size, complexity, and data availability. Over the last five decades, several data split ratios have been proposed, which include two class and three class split which include 60:40, 70:30, and 80:20 for training and testing as well as 60:20:20, 70:15:15 and 80:10:10 for training, validation and testing respectively. The datasets were split into a 60% ratio for training, 30% ratio for validation and 10% for testing. This yielded 4,181, 2,090 and 699 images respectively for the tick non tick dataset (i.e., dataset A) and 5,733, 2,866 and 958 images respectively for the tick classification (i.e., dataset B). Data split is presented in Table 2. During training, the images were augmented using Random Flips, Rotation, Zoom, Translation, Contrast and Brightness. The augmentation was carried out during training over 35 epochs (i.e., 4,778 newly augmented images are generated for every epoch).

Deep learning models

Customized CNN

Portions of this text were previously published as part of a thesis by the first author (Ibrahim, 2023).

We developed six-layer CNN known as basic model. The 6 layers in the network comprise of four convolutional layers and one fully connected layer designed with SoftMax for classification. Other layers such as the average pooling layer and flattening layers were not counted. The choice of 6-layer CNN as the base model is to check the efficiency of the network in classifying tick images. Compare to other deeper architectures, this model is very shallow as it doesn’t have sufficient number of layers in order to be able to understand underlying features. However, one of the advantages of the model is that is very small (i.e., 4.2 MB) which can be beneficial in memory constraints systems. Moreover, the model consumes the least amount of Random Access Memory (RAM) and disk space. The parameters and the number of layers of the customized CNN are shown in Fig. 3.

Model training

Training ML models requires having an environment that can facilitate efficient passage of data and utilizing enough memory so that the model can train quickly. Thus, all the three models implemented in this study were babysat. The models are trained based on gradient descent with the main objective revolving around finding the global minimum of the function in order to make learning more efficient and very low loss. Moreover, optimizers are used during training to make gradient descent run faster. Initially, both Stochastic Gradient Descent SGD and Adam optimizers are used interchangeably between models to establish the best approach. Evaluation of the two optimizers has shown that Adam optimizer performed faster and more efficient than SGD and therefore, it is selected as the default the optimizer. Equation (1) present Stochastic Gradient Descent and Adam Averaging Summations from Sums of the Loss Function:

(1) $$J\left(\theta \right)={\displaystyle \frac{1}{m}{\sum }_{i=1}^{m}L(y_i,f\left(x_{[},\theta \right).}$$

In order to train the models, the codes of setting up the environment and loading the data as well as training the model were split into cells and run independently within a Jupyter file. Jupyter files can handle chunks of code in a cell one at a time. Therefore, if one of the cells crashes, data from other cells can be retained. Thus, this feature helps in minimizing the time needed to run previous cells again. Moreover, the efficiency of the Jupyter kernels aid in training multiple models on a more powerful machine.

Three models are implemented in this study which includes untrained shallow CNN and two pre-trained models (VGG16 and ResNet50). The three models were trained using a server equipped with powerful resources. The models were trained using on a combination of Graphics Processing Unit (GPU) and Central Processing Unit (CPU), depending on what was free on the server, as it was training them in parallel. This helped reduce the time it took to train the models from days to just minutes. Paperspace server is used which significantly reduced the amount of training time. The server was a Linux machine running Ubuntu 22.04 with dual 16 GB GPU cores, 16 CPU cores and 96.6 GB RAM. The two cores were run separately to help train two models in parallel over two different Jupyter files under the same kernel. Several input sizes were evaluated which include 32 × 32, 64 × 64, 128 × 128 and 256 × 256 and 128 × 128 is selected as it exhibited optimum size for accuracy ratio. Moreover, the models were trained using 35 epochs. Transmit was employed to fetch the models after training to a personal workspace and subsequently tested locally over the frontend user interface.

Web development

Web application was created to complement the models implemented in this study. Predicting images through the Command Line Interface (CLI) can be extremely cumbersome considering the fact that command must be type in order to run each prediction. Thus, using an application that can communicate with models on the backend and produce a visual look of the results generated by the model equips users to predict images of different kinds through different models. These results are automatically saved for future review and comparisons.

The web application was built using an array of languages that weld together a coherent experience for discriminating between two tick species. The concept of the application is to use an Application Programming Interface (API) to communicate with the model backend and return results to the frontend. The model was built using Python and a popular framework known as TensorFlow. It’s only fitting that the API follows suit as well. All models were saved as keras files that ensured compatibility for calling the API. The API was built with Flask, a very popular tool for building sites and APIs using Python. It was served with a Web Server Gateway Interface (WSGI) framework named Waitress over a safe port.

The API lives on a different server from the frontend user interface. API calls are produced using a URL that is passed from the front end to the backend. On a successful result, the server returns a 200 status response with a valid result. The front end picks up the results and saves all the metadata into the database. This process varies with internet speeds as the image needs to move from the frontend server to the API server. The larger the image uploaded, the longer it will take. Therefore, the use of 128 × 128 input sizes made the whole process significantly smoother, and the results came back very quickly.

Performance evaluation

Tick vs non-ticks

The performance evaluation of models (customized CNN, ResNet50 and VGG16) trained and tested using dataset A (tick vs non-tick) produced excellent result across all metrics. Assessment of the models on the testing set has shown that ResNet50 attained the highest score across all metrics, resulting in 100%, accuracy, precision, Recall, F1-score, Specificity, AUC, Mathews Correlation Coefficient (MCC), Cohen’s Kappa (CK), balanced Accuracy and Jaccard Index (IoU). While both VGG16 and custom CNN achieved similar result across all metrics as summarized in Table 3. The testing accuracy graphs of the three models are presented in Fig. 4.

Table 3:

Performance evaluation of customized and pre-trained models.

Models	Round	Acc (%)	Prc (%)	Rec (%)	F1-S (%)	Spe (%)	AUC (%)	MCC (%)	CK (%)
Dataset A (Tick vs non-Tick)
VGG16	4	99.86	99.71	100	99.86	99.71	100	99.71	99.71
ResNet50	1	100	100	100	100	100	100	100	100
CNN	5	99.86	99.71	100	99.86	99.71	100	99.71	99.71
Dataset B (Tick identification)
VGG16	3	96.97	96.11	97.91	97.00	96.03	99.55	93.96	93.95
ResNet50	4	94.89	93.00	97.08	94.99	92.69	99.01	89.86	89.77
CNN	5	93.35	96.47	91.23	93.78	96.66	97.82	88.02	87.89

DOI: 10.7717/peerj-cs.3291/table-3

Note:

Acc, Accuracy; Prc, precision; Rec, recall; F1-S, F1-score; Sen, sensitivity; Spe, specificity; CK, Cohen’s Kappa.

Tick identification (Hyalomma and Rhipicephalus)

Evaluation of the models on test set of dataset B (two classes which include Hyalomma and Rhipicephalus) produced good result across all metrics. Contrary to dataset A, VGG16 attained the highest score across majority of the metrics with 96.97% accuracy, 97.00% F1-score and 99.55% AUC score. ResNet50 and custom CNN ranked second and third respectively across all metrics as summarized in Table 3. The testing accuracy graphs of the three models are presented in Fig. 5. Table 4 present one-way Analysis of Variance (ANOVA) p-values testing for testing differences between the three models across all five training rounds. Recall, F1-score and AUC show statistically significant differences among the three models, while precision and specificity do not differ significantly.

Table 4:

One-way ANOVA of three models deployed for tick identification.

Metrics	P-value	Significance ( $\alpha =$ 0.05)
Precision	0.3124	Not significant
Recall	0.0013	Significant
F1-score	0.0000	Significant
Specificity	0.3612	Not significant
AUC	0.0000	Significant

DOI: 10.7717/peerj-cs.3291/table-4

Confusion matrix

Ticks vs non-ticks

In order to test the models, 700 images (350 images of ticks and 350 images non ticks) are used. As shown in Fig. 6, customized CNN and VGG16 miss-labeled one image of non-tick as tick, while ResNet50 accurately labeled all images, resulting in 0 miss-labelling.

Ticks identification

In order to test the models, 958 images (479 Hyalomma and 479 Rhipicephalus images) are used. As shown in Fig. 7, customized CNN miss-labeled 16 images of Hyalomma as Rhipicephalus and 42 images of Rhipicephalus as Hyalomma. VGG16 on the other hand, miss-labeled 19 images of Hyalomma as Rhipicephalus and 10 images of Rhipicephalus as Hyalomma. While ResNet50 miss-labeled 35 images of Hyalomma as Rhipicephalus and 14 images of Rhipicephalus as Hyalomma.

ROC curve

The ROC curve signifies stable predictions for the model under different thresholds. Evaluation of the ROC curve of the models deployed for the discrimination between ticks and non-ticks has shown that all the models achieved perfect ROC score 100% as summarized in Fig. 8.

While, evaluation of the ROC curve of the models deployed for the discrimination between ticks (Hyalomma as Rhipicephalus) has shown that both VGG16 achieved perfect ROC score 100% compare to and ResNet50 (99%) and customized shallow CNN (98%) as summarized in Fig. 9.

Web development

In order to address our second objective, we developed a web app embedded with the three models for the development of DL/IoT based framework that is accessible to the public using the following link: https://tickprediction.aiiot.center/. By using the web app, users can upload images taken by smartphone or cameras and received result in less than a minute. The step-by-step process is illustrated in Fig. 10 and in a video submitted as a Supplemental File. The prediction speed of all the models is presented in Table 5.

Comparison with related work

Here, we conducted a realistic comparison of the proposed approach with existing studies. Thus, we compared the model + classifier that achieved the best result with reported studies in the literature. However, this comparison is not meant to diminish any study or to show that our study is better than the previous studies.

The study proposed by Justen et al. (2021) achieved 87.8% accuracy for three-way classification of ticks using TickIDNet. Despite the training the model with more than 90,000 images, all the three models proposed in this study achieved higher accuracy compare to TickIDNet. The study proposed by Omodior shared a lot of similarities with the current study based on the comparative analysis between customized and pre-trained models. The study achieved 80% mean prediction accuracy using small dataset in comparison with the six-layer base CNN implemented in this study which achieved 93.35%. The result achieved using customized model can be attributed to the large dataset use for training and testing (approximately 9,500).

The study proposed by Luo et al. (2022) achieved an optimum accuracy of 99.5% using Inception V3. The model is trained using augmented image. VGG16 as the best performing model in this study achieved an accuracy of 99.42% and 96.67%. Thus, the InceptionV3 deployed by Luo et al. (2022) slightly ranked higher (99.5%) compare to VGG16. The study proposed by Xu et al. (2021) implemented LeNet model which comprises of five layers. The model is trained using more than 2,000 images which result in 85% validation accuracy, which ranked lower with the base CNN (6-layers) implemented in this study. However, this can be attributed to the extensive pre-processing techniques and larger dataset. The study proposed by Akbarian et al. (2021) achieved 92.04% accuracy using InceptionResNet for the binary classification of black-legged (Ixodes scapularis) and non-black-legged ticks. However, all the three models deployed in this study achieved higher accuracy for binary classification compare with InceptionResNet. The summary of comparison with related work is presented in Table 6.

Benefits and risks of automated tick identification

Automated tick identification or discrimination between species using deep learning models like offers significant benefits such as early and rapid detection, improved public health surveillance, reduced costs etc. However, despite the advantages of AI-based tick identification, it also poses risks, especially when accuracy and other metrics are below 100%. Some of the risks of this approach include false negative and false positive which can lead to misclassification of harmful tick as harmless. Moreover, overreliance on AI-based approaches can lead to human complacency (users might skip lab confirmation for high-risk bites). Despite the growing literature on the application of AI-based techniques, there are still several issues that need to be addressed which include improving performance of AI-based models on nymphs/engorged ticks and validation in real-world settings. Therefore, future studies can be directed toward the development of hybrid systems that integrates AI-based techniques couple with human experts for validation of high-risk cases, the use of multimodal which support the combination of images and genomic data.

Clinical or practical implication

As a result of the increasing number of Crimean-Congo hemorrhagic fever (CCHF) and Mediterranean spotted fever caused by Hyalomma and Rhipicephalus, developing an AI/IoT-based identification system for tick genera can offer significant clinical and public health implications. AI/IoT-based identification system for tick genera can enhanced disease surveillance and outbreak prevention through early detection of pathogenic vectors, mapping high risk regions. Moreover, this platform supports faster and accurate identification of the two common tick genera by reducing misidentification. AI/IoT-powered platforms such as I-TickNet can be integrated into laboratory settings in order to serve as a confirmatory system as well as in agriculture and veterinary for livestock protection (Hyalomma and Rhipicephalus infest livestock) and wildlife reservoir tracking.

Limitations and future work

While genus-level identification is a valuable first step and provides some useful information, however, it is insufficient on its own to plan effective and targeted control measures. Genus-level identification is helpful as it provides information of the genus of interest (i.e., Ixodes, Amblyomma, Dermacentor, Rhipicephalus etc.), general life cycle, preferred hosts (i.e., Ixodes prefer mammals, birds, and reptiles, while Amblyomma often targets large mammals), seasonal activity, associated diseases (i.e., Ixodes are associated with Lyme disease, Amblyomma with Ehrlichiosis and Heartland virus, Dermacentor with Rocky Mountain Spotted Fever and tularaemia). Therefore, species-level identification is more critical for effective control, as it enables public health officials and pest control experts to target specific behavior, life cycle, and ecology of the pest tick, thereby, maximizing the impact of control interventions while minimizing environmental disruption and cost.

In order to directly address the need for broader validation and ensure generalizability, our future work will include a clear, multi-faceted plan:

• We are planning to seek collaborations with research institutions in different climatic and ecological regions (e.g., Southern Europe, North America, Asia) to test I-TickNet on tick specimens native to those areas. This will test its performance against different phenotypic variations within the same genera.

• The most immediate next step is to expand the model’s capability beyond Hyalomma and Rhipicephalus. We plan to include other medically important genera like Ixodes, Dermacentor, and Amblyomma.

• We plan to deploy I-TickNet in a controlled field setting with veterinary clinics or public health units in Northern Cyprus. This will allow us to gather data on its performance with real-time, user-submitted images of varying quality, which is the ultimate test for a field application.