LocustLens: leveraging environmental data fusion and machine learning for desert locust swarm prediction

Data fusion

Extensive work has been done in predicting locust breeding grounds and their attacks (Kimathi et al., 2020). All these earlier methods share specific characteristics. For most of the research in this field, the FAO desert locust database (Food and Agriculture Organization (FAO), 2023) serves as the primary data source, emphasizing various regions of Africa. In this research, a comprehensive data collection process is carried out, collecting data for 42 countries related to locust swarm attacks and corresponding start year and environmental data from TerraClimate (TerraClimate, 2023). Data fusion combines positive values of locust attacks from FAO and key environmental factors from TerraClimate datasets. TerraClimate and FAO datasets are reverse geocoded to convert longitude and latitude to country names and regions. Moreover, negative classes are added for months where locust attacks were not reported to make the dataset favorable for predictions. The resulting dataset contains essential environmental factors strongly associated with locust attacks and geographical details of these 42 locust-prone countries, including the years and months when locust attacks typically begin. This data fusion resulted in a more intricate and comprehensive dataset named as GLAD 42 (Global Locust Attack Database for 42 countries) that includes all the vital independent features required for constructing a robust classification model.

Use of lazy learners

Research in this field has three main areas: assessing locust risks, predicting breeding grounds, and forecasting locust presence. Multiple methods, such as Random Forest (RF), and neural networks along with other classification and regression algorithms, have successfully predicted African locust breeding grounds (Gómez et al., 2018; Gomez et al., 2021). As the literature review delves into, the classification research that has been done so far on locust attacks is either restricted to limited geographic regions, like Saudi Arabia, Kenya, and Africa, or it has only taken into account one or two environmental characteristics, primarily soil moisture, using remote sensing. However, a significant gap remains as broader global perspectives and comprehensive environmental factors have not been adequately addressed in the existing literature. Moreover, the majority of these studies have relied on FAO datasets, highlighting the need for more detailed and region-specific datasets encompassing a wider range of environmental factors. We have proposed LocustLens, using K-nearest neighbor (K-NN) following the renowned KISS principle as the Lazy Learner methodology to enhance our ability to predict locust attacks across 42 countries. The KISS principle is a design principle that states that most systems function best when they are maintained simply as opposed to convoluted. Significant benefits from this approach, such as those seen with Lazy Learners like K-NN, are their simplicity, making them easy to understand and apply. Furthermore, their adaptability is a notable benefit. Lazy Learners quickly adapt to changes in the dataset due to their inherent nature, eliminating the need for time-consuming retraining and re-testing methods. Our algorithm adjusts fluidly to these updates as we add new country-specific data, guaranteeing effective scaling and enabling us to quickly add unique data inputs for accurate and reliable predictions of locust attacks.

Novel methodology

We have proposed a novel methodology for building a classification model. In this novel approach, called LocustLens, we hierarchically employed the K-NN algorithm. It starts with stratified sampling, splitting the dataset into training and testing subsets. It then iterates over test samples based on the dataset creation; data subsets are created based on a country match of the test sample with corresponding fitting data while meticulously tracking the time involved. These subsets are smaller than the complete dataset, providing the performance advantage of the divide-and-conquer approach. K-NN is applied to these refined subsets for fitting. The fitted K-NN model predicts outcomes for test samples. Finally, the model computes average fitting and testing time and generates comprehensive performance evaluation results, providing insights into LocustLens’s performance in predicting locust attacks across various countries.

Sustainable machine learning

Existing studies have shown that desert locusts can change their behavior, ecology, and physiology in response to changes in climatic conditions (Symmons & Cressman, 2001). In our research paper, along with the prediction of locust attacks, we have examined the environmental impact associated with our model’s fitting and testing phases. Machine learning models can affect our climate in various ways, some of which may be subtle. Potential emissions from machine learning (ML) models are divided into three categories: (1) system emissions, (2) application emissions, and (3) computing-related emissions. Specifically, we investigated the carbon footprint generated during computational processes, considering the total electricity consumed during training and testing and how much emissions result from CPU and RAM. This study is motivated by the significant role that greenhouse gas (GHG) emissions play in contributing to climate change, which subsequently influences factors such as precipitation patterns and temperature, which are crucial determinants in locust infestation. Figure 2 represents that embodied and operational emissions comprise the computing-related climate impact of machine learning models. The arrow indicates a direct relationship between the total amount of greenhouse gas (GHG) emissions and the embodied emissions connected to computing. Stated differently, the hardware utilized in machine learning models has an environmental impact and consumes energy, which leads to a rise in greenhouse gas emissions. On the other hand, accuracy and precision in measuring or computing the environmental impact are what precision means in estimating GHG emissions. There is a greater understanding of the true environmental impact of the computing infrastructure when the estimations are more accurate.

Data sources and integration process

TerraClimate dataset

The environmental data was derived from the TerraClimate dataset (TerraClimate, 2023), which combines high-spatial-resolution data from the WorldClim dataset with coarser-resolution datasets like CRU Ts4.0 and JRA-55. WorldClim provides high-resolution, gridded climate data, including long-term averages of temperature, precipitation, and other key environmental variables, widely used for ecological and environmental research due to its detailed spatial coverage. In this study, WorldClim data is crucial for assessing the impact of climate factors like temperature and precipitation on locust behavior and outbreaks. TerraClimate data is extracted from WorldClim and it offers a comprehensive monthly dataset that includes variables such as maximum temperature, vapor pressure, accumulated precipitation, runoff, and soil moisture, spanning from 1958 to 2020 with a spatial resolution of 1/24° (4 km) (Abatzoglou et al., 2018; TerraClimate, 2023). For this study, the focus was on three key environmental features—soil moisture, maximum temperature, and precipitation—during the period 1985 to 2020, aligning with the FAO locust occurrence data. The Fig. 3 shows relationship between environmental features and target variable and correlation among them.

Data processing and feature engineering

The raw data from TerraClimate was initially in netCDF format, which required conversion to a more accessible format for integration with the FAO dataset. Three features including soil moisture, maximum temperature, and precipitation conversion of TerraClimate files which span from 1985 to 2020, are done as they were available in netCDF format. The longitude and latitude of countries are reverse geocoded to convert into region and country names. Figure 4 shows the complete data collection and fusion process.

To ensure temporal alignment, the FAO dataset’s locust occurrence records were integrated with the corresponding environmental data from TerraClimate for the same years and months. This integration was carefully managed to maintain the integrity and accuracy of the temporal and spatial dimensions of the data. Hence, our final dataset has seven independent features and one dependent feature with class labels according to locust presences as “yes” and “no”. This dataset is used in building a binary classifier for locust attack prediction.

Feature engineering

The latitude and longitude are converted to human-readable addresses using Reverse Geocoding. “Country name” and “Region” help in predicting the presence of desert locust swarms over a larger area and improve the readability of our dataset. We added a target variable called “Locust Present” (set to yes by default as the FAO dataset only contains positive class). We then selected the following Features: “Start Year”, “Start Month”, “Region”, “Country name” (extracted from latitude and longitude via Reverse Geocoding), “Precipitation”, “Soil Moisture”, “Maximum Temperature”, and “Locust Present”. Data cleaning is done by dropping any row containing a null value. The data obtained from FAO exclusively contained records of countries where locust occurrences were reported, forming the foundation of our dataset. To ensure a more comprehensive and balanced dataset, we generated a negative class representing instances when desert locust swarms were absent. To provide a concrete example, consider India in the 1990s: In 1990, desert locust swarms were observed in October and November in just one out of the initially identified six regions. To address this absence, we included additional rows for that specific month. These additional rows included the Location Name, Start Year (1990), and Start Month, indicating “no” for locust Presence in the remaining regions. We repeated this process every month, spanning from 1985 to 2020. Figure 5 is a visual representation of this process.

Following this data augmentation, we conducted rigorous data quality assessments and addressed missing values. The resulting dataset, comprising 79,528 rows and seven columns, is the foundation for our work. It allowed us to explore the temporal and spatial dynamics of desert locust swarm occurrences across various locations during the studied period with precision and accuracy. Figure 6 depicts country wise positive and negative instance. It is evident from the count that the number of instances for each country varies in terms of positive and negative class labels.

Locustlens

LocustLens aims to predict locust attacks with high accuracy and time efficiency. In LocustLens, we have hierarchically used the K-NN algorithm. K-NN is a versatile and adaptable method for classification and regression tasks that can manage several data types.

LocustLens begins with dividing the dataset into training (N) and testing (M) sets using stratified sampling. For each test sample (M[i]), we created smaller subsets from the training data, focusing on samples from the same country as the test sample while keeping track of the time it takes. The K-NN model is fine-tuned using hyperparameters like the number of neighbors, weights, and distance metrics.

• In dataset feature space, point x represents an individual test sample, denoted as M[i], for which we are making locust swarm prediction. This test sample belongs to a specific country.

• y corresponds to the class labels of the training samples, either ‘yes’ or ‘no’ within the country-specific subset that matches the country of the test sample x.

After fitting and testing LocustLens for different sets of hyperparameters the best optimal configurations were noted using randomized search. Equation (1) is used to calculate the distance to pick ‘k’ nearest neighbors. (1)${\displaystyle \text{Manhattan distance}=\sum _{r=1}^d{w}_{r\times }|x_r-z_r|}$

where:

• x represents test sample M[i]

• d represents number of dimensions or feature space denoted by r; w represents weights which in our case (w_r =1 for all r).

• z represents a training sample to which we are measuring distance.

Equation (2) is used to make predictions on data point x, and the model assigns class label C that occurs most frequently among the ’k’ nearest neighbors to x accordingly. (2)${\displaystyle C\left(x\right)=\underset{C}{argmax}\sum _{i=1}^k\mathbb{I}\left(y_i=C\right).}$

• 𝕀(y_i = C) is the indicator function that is 1 if y_i = C (i.e., if the ith neighbor belongs to class C) and 0 otherwise. C(x) is the class assigned to data point x.

• y_i is the class label of the ith neighbor

• 𝕀(y_i = C) is the indicator function that is 1 if y_i = C (i.e., if the ith neighbor belongs to class C) and 0 otherwise.

Hyperparameters selected after applying a randomized search algorithm on a set of hyperparameters for LocustLens are listed in Table 4.

Therefore, LocustLens is evaluated recursively for all countries in the dataset, resulting in a high accuracy score. LocustLens is developed using a lazy learner algorithm. We opted for this approach because lazy learners like K-NN are robust and well-suited for managing our extensive dataset covering 42 countries. This choice ensures that our model remains adaptable and responsive to evolving data conditions, maintaining accuracy and relevance as new data of countries and data of new countries are incorporated. Additionally, our model assesses the time-sensitivity of both the training and testing phases and measures carbon emissions resulting from the machine learning model. Thus, LocustLens is a highly flexible, accurate, and optimized predictive model for locust prediction. Figure 7 is a graphical representation of the working of LocustLens.

 
___________________________________________________________ 
Algorithm 1 LocustLens Algorithm_______________________________________________________________________________________________________________________________________________________________________________________________________________________________ 
Input: {(xi,yi)} where i = 1 to N 
xi = {Regioni, CountryNamei, Start Yeari, Start Monthi, PPTi, TMAXi, soil moisturei} yi = {Locust_Presenti} 
Output: 
N data subsets 
Trained K-NN 
LocustLens Classifier 
Phase I: 
Input = 7 independent features / 1 dependent feature, Output = Final Dataset 
    1.  Independent features = Start Year, Start Month, Precipitation, Maximum temperature, soil moisture, X coordinates, Y coordinates, 
    2.  Dependent features = LocustPresent (class label = “yes”) 
    3.  Reverse geocoding to get Region and Country name. 
    4.  Identify missing values. 
    5.  Feature Scaling of independent and dependent features 
    6.  Negative class addition to LocustPresent 
The output of Phase I is the Final Dataset denoted as {(xi,yi)} where i ranges from 1 to N. 
Phase II: 
Input = Final Dataset with input {(xi,yi)} 
  1.  Initialize stratified sampling: splits, shuffle, and random state. 
    2.  for i: 1 to X and Y index 
    3.     Get stratified Xtrain, Xtest 
     4.     Get stratified Ytrain, Ytest 
     5.      for j: 1 to Xtest 
     6.      Make a list: CountryName of j samples 
    7.      Get Xtrain indices where CountryName of jth sample =Xtrain samples 
    8.      Generate data subsets S based on CountryName 
    9.      Initialize KNN: weights, neighbors, distance. 
   10.      Fit KNN on S 
   11.      Get prediction on each Xtest [j] 
   12.      Record predicted vs Y values. 
   13.      end for 
   14.  end for 
   15.  Record classification report (Accuracy, Precision, Recall, F1-Score, AUC) 
   16.  Record average training and testing time 
   17.  Record carbon emissions of training and testing of the model._____________________________________________________________________________________________________________________________________________________________________________________________________________    

Comparison models

A thorough review of the literature served as the foundation for comparing LocustLens with other models. Models like support vector classifier (SVC), logistic regression (LR), K-NN, and decision trees (DT) were found to be frequently used in the classification of pest or locust attacks on crops. These models were selected because they exemplify industry standards for binary classification problems including non-imagery data, which is comparable to the GLAD42 dataset utilized in this research. Making the comparison with these models is essential to comprehending LocustLens’s performance in a proven setting. The investigation can show whether LocustLens provides any notable benefits or overcomes particular difficulties more effectively than by comparing it to these widely utilized techniques. Table 5 lists down all combinations of hyperparameters used in comparison with LocustLens.

Classification metrics

The performance metrics used to evaluate models are explained in this section. These metrics are adaptable to scalar analysis, which provides a quantitative measure of model performance, and graphical analysis, which provides visual insights into model performance (Tharwat, 2021). The area under the receiver operating characteristic curve (AUC), F1-score, precision, recall, and accuracy are the scalar metrics we have considered for evaluation. A summary of these measures is provided below:

Values predicted by the classifier can be classified into four different labels and confusion matrix can be used to visualize them. Table 6 is depicting four classification labels that can be further used to measure accuracy, precision, recall, AUC and F1-score.

1. True positive (TP): when LocustLens predicted value is ‘yes’, and ground truth is ‘yes’.

2. True negative (TN): when the LocustLens predicted value is ‘no’, the ground truth is also ‘no’.

3. False positive (FP): when LocustLens predicted value is ‘yes’, and the ground truth is ‘no’.

4. False negative (FN): when the LocustLens predicted value is ‘no’, and the ground truth is ‘yes.’

Based on the metrics mentioned above, accuracy, precision, recall, and F1-score can be calculated according to the Eqs. (3) to (6).

Accuracy: The ratio of correct predictions to total predictions. (3)${\displaystyle \text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}}$ Precision: It is the ratio of correctly predicted positive values to all the predicted positive values. (4)${\displaystyle \text{Precision}=\frac{TP}{TP+FP}}$ Recall: Recall/sensitivity is the ratio of correctly predicted positive values to all the actual positive values. (5)${\displaystyle \text{Recall}=\frac{TP}{TP+FN}}$ F1 –score: It is the harmonic mean of precision and recall. (6)${\displaystyle \text{F1-score}=\frac{2\times \text{Precision}\times \text{Recall}}{\text{Precision}+\text{Recall}}.}$

AUC: When a model makes a prediction, it frequently gives each instance a probability score, indicating how likely it is to fall into the positive class. The model’s capacity to differentiate between these two groups using these probability scores is measured by the AUC metric. The value can be between 0 and 1, with higher numbers indicating greater model performance. An AUC of 1 denotes a perfect model, while an AUC of 0.5 denotes a model that performs no better than random guessing.

Estimating carbon footprints

Locust attack results when the climate becomes favorable for their breeding. Machine learning models can delicately and variably affect the climate, affecting environmental dynamics in complex and direct ways. The environmental impact of ML models can be categorized into embodied emissions, which result from hardware production, and operational emissions, arising from ML model development, data processing, training, and inference. To quantify the carbon footprint of ML training, we employed the package called CodeCarbon (CodeCarbon, 2023), which calculates emissions based on the following factors: (7)${\displaystyle \text{Carbon Emissions}\left(C{O}_2-\text{equivalent}\right)=\text{Energy Consumption (kWh)}\times {\text{Regional Carbon Intensity (in CO}}_2\text{per kWh)}.}$

CodeCarbon tracks and records the power usage of GPU, CPU, and RAM during code execution by capturing data at regular 15-second intervals to measure electricity consumption accurately. The regional carbon intensity of electricity refers to the carbon emissions associated with generating electricity in a specific geographical area or region (Google Cloud, 2023). It is typically measured in terms of carbon dioxide CO₂ emissions per unit of electricity generated. Thus, using the same procedure, we will measure carbon emissions for our machine-learning models.