ERISNet: deep neural network for Sargassum detection along the coastline of the Mexican Caribbean

Study area

We selected the entire coast of Quintana Roo, located in the eastern zone of the Yucatan Peninsula, Mexico. One-kilometer-sized MODIS pixels in front of the beach line were selected (from 21.496124 Latitude, -87.546677 Longitude, to 18.477211 Latitude, -88.293625 Longitude), bordering the coast of Quintana Roo. This region is the main vacation destination in Mexico. In addition, the area is located where massive arrivals of Sargassum were recorded in 2015 (van Tussenbroek et al., 2017) and 2018.

Dataset definition and processing

To build the dataset, three components were developed: (1) A list of sites and dates with and without Sargassum based on official information compiled by the government of Quintana Roo (2018) and field work in 2015 and 2018; (2) sets of Aqua-MODIS imagery, both with and without Sargassum, for the coast of Quintana Roo based on the list of dates and sites mentioned above; and (3) Software was developed (extract_data.py) to add the list of sites and images set, which ultimately outputs the data set.

Selection of Aqua-MODIS swath imagery

Based on the known Sargassum arrivals in the coastal zone of Quintana Roo, Aqua-MODIS swath images were also used in the construction of the dataset. The Julian day and the Universal Time Coordinated (UTC time) of all selected swath images (with and without Sargassum) were recorded (https://lance-modis.eosdis.nasa.gov/cgi-bin/imagery/realtime.cgi) and then the PDS files (L0) were downloaded from MODIS OceanData (https://oceandata.sci.gsfc.nasa.gov/MODIS-Aqua/L0/). In total, 80 PDS files were downloaded (42 files corresponding with Sargassum dates and 38 files without). After the processing and re-projection of all files, an RGB composition for each swath image was made to allow a visual quality check of each image. Due to the presence of clouds in the area of interest, a total of 19 images were discarded (eight images with Sargassum and eleven without). Afterwards, 30 files with Sargassum and 29 files without Sargassum remained as the imagery used in the development of the dataset (MODIS-Aqua, 2018). An example of these images is shown in Fig. 1. Not all the images were ideal for network training, because of excessive cloudiness.

 
____________________________________________________ 
Pseudocode 1 Scheme of data processing 
____________________________________________________ 
  1:  database ← empty 
  2:  for all L0 datafiles do 
  3:    L1A ← modis_L1A.py(L0 file) 
  4:    GEO ← modis_GEO.py(L1A) 
  5:    L1B ← modis_L1B.py(L1A, GEO) 
  6:    L2 ← l2gen(L1B) 
  7:    Reprojected ← gpt.sh(L2) 
  8:    Data ← extract_data.py(Reprojected) 
  9:    append_database(Data) 
 10:  end for 
 11:  return database 
____________________________________________________    

ERISNet a deep learning network for Sargassum detection

ERISNet is a deep neural network designed to detect Sargassum along the coastline. ERISNet is inspired mainly on two types of architectures; Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN). An issue present in virtually all models of Machine Learning is overfitting, therefore during the design of the proposed architecture, special attention was paid to maintaining the tradeoff between optimization and generalization of the network by using different mechanisms such as dropout, batch normalization and weight regularization.

The structure of the convolutional block it is formed by four components: Convolutional layer, RELU activation function, Batch Normalization, and Dropout operation. The objective of convolutional blocks is to efficiently extract characteristics or patterns from the input dataset. The main component of the block is a convolutional layer of 1D (dimension one). After conducting numerous tests with different filters and sizes, the decision was made to use a total of [64, 128, 128] filters with a size of [8, 5, 3].

With the aim of avoiding the overfitting, three mechanisms were used: dropout regularization, weight regularization, and batch normalization. Dropout is one of the most effective and most commonly used regularization techniques for neural networks and is used to improve over-fit on neural networks. At each training stage, individual nodes are either dropped out of the net with probability 1 − p or kept in the net with probability p, so that a reduced network is left; incoming and outgoing edges to a dropped-out node are also removed.

Weight regularization is another common way to mitigate overfitting. This involved putting constraints on the complexity of a network by forcing its weights to take only small values, making the distribution of weight values more regular. There are two kinds of weight regularization: L1 and L2 regularization. L2 was used in the convolutional blocks of ERISNet. In L2 (see Eq. (1)) a “squared magnitude” of coefficient as penalty term to the loss function is added. (1)${\displaystyle L_2\left(W\right)=w_1^2+w_2^2+...+w_n^2}$

The convolutional block uses a Batch Normalization operation to increase the network performance. Batch Normalization (BN), is a technique for improving the performance and stability of ANN, providing any layer in a neural network with inputs that are zero mean/unit variance (Ioffe & Szegedy, 2015). During the learning process, the type of initialization of weights could cause a digression to gradients, meaning the gradients have to compensate for the outliers, before learning the weights to produce the required outputs. BN regularizes this gradient by normalizing activations throughout the network. It prevents small changes to the parameters from amplifying into larger and suboptimal changes in activations in gradients.

Another component that is part of ERISNet are the recurrent blocks. The main objective of these blocks is to provide memory to ERISNet. Recurrent neural networks (RNN) are a special type of neural network widely used in problems of prediction in time series. Given their design, the RNN allows information to be remembered for long periods and facilitates the task of making future estimates using historical records. Unlike traditional neural networks, LSTM networks have neuron memory blocks that are connected through layers.These memory blocks facilitate the task of remembering values for long or short periods of time. Therefore, the stored value is not replaced iteratively in time, and the gradient term does not tend to disappear when the retro propagation is applied during the training process.

Finally, as in the case of convolutional blocks, recurring blocks also make use of batch normalization to improve network performance. As can be seen in Fig. 2, ERISNet consists mainly of nine convolutional blocks and two recurring blocks.

ERISNet was designed using the programming language Python version 3.7.0 and the library Keras 2.2.4 with TensorFlow 1.10.0 as backend.

TensorFlow is an open source library developed by the Google Brain Team for numerical calculation using data flow graphing programming. The nodes in the graph represent mathematical operations, while the connections or links in the graph represent the multidimensional data sets (tensors). Tensorflow has various automatic learning algorithms and other tools that make it ideal for the development of new methods. Keras is a Python library that provides a clean and simple way to create Deep Learning models on top of other libraries such as TensorFlow, Theano or CNTK.

All the architectures presented in this work were developed and trained using a Lenovo Workstation with Intel Xeon EP processor, 64 GB of RAM, NVidia Quadro K5000 GPU running the Linux operating system Ubuntu 18.04 64 bits.

Basic statistical analysis

When each of the corrected bands (rhos and rhot) of the generated data with and without Sargassum were averaged, small differences were observed. Only the averages corresponding to bands 859 and 1,240 nm were higher in the case of the presence of Sargassum. Therefore, a powerful algorithm-tool is needed to efficiently classify the small differences among the values of each pixel and thus classify the presence/absence of Sargassum. A basic statistical analysis of data, shows why 859 and 1,240 nm are the bands which FAI index uses. Table 2 shows the means in the case of rhos bands.

In Fig. 3 behavior of rhos values used for a pixel example (Latitude 18.71582, longitude −87.69191), in the presence / absence of Sargassum are shown. Trends in Figs. 3A and 3B are similar, and differences are not clear between both cases.

We have chosen a survey (Wang, Yan & Oates, 2017) showing a wide comparison between different classification algorithms by using more than 40 different classic datasets. In those studies, authors propose three new classification algorithms based on machine learning techniques and neural networks showing a good accuracy for the different datasets.

In order to compare the performance of the proposed methodology, we have chosen two effective algorithms presented by Wang, Yan & Oates (2017).

Multilayer perceptron

The multilayer perceptron (MLP) it is defined as the base algorithm for comparison with the rest of the proposals. The MLP used is composed of an input layer, three intermediate or hidden layers, and the output layer. Each of the intermediate layers is composed of 500 neurons that use the rectified linear unit (RELU) as an activation function. To improve the generalization of the neural network, a “dropout” function with values of [0.2 0.2 0.3] respectively has been inserted at the end of each of the intermediate layers. Dropout, applied to a layer, consists of randomly dropping out (setting to zero) a number of output features of the layer during training. Finally, the network has a softmax layer which is widely used in the Multiclass single-layer classification. Formally, each of the blocks of the hidden layers is described as shown in Eq. (3).

${\displaystyle \tilde{x}=f_{\left(dropout,p\right)}\left(x\right)}$ (2)${\displaystyle y=W.\tilde{x}+b}$ ${\displaystyle h=RELU\left(y\right).}$

The MLP was used to perform the classification of the whole dataset (4515 MODIS pixels). The dataset was divided in two groups: a training and test group each with approximately the same amount of data (see Table 2). The learning process was carried out during 3000 epochs, presenting 100 data points in each one (batch size). In Fig. 4A, the result of the learning and test process is shown. On the one hand, the continuous line shows that the MLP has a good degree of optimization (close to 100%), which is to be expected given the learning capacity of this type of network. On the other hand, the dashed line shows the result of the testing process. During the testing process, a set of data which was never used throughout the training process was presented to the MLP in order to see the generalization capacity. As shown, the MLP has a good level of generalization, correctly classifying 83.76% of the test points. There is a wide difference between the optimization and generalization curves, which is usually an indicator of overfitting.

Fully convolutional network

Convolutional Neuronal Networks have shown a good performance in classification problems. The fundamental difference between a multilayer perceptron and a convolution layer is that MLP layers learn global patterns in their input feature space whereas convolution layers learn local patterns. The basic block of the FCN is composed of a set of filters that are responsible for the extraction of features from the dataset. RELU has been used as an activation function. At the end of the block, the FCN incorporates a new block called Batch normalization (BN). Batch normalization reduces the amount by which the hidden unit values shift around (covariance shift). To increase the stability of a neural network, BN normalizes the output of a previous activation layer by subtracting the batch mean and dividing by the batch standard deviation, getting ten times or more improvement in the training speed.

The FCN used is composed of three convolutional blocks: the first block of the network is composed of 128 filters with eight elements each, the second layer is composed by 256 filters with five elements each, and the last block is composed of 128 filters with three elements each. The objective of this network is to extract from each of the blocks attributes of the data, from general the particular, thus resulting in a good representation of the information contained in the data. With this representation of the data, it is possible to correctly classify information into different classes. Each of the blocks of the hidden layers are formally described by the Eq. (4).

${\displaystyle y=W\otimes x+b}$ (3)${\displaystyle s=BN\left(y\right)}$ ${\displaystyle h=RELU\left(s\right).}$

Like the MLP, the FCN was used to perform the classification of the whole dataset. With the aim of making a comparison on equal terms, both the dataset and the training parameters used for this model were the same as those presented in the MLP. Figure 4B shows the result of the FCN training and testing process. Similarly to the MLP, the FCN had a fairly high level of optimization and the power of generalization showed good results, correctly classifying the 86.38% of the data points. However, the difference between generalization and optimization suggest the possible presence of overfitting in the network once again.

ERISNet Validation

At present there are multiple validation methods for neural networks where cross-validation is the most accepted. Cross validation is a statistical method used to estimate the skill of machine learning models. The cross-validation can be divided mainly into two groups: Exhaustive cross-validation and Non exhaustive cross-validation. Among the methods of Exhaustive cross-validation, the following stand out: Leave one out cross-validation (LOOCV), Exhaustive cross validation, Leave out of cross validation, while Non exhaustive cross validation highlights: k-fold cross-validation, Holdout method and Repeated random sub sampling validation.

Due to the characteristics and size of the dataset used, k-fold was chosen (k = 5) as the cross-validation method of the ERISNet with k = 5. To carry out the cross validation the data set was divided into k parts of which k − 1 parts were used as a training set, while the remaining part was used as a validation set. In Eq. (4), the results of the cross-validation are shown. c_i expresses the number of correct classes within the dataset while e_i corresponds to the number of classes correctly classified by the model. (4)${\displaystyle {MPCE}_k=\frac{1}{k}\sum _{i=1}^k\frac{e_i}{c_i}.}$

ERISNet was trained, tested, and compared with the rest of its competitors. As in the previous cases, the same criteria were used, that is, the total data set was used by using k-fold cross validation. The algorithm was trained during 3,000 epochs with a batch size of 100 data points, the same seed of random numbers used by MLP and FCN was also used. As can be seen in Fig. 4C, unlike what happened in the case of the MLP and the FCN, the difference between the capacity of optimization and generalization of the network was lower, suggesting that there was no overfitting in the network during the training process. It is important to mention that the level of optimization of the network was less than its competitors, which suggests that if the network is trained during a higher number of epochs this could improve and the generalization could be higher. After the network training, ERISNet obtained a 90.08% success for the classification test points, implying an increase of 7% with respect to the MLP and 4.1% with respect to the FNC.

Figure 4D shows a comparison on the generalization of the MLP, the FCN, and ERISNet. As illustrated, the behaviors of the MLP and the FCN are very similar. However, ERISNet presented an increase in the generalization capacity. As summary, after the dataset classification by using the methodologies mentioned above, the MLP performance was 83.76% of accuracy followed by the FCN with 86.38%, and the highest accuracy was reached by ERISNet with 90.08%. Based on all the previous tests, it is concluded that the proposed ERISNet algorithm is capable of classifying more precisely new points than other studies.

FAI index

There are only few studies of Sargassum detection along Caribbean coastlines. Therefore, there are no algorithms or methodologies that allow us to make a direct comparison between our proposal and other studies, nevertheless we have calculated the FAI index to the dataset used in the present study in order to know the behavior of our dataset from the point of view of this index. Figure 5 shows the boxplot diagram for the FAI index results. A clear statistical difference can be observed between both datasets. It can be seen that the median of the pixels FAI in the presence of Sargassum is slightly higher than the median of the pixels without Sargassum. The median of the pixels without Sargassum is closer to zero.

It is important to note that 50% of the data with Sargassum are less compact than that of those without Sargassum, which implies a greater distribution of the values within this 50%. The largest difference appears in the lower whiskers of the Sargassum boxplot. This difference indicates that the Sargassum FAI values reflect a greater bias with respect to those points without the presence of Sargassum (Fig. 5). Table 3 shows the results for the computation of the traditional statistical values made to the FAI index on the data set.

Sargassum detection is a complex issue, and for that reason our dataset had to be built with official information of dates, sites and with field work. Our dataset is significative to provide remote sensing information about Sargassum, until now not available for this region. Hence, it is possible to apply the dataset in other methodologies or for training other algorithms. Thus to compare the classification performance of ERISNet, two effective algorithms were chosen (Wang, Yan & Oates, 2017). As shown in Fig. 4D, ERISNet obtained the best performance.

Sargassum detection along the coastline is a challenge for conventional techniques used in remote sensing. The presence/absence of Sargassum on satellite imagery along the coastline is not as clear as in the open sea, because there are several transitional ecosystems. Under these conditions the high classification performance of ERISNet allows to observe the small differences between the values of the bands used for the classification and to determine the presence or absence of Sargassum in each pixel with a maximum accuracy of 90.08%. This is highlighted for an ANN, since increase 1% unit in the classification requires high performance design and implementation.

Traditionally, detection of suspended matter in open waters is accomplished through satellite products or through well established indexes (Hu, 2009; Hu et al., 2015; Hu et al., 2016; Cuevas, Uribe-Martínez & Liceaga-Correa, 2018). Therefore, the present study is innovative, since it has a high classification accuracy of pixels with presence/absence of Sargassum, using as input data the corrected bands rhos and rhot. The present research showed that under conditions of high concentration of Sargassum, as those presented along the coast of Quintana Roo in 2015 and 2018, it was possible to detect Sargassum with MODIS data.

Although the coast of the Mexican Caribbean has high economic and ecological importance, there is no monitoring system that contributes to make decisions facing threats of massive arrival of Sargassum. Therefore, the present work is very relevant for this region, because it offers the basis for an early warning system design.

ERISNet could be applied to other coastal areas of the Caribbean region, thereby we propose to design an artificial neural network capable to detect Sargassum in open waters and to build new training datasets based on satellite products and well established vegetation indexes.