Analysis of the performance of Faster R-CNN and YOLOv8 in detecting fishing vessels and fishes in real time

Fish detection

Object detection in computer vision is a fundamental task that has been extensively researched over the past decades. This discipline aims to identify and locate specific objects in images or video sequences (Wang & Xiao, 2023). It has varied applications, ranging from face recognition to the detection of autonomous vehicles. Several approaches have been developed to address this problem, including feature-based methods such as SIFT and HOG descriptors, as well as approaches based on convolutional neural networks (CNN).

Almero et al. (2020) invented Faster R-CNN, and Rosales et al. (2021) also used Faster R-CNN and deep transfer learning to realize K-complex detection in EEG waveform images, which are more accurate, faster, and very close to real-time performance.

Fish detection using imaging technologies and computer vision systems is essential for fish monitoring and meeting growing global demands. A hybrid approach  (Wang & Xiao, 2023) combines a classification tree and an artificial neural network to solve this problem. This hybrid model achieves 93.6% training and 78.0% testing accuracy, providing a competitive solution. A Faster R-CNN (Weihong et al., 2023) was employed to detect fish with a high accuracy of 99.95% and an average IoU of 0.7816.

Recent research (Ben Tamou, Benzinou & Nasreddine, 2021) improved the Faster R-CNN model for marine organism detection. Improvements include the use of Res2Net101, OHEM algorithm, GIOU and Soft-NMS, leading to a significant performance improvement with an mAP@0.5 of 71.7%. Another study Zhao (2023), Prasetyo, Suciati & Fatichah (2020) proposed in situ detection of underwater jellyfish by improving the Faster R-CNN model. This improved method provides higher training speed and higher accuracy, opening new perspectives for marine ecosystem management. Adiwinata et al. (2020) used Faster R-CNN for fish species classification, achieving an accuracy of 80.4%. This research contributes to the conservation of marine species. Reddy Nandyala & Kumar Sanodiya (2023) explored the use of YOLOv5 and YOLOv8 with synthetic data for underwater object detection. YOLOv8 has proven to perform well in real-world conditions, which is crucial for underwater environments.

Patro et al. (2023) introduced an innovative underwater fish detection system using YOLOv5-CNN, with an average accuracy of 0.86, promising automated monitoring of aquatic resources. In Büyükkanber, Yanalak & Musaoğlu (2023) the author presented a model for detecting fish with abnormal behavior, using YOLO v8 and Deep Sort, improving detection despite fish occlusion. Han et al. (2021) studied the performance of YOLO and Mask R-CNN for fish head and tail segmentation, showing that YOLO slightly outperformed Mask R-CNN in terms of precision and recall.  Sirisha et al.  analysis of the YOLO architecture and its variants, highlighting their performance in object detection. YOLO demonstrates state-of-the-art performance in terms of accuracy, speed and memory consumption, but encounters limitations such as detection of small objects and sensitivity to aspect ratios. Despite the importance of this work, it needs to be improved if it is to be applied to surveillance. While this study worked on bases for people, dogs, cows, trains, cars and motorcycles, our approach has been to work on the maritime domain for boats and fish. The use of R-CNN with YOLO was to try and compare the two approaches to try and fill the possible gaps in YOLO V8. Sirisha et al. (2023)

This work is the result of collaboration between us the various academic researchers and the Tunisian Ministry of Agriculture and Maritime Fisheries. The database was provided by the partner (Ktari & Frikha, 2024) and we used it to obtain the various results.

This work shows significant advances in fish detection using Faster R-CNN and YOLO, contributing to the monitoring of underwater environments and the conservation of marine species. The main objective was to develop a Poof of Concept for an intelligent system for detecting fish and boats. In Tunisia, more and more problems are caused by pollution and the over-exploitation of marine resources. Within the framework of this project, the aim was to use the data proposed by the partner to propose an intelligent approach. We applied Faster R-CNN, YOLOv8 for fish and vessel detection.

Ship detection

Vessel detection from remote sensing images is crucial in various maritime surveil-lance applications, such as maritime traffic control, combating illegal fishing, and security. Deep learning methods, including YOLOv8, have shown promising results in improving vessel detection from remote sensing images (Hu et al., 2021). Additionally, improvements have been made to efficiently detect ships at small scales using attention mechanisms and a new feature pyramid structure (Zhu et al., 2021).

Another innovative approach proposes the use of spatial attention and channel mechanisms to improve the accuracy of small vessel detection without significantly increasing computational resources (Munin et al., 2023). This method also uses an innovative loss function to strengthen model training. The results show superior performance compared to other commonly used approaches.

For ship detection in synthetic aperture radar (SAR) images, the Duplicate Bilateral YOLO (DB-YOLO) is presented, providing real-time performance while maintaining high accuracy (Loran et al., 2022). This method uses a single-stage network and a duplicate bilateral feature pyramid to handle ship diversity in SAR images, achieving impressive results on two SAR ship datasets.

Finally, one study focuses on the identification and classification of ships to control marine pollution (Ke et al., 2021). The YOLOv8 model is evaluated for its ability to detect and classify vessels in real time. The results are very promising with a mAP@50 of 98.9%, but it is worth noting high GPU consumption. These findings are essential for the development of marine pollution control technologies and have a significant impact on the maritime industry, policy makers and researchers engaged in environmental protection.

Ship detection using Faster R-CNN has seen significant advancements in various maritime applications. Traditional systems such as AIS and marine radars are insufficient, hence the importance of airborne radars in the deep sea. To improve the accuracy, (Li, Zhang & Wang, 2020) introduced deformable convolution kernels in Faster R-CNN to model efficiently geometric transformation of ships. This improvement led to an increase in average accuracy of 2.02%.

Gong et al. (2023) proposed an approach for a speed of detection is crucial, especially in emergency situations. An innovative approach was proposed, including a light-weight baseline network, a scale selection method based on K-Means, and the use of RoIAlign. This method achieved an average accuracy of 0.898, 2.78% better than the classic Faster R-CNN, with 800% faster detection speed.

Li, Qu & Shao (2017) present a synthetic aperture radar images challenges due to the small size of the targets. The SSPNet network was developed with specific modules for the detection of small targets, resulting in an average accuracy (AP50) of 91.57%.

Wei et al. (2020) show that the use of deep learning for SAR vessel detection is growing. A dataset was created to evaluate the algorithms, and strategies such as feature fusion and transfer learning were adopted. The results show better accuracy and increased efficiency.

Remote sensing imagery is also an area of interest, where image pre-processing and the introduction of dilated convolution in Faster R-CNN have improved vessel detection (Maity, Banerjee & Sinha Chaudhuri, 2021). An integrated software platform facilitates training of the vessel detection network.

Finally, river vessel detection was addressed with a method based on a regional convolutional neural network (Chao et al., 2018). This approach reduced false alarms and improved accuracy.

These advances demonstrate the growing importance of Faster R-CNN and deep learning for ship detection in a variety of maritime and remote sensing environments. Continuous improvements are needed to ensure safety at sea and improve management of marine resources. Faster R-CNN and YOLO (You Only Look Once) are two of CNN’s most influential architectures for object detection. Faster R-CNN introduces the region proposal network (RPN) to generate potential regions of interest, improving detection efficiency and accuracy. YOLO, on the other hand, pro-poses a one-step approach that simultaneously predicts bounding boxes and object classes in a single network pass, enabling real-time detections.

This section of the literature highlights the importance of object detection at sea and sets the scene for our comparative study of Faster R-CNN and YOLOv8 in this specific context.

Description of maritime monitoring data used

The maritime monitoring data used in this study is essential to evaluate the performance of the Faster R-CNN and YOLOv8 methods (Terven & Cordova-Esparza, 2023) in detecting fishing vessels and fish schools at sea. This data includes a collection of images and of video footage captured in various marine environments, covering a variety of scenarios and conditions. They were obtained from sources such as maritime surveillance cameras, drones and satellites. The diversity of these data reflects the complexity of the marine environment and ensures rigorous evaluation of model performance.

Faster R-CNN overview

Faster R-CNN, based on CNN as shown on Fig. 1, is a two-stage object detection model. In its first step, it uses a RPN to generate potential regions of interest. In the second step, a convolutional neural network performs classification and refinement of the bounding boxes of the detected objects. This architecture allows precise localization and classification of objects in a single approach as proposed by Zhang et al. (2022).

YOLOv8 overview

Figure 2 shows the YOLOv8 architecture, an acronym for “You Only Look Once version 8,” is an object detection model that is characterized by its one-step approach. It divides the image into a grid and simultaneously predicts the classes and bounding boxes of objects in each grid cell. YOLOv8 (Ultralytics, 2024) is known for its speed of execution and its ability to handle real-time detections.

Affes et al. (2023) proposed a comparison between different YOLO implementation for an intelligent video surveillance.

Data preprocessing and dataset preparation

The maritime surveillance data underwent an exhaustive pre-processing process to ensure data quality and adapt it to the requirements of Faster R-CNN and YOLOv8 models. The process includes image normalization to ensure consistent brightness and contrast levels, as well as noise reduction to improve the visual clarity of complex ocean scenes. Additionally, target objects, including fishing boats and fish, were annotated with extreme precision through the use of Autodistill, a computer-assisted annotation method that facilitates high-quality annotation. Additionally, the image database has been expanded to include a variety of marine scenes and conditions, ensuring a more complete representation of the potential challenges models face when detecting objects in marine environments.

The use of the dataset for the different algorithms tested was as follows: 8.78% (63 images) of images for the test, 71.70% (448 images) for the train and 19.50% for the validation (128 images).

Performance evaluation

The performance evaluation of the models was carried out using standard metrics such as precision, recall and F-measure. The execution time required for real-time detections in various maritime surveillance scenarios was recorded. The performance of computer vision models can be measured using a metric called average precision (mAP). It is equal to the average of the average precision measures for all classes of the model and provides a baseline measure for comparing different versions of the same model, taking into account both precision and recall (Affes et al., 2023). mAP is a metric that ranges from 0 to 1, and when a model has a high mAP, it means it has a lower false negative rate and a lower false positive rate. Due to the direct relationship, the precision and recall of the model will increase as the amplitude of the mAP increases. The detailed results of these evaluations are presented in the following section for in-depth analysis

Faster R-CNN model results

Table 1 below represents the fish and fishing vessel detection results with the various performance metrics, including precision, recall, F1 score and mean average precision (mAP).

Evaluation of the performance of fish and fishing vessel classification models re-veals significant difficulties. Accuracy for fish ranged from 0% to 45%, indicating limited ability to accurately predict different classes of fish. For fishing boats, the results were slightly better, with an accuracy of 62%, but still poor overall. Furthermore, the recall rates for both categories were equally unsatisfactory, ranging from 0% to 57% for fish and from 0% to 52% for fishing vessels, highlighting the gap in true positive detections. The F1 values for these two categories show an unstable balance between precision and recall, reaching 51% for fish and 57% for fishing boats, which is still not enough. Furthermore, the mAP for fish is estimated to be 0.16, indicating that the overall quality of predictions for different fish categories is relatively low, while the mAP for fishing vessels is even lower at 0.09, indicating that the overall quality of predictions for different fish categories is relatively poor.

Table 2 shows the fish detection confusion matrix, while Table 3 describes the fishing vessel detection confusion matrix. Confusion matrices provide crucial information about the performance of the fish and fishing vessel detection model. In the confusion matrix for fish detection, the largest values include the high number of false negatives in categories fish, jellyfish, and shark, indicating that the model has difficulty identifying these specific classes of fish. Similarly, for the detection of fishing vessels, the significant values lie in the high false positives in categories 3, 4, and 6, indicating frequent confusions between these classes of fishing vessels. The high numbers in the diagonal cells of both matrices indicate relatively adequate ability to predict some classes, although improvements are needed to reduce classification errors and increase overall precision and recall.

Table 2:

Fish detection confusion matrix.

DOI: 10.7717/peerjcs.2033/table-2

Table 3:

Fishing vessel detection confusion matrix.

DOI: 10.7717/peerjcs.2033/table-3

YOLOv8 model results

Before presenting Table 4 below, it is important to note the remarkable performance of the YOLOv8 model in detecting fish and fishing vessels. For fish, the model achieved a precision of 75.97%, a recall of 60.72%, and an mAP50 of 70.91%. In the case of fishing vessels, the model demonstrated exceptional precision of 87.42%, recall of 90.04%, and mAP50 and mAP50-95 of 93.97% and 91.20% respectively. These performances demonstrate the effectiveness of the YOLOv8 model in both detection domains.

These results indicate the reliability of the YOLOv8 model for detecting both fish and fishing vessels, highlighting its ability to provide accurate predictions in both of these object detection scenarios. Figure 3 illustrates performance metrics for fish detection while Fig. 4 describes performance metrics for detection of fish vessels.

Performance analysis of the YOLOv8 model for detecting fish and fishing vessels highlights impressive results. In the case of fish, the model displays an accuracy of 75.97%, meaning that nearly 76% of the fish detected are correctly identified. The recall of 60.72% indicates that the model manages to capture more than half of the fish present in the images. The mAP50 of 70.91% underlines the effectiveness of the model in maintaining a high average precision over 50% of the confidence thresholds, despite the complexity of the task. Regarding fishing vessels, the model demonstrates an exceptional accuracy of 87.42%, highlighting its ability to minimize false detections. A recall of 90.04% indicates that the model identifies the vast majority of real fishing vessels. The values of mAP50 (93.97%) and mAP50-95 (91.20%) highlight the model’s ability to achieve exceptional performance across various confidence thresholds. These results demonstrate the effectiveness and reliability of the YOLOv8 model in accurately detecting fish and fishing vessels, making it a powerful tool for maritime surveillance applications.