TS-YOLO: a lightweight real-time traffic sign detector in resource-constrained environments

Traffic sign detection: existing approaches

Deep learning-based traffic sign recognition has attracted considerable attention. Early studies combined faster region-based convolutional neural network (Faster R-CNN) with MobileNet (Li & Wang, 2018; Howard et al., 2017), whereas subsequent work integrated the Hue-Saturation-Value (HSV) color space into enhanced LeNet-5 architectures (Cao et al., 2019; LeCun et al., 2002) and optimized versions of mask region-based convolutional neural network (Mask R-CNN) (Tabernik & Skočaj, 2019; He et al., 2017). Efforts to create lightweight models have encompassed knowledge distillation (Zhang et al., 2020), improvements to YOLOv4-Tiny (Bochkovskiy, Wang & Liao, 2020; Wang et al., 2021), and the use of generative adversarial networks (GANs) to generate diverse training samples (Dewi et al., 2021). Additionally, adaptations of YOLOv5 have been proposed to enhance small traffic sign detection (Chu et al., 2023; Jocher, Chaurasia & Qiu, 2020). An improved sparse region-based convolutional neural network (Sparse R-CNN) (Liang et al., 2022) incorporates coordinate attention and a feature pyramid to enhance small traffic sign detection. The accompanying self-adaption augmentation (SAA) and detection time augmentation (DTA) modules mitigate practical challenges, such as illumination variation. On the Beijing Union University Chinese Traffic Sign Detection Benchmark (BCTSDB) dataset, this configuration achieves 99.1% AP50 but operates at a relatively low frame rate of 18 frames per second (FPS). TSD-YOLO (Du et al., 2024), by contrast, employs SPD-Conv to preserve spatial details, SK attention to suppress background noise, and Weighted Intersection over Union v3 (WIoUv3) to alleviate class imbalance. It achieves 90.6% mean Average Precision at intersection over union threshold of 0.50 (mAP50) on the TT100K dataset at a high speed of 59 FPS, demonstrating its suitability for real-time autonomous driving scenarios.

Nonetheless, many existing methods still struggle to balance accuracy with speed under challenging conditions such as diminutive object sizes, drastic viewpoint changes, and complex illumination (He et al., 2017; Zhang et al., 2020; Bochkovskiy, Wang & Liao, 2020; Wang et al., 2021; Dewi et al., 2021). These constraints necessitate detectors that are simultaneously robust and efficient. Such challenges become even more pronounced in resource-constrained in-vehicle environments, where high computational overhead is untenable (Alaba, Gurbuz & Ball, 2024). Accordingly, designing a robust, lightweight network capable of operating across diverse driving scenarios remains imperative.

General object detection frameworks

Object detection automatically classifies and localizes objects in images and videos, thereby producing category labels and bounding boxes for each detected instance (Chen et al., 2024). Existing methods are commonly categorized by their processing pipeline (one-stage vs. two-stage) and by the mechanism used to generate bounding boxes (anchor-based vs. anchor-free).

One-stage detectors employ a linear-regression paradigm to regress bounding boxes, thereby achieving the speed required for real-time applications (Zhang & Cloutier, 2022). These algorithms are regarded as end-to-end detectors. Currently, numerous state-of-the-art models follow the one-stage paradigm, whose steadily improving accuracy provides an effective balance between speed and precision.

As the name implies, two-stage detectors divide the process into two sequential stages. For instance, in Faster R-CNN, the first stage generates region proposals via a region proposal network (RPN) (Ren et al., 2015). In the second stage, a classification head assigns categories and a regression head refines the coordinates, producing the final prediction boxes. Such algorithms attain high accuracy with low miss and false-positive rates; however, their extended training time and slower inference hinder real-time deployment.

Relative to two-stage methods, one-stage detectors extract features and regress boxes in a single pass, after which non-maximum suppression (NMS) is applied (Yu et al., 2023). By contrast, two-stage algorithms insert an RPN to generate region proposals before classification and regression.

Anchor-based detectors remain predominant in object detection (Zhang et al., 2020; Chu et al., 2023; Ren et al., 2015; Redmon & Farhadi, 2018; Dai et al., 2016; Redmon & Farhadi, 2017; Wang et al., 2019); representative examples include YOLOv2–YOLOv5 (Zhang et al., 2020; Chu et al., 2023; Dai et al., 2016; Redmon & Farhadi, 2017), YOLOv7 (Wang, Bochkovskiy & Liao, 2023), RetinaNet (Lin et al., 2017), Single Shot MultiBox Detector (SSD) (Liu et al., 2016), and Efficient Object Detection (EfficientDet) (Tan, Pang & Le, 2020). Additionally, two-stage models—such as Faster R-CNN (Ren et al., 2015), Feature Pyramid Network (FPN) (Lin et al., 2017), Cascade R-CNN (Cai & Vasconcelos, 2018), and Trident-Net (Li et al., 2019)—also rely on anchor-based mechanisms.

Anchor-free approaches have likewise gained popularity and can be broadly divided into key-point-based and dense-prediction-based methods. Key-point-based detectors first locate the top-left and bottom-right corners of an object and then associate these points with offset vectors to form the prediction box. For instance, YOLOX (Ge et al., 2021), CenterNet (Duan et al., 2019), CornerNet (Law & Deng, 2018), ExtremeNet (Zhou, Zhuo & Krahenbuhl, 2019), YOLOv6 (Li et al., 2022), and YOLOv8 all utilize anchor-free key-point mechanisms. Dense-prediction methods, by contrast, directly classify each pixel (x, y) inside a candidate box as positive or negative. They subsequently regress four distances—from the point to the top, bottom, left, and right edges—to recover the bounding box. Notable dense-prediction, anchor-free detectors include Feature Selective Anchor-Free Module (FSAF) (Zhu, He & Savvides, 2019), Fully Convolutional One-Stage Object Detector (FCOS) (Tian et al., 2019), Fovea-Based Box Prediction Network (FoveaBox) (Kong et al., 2020), and Soft Anchor-Point Detection (SAPD) (Zhu et al., 2020). Both paradigms typically exhibit strong performance on small objects while imposing lower hardware requirements.

Rationale for baseline selection

Ensuring real-time detection within stringent hardware constraints is paramount in safety-critical domains, such as autonomous vehicles and advanced driver-assistance systems (ADAS). Traffic-sign detection exemplifies this challenge, as signs are typically small, may appear under adverse weather conditions, and necessitate immediate processing to ensure driver safety. Consequently, one-stage, anchor-free detectors are increasingly attractive, owing to their low latency and ease of deployment on embedded platforms.

Guided by these considerations, we adopt YOLOv8—an anchor-free, one-stage detector—as the baseline for our study. Its streamlined architecture and training strategies are leveraged to address small-object detection under resource constraints and in complex road environments. By incorporating targeted training schemes that account for variations in sign scale and orientation, the resulting TS-YOLO model offers a promising route toward higher detection accuracy and robustness in complex, real-world road scenarios.

Architecture

RepVGGBlock

Numerous convolutional neural network architectures utilize multi-branch designs to capture richer features, as observed in architectures like ResNet (He et al., 2016) and Inception (Szegedy et al., 2015). However, these designs may hinder parallelism and reduce inference efficiency. RepVGG (Ding et al., 2021) addresses this issue by training with multiple branches but consolidating them into a single branch during inference, thus maintaining strong representational capacity without incurring additional runtime overhead. In the training phase, RepVGG employs a 3 × 3 convolution, a 1 × 1 convolution, and an identity mapping. These components are re-parameterized into a single 3 × 3 convolution during inference.

Thus, RepVGG overcomes these limitations by utilizing a multi-branch configuration for training and transitioning to a single-branch architecture for inference. Specifically, during the inference phase, the network relies on a streamlined 3 × 3 convolutional structure integrating ReLU activations. The core concept of RepVGG lies in structural re-parameterization, where a multi-branch network trained in the training phase transitions into a single-branch during inference, thereby ensuring efficient performance without additional runtime overhead.

Figure 1 illustrates that the RepVGG network consists of a sequence of identical RepVGGBlocks. In the training phase, each block comprises a 3 × 3 convolution, a 1 × 1 convolution, and an identity mapping. During inference, the multiple branches are consolidated into a single convolutional operation, obviating the need for separate 1 × 1 convolutions and identity mappings. ResNet employs a residual connection to model information flow as $y=x+f\left(x\right)$, where $f\left(x\right)$ represents a 3 × 3 convolution for feature representation and $x$ denotes an identity mapping. This mechanism facilitates the acquisition of semantic features. Building on this concept, the RepVGGBlock integrates ResNet’s residual connection and enhances it with a 1 × 1 convolutional branch, yielding three distinct convolutional branches for the shortcut connection. The information flow in RepVGGBlock is expressed as $y=x+g\left(x\right)+f\left(x\right)$, where $g\left(x\right)$ represents the shortcut connection via a 1 × 1 convolution.

Network structure

The RepVGGBlock utilizes model reparameterization to merge multi-branch convolutions into a single-branch convolutional network. During training, the kernel parameters from three convolutional branches are merged into those of a single branch, which is subsequently used for inference and prediction. This approach enables a seamless transition from training to inference phase.

Inspired by CSPNet (Wang et al., 2020), YOLOv8 integrates the cross stage partial (CSP) architecture into its backbone and neck structures and incorporates the CSP Bottleneck with 2 Convolutions (C2F) module. This module inherited the cross-stage partial connection design from CSPNet by splitting the feature map into two branches (main path and shortcut path). It enables cross-stage gradient flow bifurcation, effectively alleviating the vanishing gradient problem and enhancing the deep network’s feature representation capability.

More specifically, the RepVGGBlock concept is embedded into the CSP module, yielding the RepCSP block, which merges multi-branch convolutions into a single branch during inference. The RepVGGBlock enhances model training through structural re-parameterization and can subsequently be fused into a single branch during inference, thereby reducing the number of parameters and computational load while increasing inference speed. The CSP module facilitates the extraction of rich semantic features. By integrating the RepVGGBlock into the CSP module and simplifying the CSP block structure, the model’s detection performance is enhanced. The structural diagram of the RepCSP block is presented in Fig. 2.

To systematically evaluate the feature enhancement mechanism of the RepCSP module, this study employed Grad-CAM++ (Chattopadhay et al., 2018) as an attention visualization framework. As illustrated in Fig. 3, compared to the YOLOv8 baseline, TS-YOLO exhibits enhanced activation responses in the edge regions of traffic signs within its attention heatmaps, indicating improved spatial focus on critical target areas. This enhanced feature representation can be attributed to the multi-scale feature preservation achieved by the re-parameterized branch, where structural re-parameterization facilitates cross-layer feature integration through learnable connection pathways.

The final architecture of TS-YOLO is presented in Fig. 4, with modified components labeled using “△”. All C2F modules in DarkNet-53 and the neck are replaced with RepCSP blocks. The proposed RepCSP-based backbone architecture enables the network to learn discriminative features more effectively while preserving structural simplicity for deployment. Figure 5 presents the visualization of feature maps extracted by TS-YOLO.

The RepCSP block incorporated into TS-YOLO employs a re-parameterization design. During training, the block comprises three branches: a 3 × 3 convolution, a 1 × 1 convolution, and an identity connection. Although this configuration enhances the representational capacity of the model, it introduces additional computational overhead, as evidenced by increased parameter counts and GFLOPs. During inference, re-parameterization merges the weights of the multi-branch structure into a single convolutional kernel, effectively converting the block into a single 3 × 3 convolutional layer and eliminating the overhead associated with the auxiliary branches. At this stage, the executed computational path, in comparison to the C2F block of YOLOv8, entails no redundant operations. After the RepCSP block is re-parameterized, the TS-YOLO model exhibits higher theoretical GFLOPs than YOLOv8 (see Ablation Studies); nevertheless, its practical inference efficiency improves owing to the following factors:

• (1)

  Branch fusion reduces memory traffic: multi-branch structures require multiple read and write operations of intermediate feature maps, whereas a single-branch implementation completes the computation in a single pass. Because memory-access latency frequently constitutes the primary bottleneck on modern hardware—particularly graphics processing units (GPUs)—its reduction directly reduces inference time.

• (2)

  A single-branch 3 × 3 convolution is more amenable to optimization by low-level acceleration libraries (e.g., cuDNN), thereby achieving higher computational throughput that compensates for the marginal increase in theoretical GFLOPs.

Consequently, while the pre-fusion multi-branch structure inflates the theoretical computational load (GFLOPs), the fused single-branch variant exhibits, during inference, a more compact computational graph, higher operational density, and a superior memory-access pattern. These characteristics allow it to match or even surpass the inference speed of ostensibly simpler models that lack such optimizations. In essence, re-parameterization exchanges structural complexity during training for enhanced efficiency during inference.

Training tricks

Experiments settings

The experimental setup employs the Windows 11 operating system (OS), with computations performed using an NVIDIA GeForce RTX 4060Ti 16GB GPU. The framework employed is PyTorch, with torch version 2.1.2, CUDA version 12.1, and Python version 3.9.

In this study, the YOLOv8n variant was employed under strictly controlled conditions to ensure methodological consistency. For training, an input resolution of 640 × 640 was used, along with a batch size of 32, the Sigmoid Linear Unit (SiLU) activation function, and eight parallel data-loading workers. Additionally, online data augmentation was applied, including adjustments to hue, saturation, and value, as well as translation, scaling, and mosaic techniques (Bochkovskiy, Wang & Liao, 2020). Complete hyperparameter specifications, including data augmentation parameters and ablation study settings, are systematically documented in Table 1.

Dataset

We utilized the publicly available CCTSDB2021 traffic sign detection dataset (Zhang et al., 2022a, 2022b, 2022c), which contains 17,856 images and 30,065 traffic signs (as shown in Table 2). To ensure reproducibility, the dataset was randomly partitioned into training, test, and validation sets at an 8:1:1 ratio. This dataset comprises three primary traffic sign categories: prohibitory, warning, and mandatory. The distribution of instance counts across the three categories (prohibitory, warning, and mandatory) is presented in Table 3. Furthermore, all images were captured under typical road conditions in China. These images, recorded from multiple angles in real-world driving scenarios (e.g., from within a vehicle or via dashcams), enhance the dataset’s authenticity and practical relevance. Moreover, the CCTSDB2021 dataset, collected from multiple perspectives, encompasses challenging data categories, including adverse weather conditions (fog, snow, rain, night, sunny, and cloudy), low-light environments, complex occlusion scenarios, and ultra-small object instances, thereby comprehensively covering edge cases for environmental perception in autonomous vehicles.

Evaluation metrics

The proposed method is evaluated from three perspectives: model complexity, detection accuracy, and inference speed.

Model complexity is measured by the number of parameters and floating-point operations (GFLOPs).

Detection accuracy is quantified using mean Average Precision (mAP), Average Precision (AP), and precision.

• •

  [email protected]: the mean AP across all classes at an IoU threshold of 0.5;

• •

  [email protected]: the mean AP computed over ten IoU thresholds ranging from 0.5 to 0.95 with a step of 0.05;

• •

  [email protected] and [email protected] denote the AP under the respective IoU settings.

Inference speed is represented by the inference time per image.

Intersection over Union (IoU) quantifies the overlap between a predicted bounding box (P) and its ground-truth counterpart (G), defined as: $IoU\left(P,G\right)={\displaystyle \frac{\left|\mathrm{P}\cap \mathrm{G}\right|}{\left|\mathrm{P}\cup \mathrm{G}\right|}}$, where ∩ and ∪ denote the intersection and union operators, respectively. IoU values range from 0 to 1, with higher values indicating greater overlap and thus more accurate predictions.

Ablation studies

In this subsection, we evaluated the significance of our TS-YOLO model through a series of ablation studies (Table 4), which assessed the impact of each component. We further illustrated how the performance of our object detector can be progressively enhanced by sequentially integrating each optimized module, thereby validating the effectiveness of these improvements.

A→ B: Model B was first obtained by refining the original YOLOv8 (hereafter Model A) via epoch optimisation. By default, YOLOv8 is trained for 500 epochs. To mitigate over-fitting, we adopted a YOLOX-inspired strategy that disables data augmentation during the last ten epochs. We therefore carried out a series of experiments with total training epochs ranging from 300 to 700 to identify the optimal training duration and augmentation cut-off. As summarised in Table 5, training for 600 epochs and discontinuing data augmentation at epoch 570 yielded the best overall performance. Under these settings, [email protected] reached 98.2%, an improvement of 0.1 percentage points over the baseline configuration (500 epochs, augmentation disabled during the last ten epochs). Similarly, [email protected] increased to 78.7%, reflecting a 0.7 percentage-point gain. Figure 6 contrasts the training performance before and after epoch optimisation, showing that the [email protected] curve benefits significantly from the revised strategy.

B→ C: Subsequently, we validated the efficacy of the optimised network architecture using the epoch-optimised Model B as the experimental baseline. Because the architecture had changed, we omitted pre-training and trained the model from scratch. The resulting model is referred to as Model C (TS-YOLO), which incorporates the architectural refinements. As reported in Table 4, [email protected] increased to 98.4% (+0.2%) and [email protected] rose to 79.1% (+0.4%) after applying the architectural refinements. Relative to Model A, TS-YOLO improves [email protected] by 0.3 percentage points and [email protected] by 1.1 percentage points. Notably, these gains incur only 3.51 M additional parameters and 2.3 GFLOPs, while the inference speed remains 0.6 ms per image. Both TS-YOLO and YOLOv8 achieve the same 0.6 ms inference time on an NVIDIA RTX 4060 Ti. This parity confirms the practical efficacy of the single-branch computation realised through re-parameterising the RepCSP block. Theoretical GFLOPs quantify the static number of floating-point operations (Redmon & Farhadi, 2018), whereas actual inference speed depends on dynamic factors such as hardware parallelism, memory bandwidth, and computational paths (e.g., branch count) (Jegham et al., 2024). Therefore, the TS-YOLO results illustrate how re-parameterisation reconciles theoretical and practical efficiency.

Figure 7 visually compares the mAP training curves and highlights the enhanced convergence of the optimized model. Precision is likewise essential: as shown in Fig. 8, YOLOv8 and TS-YOLO achieve final precisions of 96.4% and 97.3%, respectively, corroborating the effectiveness of the improvements. Overall, the smoother training curves underscore the improved convergence behaviour attained through these optimizations.

C→D: Following the optimization of the number of training epochs and the network architecture, we compared Model D (pretrained on ImageNet) with Model C (randomly initialized). The experimental results indicate that a network trained for 600 epochs from random initialization surpasses its counterpart initialized with pretrained weights. Specifically, the non-pretrained model attains an [email protected] that is 0.1 percentage points higher. These observations imply that pretraining chiefly accelerates early-stage convergence but confers no benefit on final accuracy. Because ImageNet pretraining emphasizes global semantic classification rather than spatial localization, its contribution to object detection tasks, in which precise spatial prediction is indispensable, is inherently constrained.

Results validated through ablation studies indicate considerable improvements in traffic sign detection. Specifically, the proposed method achieves an [email protected] of 98.4% and an [email protected] of 79.1%, processes a frame in only 0.6 ms, and maintains a compact model size of 14.98 M parameters with a computational cost of 8.1 GFLOPs. The proposed architecture is well-suited to in-vehicle embedded platforms, delivering real-time inference at 30 FPS even under resource-constrained conditions.

Table 6 compares the AP of three traffic-sign categories—prohibitory, warning, and mandatory—before and after the proposed improvements. As illustrated, the enhanced TS-YOLO surpasses the baseline model in every subcategory, yielding higher [email protected] and [email protected] values.

Figure 9 presents a comprehensive performance comparison between TS-YOLO and the baseline YOLOv8 across three categories of challenging scenarios:

• (1)

  Adverse weather conditions, including rain (Figs. 9K, L), low-light nighttime scenes (Figs. 9E–H), and fog (Figs. 9I, J);

• (2)

  Diverse road types, comprising highways (Figs. 9A, B), urban streets (Figs. 9C–F), and national highways (Figs. 9G, H);

• (3)

  Complex target characteristics, namely small objects, occlusions, deformed signs, and motion-blurred instances.

The results in Fig. 9 confirm that, compared with YOLOv8, TS-YOLO reduces false detections and slightly increases overall accuracy. Notably, TS-YOLO maintains more stable accuracy in adverse weather and lowers the false-positive rate for small or occluded targets in complex road scenes, even though the visual differences between the two models are subtle.

Comparisons

Under identical training configurations (batch size of 32, 600 epochs, and random weight initialization), TS-YOLO was benchmarked against thirteen mainstream detectors, including YOLOv3–YOLOv8, YOLOX, EfficientDet-d0, SSD, RetinaNet, Faster R-CNN, and CenterNet. The minimal variants of YOLOv5, YOLOv6, and YOLOX (i.e., the “n” versions) were selected to align with YOLOv8. As shown in Table 7, TS-YOLO achieves the highest accuracy among all evaluated models, attaining a [email protected] of 98.4% and a [email protected] of 79.1%, which exceed those of YOLOv8 by 0.3% and 1.1%, respectively. While TS-YOLO’s parameter count (14.98 M) and computational requirement (10.4 GFLOPs) are marginally greater than those of the YOLOv8 baseline (11.47 M, 8.1 GFLOPs), they remain an order of magnitude lower than those required by YOLOv7 (141.93 M, 105.1 GFLOPs), achieving 0.5% higher [email protected] and 0.8% higher [email protected] with only one-tenth the resources. When compared with the lightweight YOLOv5 (6.74 M, 4.2 GFLOPs) and YOLOv6 (4.30 M, 11.06 GFLOPs) variants, TS-YOLO incurs only a modest increase in model complexity, yet yields gains of approximately 0.8–0.9% in [email protected] and 2.9–3.4% in [email protected], thereby demonstrating an improved accuracy–efficiency trade-off. Against traditional two-stage and anchor-based detectors—SSD (26.23 M, 62.75 GFLOPs, 58.4%/24.7%), RetinaNet (37.97 M, 170.09 GFLOPs, 21.2%/15.9%), and Faster R-CNN (137.10 M, 370.21 GFLOPs, 72.1%/35.1%)—TS-YOLO achieves an order-of-magnitude lower computational load, while providing improvements of at least 26.3% in [email protected] and 43.2% in [email protected]. Furthermore, TS-YOLO surpasses CenterNet (32.67 M, 70.22 GFLOPs, 86.1%/39.7%) by 12.3% in [email protected] and 39.4% in [email protected], while utilizing less than half of the computational resources. These findings confirm that TS-YOLO offers a superior compromise between model size, inference speed, and detection accuracy, rendering it particularly suitable for real-time, resource-constrained in-vehicle applications and complex roadway environments.