Steel surface defect detection based on multi-scale dynamic convolution and lightweight cross-stage fusion

Overview

This section proposes steel surface defect detection based on multi-scale dynamic convolution and lightweight cross-stage fusion (LightSDD). Figure 1 illustrates the LightSDD network architecture. Firstly, the C3k2_PKI (Khanam & Hussain, 2024), which is built upon dynamic multi-scale perception, is meticulously designed to construct a multi-level feature pyramid network. Shallow high-resolution features (1/8 scale) are adept at retaining detailed textures. Middle-level features (1/16 scale) strike a balance between semantics and positioning accuracy. Deep low-resolution features (1/32 scale) focus on capturing global context. The module effectively facilitates adaptive matching of cross-scale receptive fields by integrating multi-scale convolutional kernels (Qi & Jing, 2022) (varying from 3 × 3 to 11 × 11) with a dynamic weight fusion mechanism. This, in turn, enhances the model’s generalization capabilities in addressing defects of diverse dimensions. Subsequently, the YOLOv11 detection head undergoes substitution with the LSCD head. This head innovatively adopts channel compression (Cheng et al., 2021) and cross-stage fusion (Wang et al., 2024) strategies. These strategies reduce parameters and elevate positioning accuracy, catering to the stringent robustness demands inherent in industrial quality inspection scenarios. Finally, these two components are integrated into the YOLOv11 framework to form the end-to-end LightSDD architecture, achieving an optimal balance between multi-granularity feature representation and computational efficiency. The pseudocode of the LightSDD is presented in Algorithm 1.

In Algorithm 1, Step 3 involves the backbone feature extraction stage. Here, the input is an industrial quality inspection image ( $x\in R^{3\times H\times W}$). The backbone network extracts multi-scale features $(F_{P3}\in R^{\mathrm{C}\times \frac{H}{8}\times \frac{W}{8}}$, $F_{P4}\in R^{\mathrm{C}\times \frac{H}{16}\times \frac{W}{16}}$ and $F_{P5}\in R^{\mathrm{C}\times \frac{H}{32}\times \frac{W}{32}})$. In this context, H and W represent the height and width of the original image, and C denotes the feature dimension. $F_{P3}$, $F_{P4}$ and $F_{P5}$ correspond to shallow high-resolution, middle mid-resolution, and deep low-resolution features, respectively.

Steps 5–11 form the dynamic convolution optimization module. To perform parallel multi-scale kernel extraction on the features, $F_{\mathrm{s}}$(s ∈ {P3, P4, P5}) from the previous layer, 3 × 3, 5 × 5, 7 × 7, 9 × 9, and 11 × 11 convolution kernels are applied to generate multi-branch features $F_s^k$. Subsequently, the squeeze-and-excitation (SE) module produces branch weights, $a_k$ = [ $a_1$, $a_2$, $a_3$, $a_4$, $a_5$], and a weighted summation is performed to achieve dynamic weight fusion, updating the features $\widetilde{F_s}$ of the current scale.

Steps 14–17 describe the lightweight cross-stage detection head. First, $\widetilde{F_{P4}}$ and $\widetilde{F_{P5}}$ are upsampled via bilinear interpolation to align their resolutions of $\widetilde{F_{P3}}$, yielding $F_{P4}^{up}$ and $F_{P5}^{up}$. Subsequently, A 1 × 1 convolution is then used to compress the channels of the upsampled features, unifying them to 256 dimensions, resulting in $F_{P4}^{compressed}$ and $F_{P5}^{compressed}$.

Steps 19–23 focus on feature fusion and enhancement. Initially, the $F_{P3}$, $F_{P4}^{compressed}$ and $F_{P5}^{compressed}$ are concatenated to generate fused features ( $F_{concat}$). Subsequently, a lightweight transformer is employed to model long-range dependencies, thereby producing enhanced features ( $F_{trans}$). The process utilizes group convolution ( ${GroupConv}_{3\times 3}$) to compress $F_{trans}$ through channel grouping. Finally, dynamic channel recalibration is performed using an SE module, resulting in the final features ( $F_{final}$).

Steps 25–29 address dynamic parameter adjustment. Specifically, if the model operates on edge devices and the average forward inference latency exceeds the predefined threshold, the 11 × 11 kernel is downgraded to a 7 × 7 kernel. Additionally, low-sensitivity channels are pruned based on gradient sensitivity analysis (Wang et al., 2018) to minimize computational overhead.

Steps 31–32 outline the model output stage. In this stage, the detection layer predicts the final feature ( $F_{final}$) to generate bounding box predictions (bbox_pred) and class predictions (cls_pred).

C3k2 poly kernel inceptional design

1) Fine-grained feature fusion via multi-scale dynamic convolution. The C3k2 poly kernel inceptional (C3k2_PKI) incorporates the idea of multi-scale convolutional kernels, performing fine-grained convolution operations on different scales of the input image to extract rich feature representations. Given an input feature map, $F_s\in R^{3\times H\times W}$(s $\in$ {P3, P4, P5}), the module initially applies parallel multi-scale convolutions, as shown in Eq. (1).

(1) $$F_s^k=Con{v}_{k\times k}\left(F_s\right),k\in K=\left\{3,5,7,9,11\right\}$$where, $Con{v}_{k\times k}$ denotes a $k\times k$ convolutional kernel, where each convolutional branch captures both local structural and global semantic information at its corresponding scale.

After obtaining multi-scale features, the method designs a dynamic weight computation mechanism to avoid information redundancy caused by simple concatenation or summation operations. It introduces a lightweight attention module (Wu et al., 2025) to generate adaptive weights. First, global average pooling (GAP) (Jin et al., 2022) aggregates the output of each scale branch, as shown in Eq. (2).

(2) $$z_s=GAP\left(F_s^k\right)={\displaystyle \frac{1}{H_s{W}_s}\sum _{i=1}^{H_s}\sum _{j=1}^{W_s}{F}_s^k\left[:,i,j\right]\in R^{C_s}}$$where, $H_s$ and $W_s$ represent the height and width of the feature maps at different levels.

Next, a series of fully connected layers (Basha et al., 2020) and nonlinear activation functions (Hsiao et al., 2019) transform these statistics to generate corresponding weight vectors, where $\delta$ denotes the ReLU activation function, as shown in Eq. (3).

(3) $$\mathit{u}=\delta \left({\mathit{W}}_1{z}_s+{\mathit{b}}_1\right).$$

Subsequently, the softmax function normalizes the weights of each branch to ensure learnable and dynamically adjusted contribution ratios across multi-scale features, as shown in Eq. (4).

(4) $${\mathit{a}}_{\mathit{k}}=Softmax\left({\mathit{W}}_2\mathit{u}+{\mathit{b}}_2\right)$$where, ${\mathit{W}}_1\mathrm{a}\mathrm{n}\mathrm{d}{\mathit{W}}_2$ are fully connected layer weights of the SE module, ${\mathit{b}}_1$ denotes the bias of the channel attention intermediate layer, and ${\mathit{b}}_2$ corresponds to the bias of the channel attention output layer, satisfying ${\sum }_{k\in K}{a}_k=1$.

Finally, the method performs weighted summation on multi-scale features according to their corresponding weights, producing refined fused features, as formulated in Eq. (5).

(5) $$\widetilde{F_s}=\sum _{k\in K}{a}_k\cdot F_s^k.$$This method enables cross-resolution feature interaction (Lu et al., 2025) and enhances small-object sensitivity, effectively allowing the model to learn objects at various scales.

2) Dynamic parameter control mechanism. First, the input feature analysis performs global average pooling on the multi-scale feature maps (P3, P4, P5) output by the backbone, generating channel descriptor vectors as shown in Eq. (6), where $C_s$ denotes the number of channels.

(6) $$v_s=GAP\left(\widetilde{F_s}\right)\in R^{c_s}.$$

The scale-specific descriptor vectors are subsequently concatenated to form a unified multi-scale feature representation, as formulated in Eq. (7).

(7) $$v_{global}=v_{P_3}\oplus v_{P_4}\oplus v_{P_5},C_{sum}=C_{P_3}+C_{P_4}+C_{P_5}.$$

This vector is fed into a two-layer fully connected network multilayer perceptron (MLP, with hidden layer dimension $\frac{C}{2}$), outputting a score vector S_k = [ $S_1,S_2,S_3$], representing the saliency scores for small, medium, and large targets, respectively. As shown in Eq. (8):

(8) $$S_k=X_3\cdot \delta \left(X_2\cdot v_{global}+d_1\right)+d_2.$$where, ${\mathit{X}}_2$ denotes the weight matrix of the MLP hidden layer, ${\mathit{X}}_3$ represents the weight matrix of the MLP output layer, ${\mathit{d}}_1$ is the bias term of the global feature hidden layer, and ${\mathit{d}}_2$ is the bias term of the importance score output layer.

Finally, the scores undergo normalization via the softmax function to constrain their distribution, as shown in Eq. (9):

(9) $$\widehat{s_k}={\displaystyle \frac{e^{S_k}}{{\sum }_{i=1}^3{e}^{S_i}},k\in \{1,2,3\}}$$where, $\widehat{s_1}$, $\widehat{s_2}$, $\widehat{s_3}$ represent the importance weights for small, medium, and large targets, satisfying ${\sum }_{k=1}^3\widehat{s_k}=1$.

The parameter dynamic adjustment adaptively selects convolution kernel sizes and computational intensity based on the importance weights $s_k$ and the real-time workload. If the inference latency exceeds a threshold, the 11 × 11 kernel is automatically replaced with a 7 × 7 kernel, as shown in Eq. (10):

(10) $$k_{max}=\{\begin{array}{cc}11,\hfill & {\tau }_{latency}\le {\tau }_{threshold}\hfill \\ 7,\hfill & otherwise\hfill \end{array}.$$The structure of the C3k2_PKI is illustrated in Fig. 2.

Lightweight shared convolutional detection head design

1) Cross-stage feature fusion. First, the feature maps P3 (1/8 scale), P4 (1/16 scale), and P5 (1/32 scale) are extracted from the backbone, with channel numbers C₃ = 256, C₄ = 512, and C₅ = 1,024, respectively.

Then, bilinear interpolation is applied to P4 and P5 to upscale their resolution to match P3 (1/8 scale). A 1 × 1 convolution is used to compress the channels of P4 and P5 to 256, aligning them with P3, as shown in Eq. (11).

(11) $$F_s^{compressed}=Con{v}_{1\times 1}(\widetilde{F_s},C_{out}=256).$$

Finally, the aligned feature maps are output as ${\mathit{F}}_{\mathit{P}3}$, $F_{P4}^{compressed}$ and $F_{P5}^{compressed}$.

The bidirectional feature interaction consists of a bottom-up path and a top-down path, which model cross-stage dependencies through a lightweight attention mechanism (Xu et al., 2025). In the bottom-up path, the P3 features are concatenated with the compressed P4 and P5 features to generate a global context vector, as shown in Eq. (12).

(12) $${\mathit{F}}_{\mathit{c}\mathit{o}\mathit{n}\mathit{c}\mathit{a}\mathit{t}}={\mathit{F}}_{\mathit{P}3}\oplus F_{P4}^{compressed}\oplus F_{P5}^{comprssed}.$$

Long-range dependencies are modeled using a lightweight transformer encoder (four attention heads, hidden layer dimension of 128), as shown in Eq. (13).

(13) $${\mathit{F}}_{\mathit{t}\mathit{r}\mathit{a}\mathit{n}\mathit{s}}=\mathrm{T}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{s}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{e}\mathrm{r}({\mathit{F}}_{\mathit{c}\mathit{o}\mathit{n}\mathit{c}\mathit{a}\mathit{t}}).$$

The top-down path splits F_trans into three components, which are then element-wise added to the original P3, P4, and P5 features, respectively, for information feedback, as formulated in Eq. (14).

(14) $${\mathit{F}}_{\mathit{s}}^{\mathit{e}\mathit{n}\mathit{h}\mathit{a}\mathit{n}\mathit{c}\mathit{e}\mathit{d}}={\mathit{F}}_{\mathit{s}}+Slices({\mathit{F}}_{trans}),s\in \{3,4,5\}.$$The channel compression and enhancement strategy retains key information by reducing redundant channels. Dimensionality reduction is performed via group convolution, where the concatenated features ${\mathit{F}}_{\mathit{c}\mathit{o}\mathit{n}\mathit{c}\mathit{a}\mathit{t}}$ ∈ $R^{H\times W\times 768}$ are compressed to 128 channels using group convolution (groups = 8), as formulated in Eq. (15).

(15) $$F_{fused}=\mathrm{G}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}3\times 3(F_{trans},\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}\mathrm{s}=8).$$

The process employs dynamic channel recalibration through a lightweight Squeeze-and-Excitation (SE) module to adaptively adjust channel-wise weights, as formulated in Eq. (16).

(16) $${\mathit{F}}_{final}=\sigma \left(MLP\left(GAP\left(F_{fused}\right)\right)\right)\cdot {\mathit{F}}_{fused}.$$

2) Parameters optimization. For each output channel of each convolutional layer in the detection head, calculate the mean of the absolute values of the weight gradients, as shown in Eq. (17).

(17) $$\mu ={\displaystyle \frac{1}{C}\sum _{c=1}^C|\text{▽}{W}_c|,C=C_s.}$$Here, C represents the number of channels in the current layer, and $\text{▽}{W}_c$ denotes the weight gradient of the C-th channel. Channels with gradient sensitivity below the mean μ are pruned, retaining only the important ones.

Finally, channels in the LSCD detection head with gradient magnitudes below the mean are sparsified. The retention rate r is computed as shown in Eq. (18).

(18) $$r=1-{\displaystyle \frac{{\sum }_{c=1}^C\mathrm{I}\mathrm{I}(|\text{▽}|<\mu )}{C}.}$$

The structure of the used LSCD is illustrated in Fig. 3.

Experimental setup, dataset, and evaluation metrics

This study conducts experiments on the Windows 11 operating system, utilizing Python 3.10.16 and PyTorch 2.5.1 as the deep learning framework and PyCharm as the development environment. The hardware consists of an Intel Core i5-13500H CPU and an RTX 4050 GPU with 6 GB of VRAM, utilizing CUDA 12.4 for GPU acceleration. The hyperparameter configuration for training the LightSDD model is shown in Table 1.

This article adopts the NEU-DET and GC10-DET datasets for testing. The NEU-DET steel defect dataset consists of six types of defects: crazing, inclusion, patches, pitted surface, rolled-in scale, and scratches, with 300 images per category, totaling 1,800 images. The GC10-DET steel surface defect dataset includes ten types of defects: punching, welding line, crescent gap, water spot, oil spot, silk spot, inclusion, rolled pit, crease, and waist folding. NEU-DET can be downloaded at http://faculty.neu.edu.cn/songkechen/zh_CN/zdylm/263270/list. GC10-DET can be downloaded at https://github.com/lvxiaoming2019/GC10-DET-Metallic-Surface-Defect-Datasets. During the experiment, a five-fold cross-validation is employed to establish training sets, test sets, and validation sets for both the NEU-DET and GC10-DET datasets, ensuring rigorous and objective verification results.

This article employs Precision, Recall, mean Average Precision (mAP), Parameters, and Floating Point Operations (FLOPs) as metrics to evaluate model performance. Specifically, mAP50 denotes the mean average Precision calculated at an Intersection over Union (IoU) threshold of 0.5, while mAP50~95 represents the average of multiple mAP values obtained at IoU thresholds ranging from 0.5 to 0.95. Parameters refer to the total count of learnable parameters that require optimization during model training. In contrast, FLOPs quantify the computational complexity of the model by measuring the total number of floating-point operations required for a single forward propagation.

Ablation study

To evaluate the performance of different modules in LightSDD, YOLOv11 is used as the baseline (denoted as S0), and an ablation study is conducted on the NEU-DET dataset using the configuration specified in Table 1. The performance metrics of the ablation experiments are summarized in Table 2.

From the observations in Table 2:

• (1)

  The C3k2_PKI module adaptively integrates features across multiple scales, enhancing the model’s perception capability for micro-defects and complex backgrounds. As a result, S1 improves mAP50 by 0.29%. However, due to the increased computational cost from using larger convolutional kernels, mAP50~95 experiences a slight decrease of 0.35%.

• (2)

  S2 reduces parameters by 6.20%, demonstrating that the lightweight cross-stage detection head (LSCD) effectively reduces redundant parameters through channel compression and group convolution, while preserving detailed information in cross-resolution features. This optimization improves detection accuracy.

• (3)

  S3 (LightSDD) achieves more robust feature representation through collaborative optimization while maintaining parameters at 2.42M. Although mAP50 shows a slight decline (78.00% → 76.81%) due to the reallocation of computational resources, LightSDD achieves the highest mAP50 of 44.86%, marking a 0.16% improvement over the baseline S0, and demonstrates strong performance in key defect categories.

• (4)

  The change in mAP50 of LightSDD is statistically significant, indicating a systematic shift in model behavior under relaxed thresholds. However, the change in mAP50-95 does not reach statistical significance, which may be attributed to interaction effects between modules.

Therefore, we evaluate alternative design schemes to determine the contributions of model components. This article conducts experimental comparisons of different attention mechanisms in LSCD.

As shown in Table 3, the SE attention mechanism employed in LSCD consistently delivers leading performance across core detection metrics. Its mAP50-95 reaches 44.86%, outperforming GlobalPool (43.33%), ChannelMax (44.27%), and StaticSpatial (42.96%), while demonstrating optimal stability with a standard deviation of ±0.33%. In terms of comprehensive evaluation metrics, LSCD achieves the highest F1-score (71.55%), reflecting the best balance between precision and recall. Although ChannelMax exhibits a slightly higher recall rate (73.32%) compared to LSCD (72.70%), LSCD validates the effectiveness of its structural design through superior overall accuracy and stability.

As shown in Table 4, in terms of detection accuracy, although the large-kernel version achieves a slightly higher mAP50 (77.41%), LightSDD reaches the highest value of 44.86% on the stricter mAP50-95 metric, with the smallest standard deviation (±0.33%), demonstrating superior detection stability. Regarding computational efficiency, LightSDD maintains a relatively low parameter count (2.42M) and computational cost (6.90G FLOPs) while achieving an optimal balance between accuracy and efficiency. Although the small-kernel version has the lowest computational cost, its lower mAP50-95 (43.74%) indicates that excessive lightweighting may compromise model performance.

Additionally, to visually observe the performance changes and states of the model during training and validation, loss curves (Fig. 4) and mAP curves (Fig. 5) are plotted. As shown in Fig. 4, LightSDD (i.e., S3) demonstrates significantly faster loss reduction than S0, indicating a more rapid convergence. The loss curve of LightSDD also exhibits smoother fluctuations, indicating enhanced model stability and reduced risk of overfitting. As shown in Fig. 5, the mAP curve shows that S3 achieves metric improvements more quickly than S0, and the overall curve is smoother, reflecting enhanced model stability.

Finally, we observe the Grad-CAM heatmaps of six types of defects in the NEU-DET dataset, as shown in Fig. 6. Figure 6 indicates that LightSDD provides more comprehensive and accurate identification of key defect features.

In conclusion, both C3k2_PKI and LSCD have played positive roles. The feature fusion strategy of LightSDD, which combines C3k2_PKI and LSCD, has a synergistic effect and possesses the comprehensive detection capability for targets of different scales.

Performance comparison

Seven object detection models, namely YOLOv3-tiny, YOLOv5n, YOLOv6n, YOLOv8n, YOLOv9t, YOLOv10n and YOLOv11, are selected as comparative methods. A five-fold cross-validation experiment is conducted to validate the advantages of the improved model proposed in this article. The test results on both the NEU-DET and GC10-DET datasets are presented in Table 5.

As shown in Table 5, LightSDD achieves the highest mAP50~95 of 44.86% on the NEU-DET dataset among all compared models. With only 2.42M parameters, it reduces the parameter count by 41.80% compared to YOLOv6n (4.16M), which has the closest detection accuracy. Its computational cost (FLOPs = 6.90G) is also 40.00% lower than that of YOLOv6n (11.50G), demonstrating that the lightweight design significantly optimizes resource usage while maintaining competitive accuracy. Compared to YOLOv11, LightSDD improves mAP50~95 by 0.16% (44.70% → 44.86%), although mAP50 decreases by 1.19% (78.00% → 76.81%). This indicates that the model offers stronger robustness under stricter IoU thresholds, while the slight drop in mAP50 may result from reduced classification confidence due to lightweight compression.

Meanwhile, LightSDD demonstrates comprehensive performance advantages on the GC10-DET dataset, achieving the highest mAP50 (64.92%) and mAP50~95 (33.39%), as well as the best Recall (63.58%). Although its Precision (66.89%) is slightly lower than the top-performing YOLOv11 (68.38%), it still surpasses all other compared methods. More importantly, LightSDD shows the lowest standard deviation in both mAP50~95 and Recall, highlighting its superior stability and robustness. In terms of model efficiency, LightSDD maintains the lowest number of parameters (2.42M) and a relatively low computational cost (FLOPs = 6.90 G), further confirming its high computational efficiency.

Additionally, experiments repeatedly conducted on both the NEU-DET and GC10-DET datasets not only validate the stability and generalization capability of LightSDD across different defect distributions and complex scenarios, but also demonstrate that its performance improvement does not merely represent minor enhancements to YOLO-based baseline models. Instead, it reflects a systematic breakthrough in multi-scale feature modeling and lightweight optimization. By consistently achieving higher mAP50–95 and recall rates across diverse data characteristics, LightSDD proves its robustness and scalability, providing transferable technical references for industrial defect detection research under computational constraints. We also document the training durations from five-fold cross-validation experiments in Table 6 to facilitate the reproduction of our work by other researchers.

In summary, LightSDD achieves an effective balance between accuracy and efficiency in steel surface defect detection tasks, offering leading comprehensive accuracy, high stability, and notable lightweight characteristics.

Performance testing on edge devices

To evaluate the performance of the proposed LightSDD on edge devices, this section deploys LightSDD to the mobile terminal Samsung Galaxy S23 Ultra using the NCNN framework. The basic configuration of Samsung Galaxy S23 Ultra is as follows: Snapdragon 8 Gen 2 processor, 8GB RAM, 256GB ROM, and Android 16.0 operating system. First, the model weights trained under the PyTorch framework are converted into a format compatible with the NCNN framework. Subsequently, inference code is developed in C++ based on the NCNN inference framework. Finally, the APK file is built and generated using Android Studio. The development tools used during the deployment process are listed in Table 7.

After installing the LightSDD package compiled by Android Studio on the Samsung Galaxy S23 Ultra, 30 image samples were randomly selected from the NEU-DET test set for verification, with five samples per category. The test results are shown in Table 8, and the detection effects are illustrated in Fig. 7. As shown in Table 8, LightSDD exhibits excellent inference efficiency across various input resolutions, achieving near real-time detection performance that meets the stringent speed requirements of industrial production lines. Furthermore, the model’s memory usage during operation remains consistently below 5MB, indicating that its lightweight design effectively controls memory overhead, aligning well with the resource-constrained nature of edge devices. From the detection examples shown in Fig. 7, it can be shown that the model accurately identifies and locates all six types of defects in the NEU-DET dataset. These results demonstrate superior detection effectiveness, indicating that the model possesses strong robustness and practical applicability.