MDMU-Net: 3D multi-dimensional decoupled multi-scale U-Net for pancreatic cancer segmentation

Overall structure of MDMU-Net

Figure 1 illustrates the overall architecture of the proposed MDMU-Net, which is based on a U-shaped design and divided into four levels, encompassing both the encoder and decoder components. Given a set of 3D image volumes ${\mathrm{V}}_{\mathrm{i}}={\mathrm{X}}_{\mathrm{i}},{\mathrm{Y}}_{\mathrm{i}},\left(\mathrm{i}=\mathrm{1...}\mathrm{L}\right)$, random sub-volumes ${\mathrm{P}}_{\mathrm{i}}\in {\mathcal{R}}^{\mathrm{C}\times \mathrm{D}\times \mathrm{H}\times \mathrm{W}}$ are extracted as encoder inputs, where L denotes the total number of samples, C represents the number of channels, and D, H, W correspond to the depth, height, and width dimensions respectively. Initially, the input feature map is mapped to 32 dimensions through a convolutional layer with a stride of 2, a kernel size of 7 × 7 × 7, and padding of 3, laying the foundation for subsequent feature extraction. During the encoding phase, the down-sampling module reduces the resolution of the feature map using a 2 × 2 × 2 convolutional kernel with a stride of 2. Simultaneously, the multi-dimensional decoupled multi-scale convolutional module plays a critical role in extracting image features, while the multi-dimensional decoupled channel attention mechanism is applied in parallel to adjust channel weights and emphasize important channels. In the decoding phase, up-sampling is performed using a 2 × 2 × 2 deconvolutional kernel with a stride of 2, gradually restoring the image resolution. Residual skip connections between the encoder and decoder stabilize semantic information, ensuring effective information transfer across different levels. After concatenating the decoder output with the encoded information, further fusion is achieved through a residual block incorporating multi-dimensional decoupled spatial attention, suppressing irrelevant spatial information and enabling precise localization of the target region. Finally, a residual block fuses the decoded feature map with the input image, and the output is combined with the softmax activation function to produce a 3D segmentation mask with the same dimensions as the original image.

Multi-dimensional decoupled multi-scale extraction module

The challenge of lightweight segmentation networks lies in improving network speed, reducing the number of parameters, and maintaining segmentation accuracy under limited computational budgets. We use depthwise convolution as the basic building block for the network, which reduces model complexity and the number of parameters while ensuring accuracy (Chen et al., 2024a). Inspired by the Inception module and leveraging the multi-dimensional structural characteristics of 3D medical images, we decouple and extract features in parallel from the D, H, and W dimensions, enhancing the network’s perception of spatial information.

However, while the multi-dimensional feature module reduces network complexity, it faces issues such as a reduced receptive field and a lack of spatial contextual information, making it difficult to accurately locate pancreatic organs and reconstruct tumor information. Although large-kernel dimension extraction can expand the receptive field, it may overlook smaller tumor regions. To address this, we introduce a multi-scale extraction method, employing three different kernel sizes (5, 7, 9) within the same dimension to extract multi-scale features, expanding the receptive field and enriching multi-scale information while maintaining dimensional and neighboring spatial relationships (with kernel sizes of 3 in other dimensions). The multi-dimensional multi-feature extraction structure is illustrated in Fig. 2.

After feature extraction, we introduce pointwise scaling convolution (PSC) to independently and linearly scale the features of each channel, enriching feature representation through expansion and compression while reducing redundant cross-channel context. To avoid redundancy in multi-scale information fusion, a depthwise gating mechanism is added to suppress redundant information, facilitating subsequent multi-scale fusion. Inspired by Ma et al. (2024), we use multiplication as the method for multi-scale fusion, which maps inputs to a high-dimensional nonlinear feature space, better integrating multi-scale feature information. To preserve detailed information, the fourth branch employs a 3 × 3 × 3 residual block. Finally, the information from the three branches (d, h, w) is adjusted through a shared-parameter 1 × 1 × 1 convolution and concatenated with the information captured by the residual block, further enhancing detailed information. The mathematical expression is as follows:

(1) $$x_d,x_h,x_w=D\left(x\right),H\left(x\right),W\left(x\right)$$ (2) $$D\left(x\right)=Con{v}_{1,1,1}\left(x+{\prod }_{i=5,7,9}DG\left(PSC\left(DW{C}_{i,3,3}\left(x\right)\right)\right)\right)$$ (3) $$H\left(x\right)=Con{v}_{1,1,1}\left(x+{\prod }_{i=5,7,9}DG\left(PSC\left(DW{C}_{3,i,3}\left(x\right)\right)\right)\right)$$ (4) $$D\left(x\right)=Con{v}_{1,1,1}\left(x+{\prod }_{i=5,7,9}DG\left(PSC\left(DW{C}_{3,3,i}\left(x\right)\right)\right)\right)$$ (5) $$x_d^{\ast },x_h^{\ast },x_w^{\ast },x_{res}=Conv1,1,1\left(x_d,x_h,x_w\right),Res\left(x\right)$$ (6) $$x_{out}=Leaky\mathrm{Re}LU\left(Con{v}_{1,1,1}\left(Concatenate\left(x_d^{\ast },x_h^{\ast },x_w^{\ast }\right)\right)\right)$$ (7) $$y=Leaky\mathrm{Re}LU\left(Con{v}_{1,1,1}\left(Concatenate\left(x_{res},x_{out}\right)\right)\right)+x.$$Here, y represents the output feature map of the multi-dimensional multi-scale extraction module, and x is the input feature map. DWC denotes depthwise convolution, with subscripts indicating the corresponding kernel sizes. DG represents the depthwise gating mechanism. Res is the residual block. Concatenate denotes the concatenation operation. LeakyReLU is the activation function.

The input feature map x undergoes depthwise convolutions $\mathrm{D}\left(\mathrm{x}\right)$, $\mathrm{H}\left(\mathrm{x}\right)$ and $\mathrm{W}\left(\mathrm{x}\right)$ to independently extract key features along the D, H, and W dimensions, while the residual block $\mathrm{R}\mathrm{e}\mathrm{s}\left(\mathrm{x}\right)$ extracts detailed information, resulting in four new feature maps ${\mathrm{x}}_{\mathrm{d}},{\mathrm{x}}_{\mathrm{w}},{\mathrm{x}}_{\mathrm{h}},{\mathrm{x}}_{\mathrm{r}\mathrm{e}\mathrm{s}}$. Then, ${\mathrm{x}}_{\mathrm{d}},{\mathrm{x}}_{\mathrm{w}},{\mathrm{x}}_{\mathrm{h}}$ are transformed through a shared 1 × 1 × 1 convolutional mapping to obtain ${\mathrm{x}}_{\mathrm{d}}^{\ast },{\mathrm{x}}_{\mathrm{h}}^{\ast },{\mathrm{x}}_{\mathrm{w}}^{\ast }$ enhancing inter-channel information interaction. The concatenate operation aggregates the multi-layer decoupled 3D spatial information to produce ${\mathrm{x}}_{\mathrm{o}\mathrm{u}\mathrm{t}}$, which captures the variations in the image along the D, H, and W dimensions. This is concatenated with ${\mathrm{x}}_{\mathrm{r}\mathrm{e}\mathrm{s}}$ to enhance detailed information. Finally, the result is mapped through a 1 × 1 × 1 convolution and activated by LeakyReLU combined with the input x to prevent network degradation. This module, through its hierarchical decoupled multi-scale feature extraction and effective fusion strategy, is capable of capturing the complex positional variations of the pancreas and the subtle features of tumor regions.

Multi-dimensional decoupled channel attention module

The multi-dimensional decoupled multi-scale extraction module, while capable of extracting rich features, is prone to channel redundancy, which can affect segmentation results. The Squeeze-and-Excitation (SE) (Hu, Shen & Sun, 2018) module addresses this by modeling channel dependencies and automatically determining the importance of channels to enhance useful channels and suppress irrelevant ones. However, the SE module relies solely on global average pooling, resulting in a single feature representation that may lose local important features and lacks multi-scale understanding, making it difficult to capture multi-scale information such as the positional distribution of pancreatic organs in 3D data.

To address this issue, we propose a multi-dimensional decoupled channel attention module (structure shown in Fig. 3). This module can independently learn features from multiple dimensions, utilizing both global average pooling and global maximum pooling to capture average and salient feature responses. Specifically, the results from MDDMS are fed in parallel into three decoupled convolutions, generating multi-scale results across three dimensions. Subsequently, global average pooling and global maximum pooling are applied to each of the three scales, followed by shared PCEC (Pointwise Convolution with Expansion and Compression, ratio r = 4) processing. The two vectors from each dimension are summed to produce three vectors. Finally, through concatenation and a fully connected layer (FC), a weight vector containing channel importance information is generated. The mathematical expression for this module is as follows:

(8) $$x_{ca\mathrm{\_}d},x_{ca\mathrm{\_}h},x_{ca\mathrm{\_}w}=Con{v}_{5,1,1}\left(x_{in}\right),Con{v}_{1,5,1}\left(x_{in}\right),Con{v}_{1,1,5}\left(x_{in}\right)$$ (9) $$x_{ca\mathrm{\_}d\mathrm{\_}out}=PCEC\left(GMP\left(x_{ca\mathrm{\_}d}\right)\right)+PCEC\left(GAP\left(x_{ca\mathrm{\_}d}\right)\right)$$ (10) $$x_{ca\mathrm{\_}h\mathrm{\_}out}=PCEC\left(GMP\left(x_{ca\mathrm{\_}h}\right)\right)+PCEC\left(GAP\left(x_{ca\mathrm{\_}h}\right)\right)$$ (11) $$x_{ca\mathrm{\_}w\mathrm{\_}out}=PCEC\left(GMP\left(x_{ca\mathrm{\_}w}\right)\right)+PCEC\left(GAP\left(x_{ca\mathrm{\_}w}\right)\right)$$ (12) $$y_{ca}=x_{in}\times Sigmod\left(FC\left(Concatenate\left(x_{ca\mathrm{\_}d\mathrm{\_}out},x_{ca\mathrm{\_}h\mathrm{\_}out},x_{ca\mathrm{\_}w\mathrm{\_}out}\right)\right)\right)+x_{in}.$$Here, ${\mathrm{x}}_{\mathrm{i}\mathrm{n}}$ is the input. Conv denotes the convolution module, with subscripts indicating the corresponding kernel sizes. PCEC stands for pointwise compression and expansion convolution. GAP represents global average pooling, GMP represents global maximum pooling, and FC denotes a fully connected layer.

The input ${\mathrm{x}}_{\mathrm{i}\mathrm{n}}$ undergoes three parallel convolutional computations, resulting in three outputs ${\mathrm{x}}_{\mathrm{c}\mathrm{a}\mathrm{\_}\mathrm{d}},{\mathrm{x}}_{\mathrm{c}\mathrm{a}\mathrm{\_}\mathrm{h}},{\mathrm{x}}_{\mathrm{c}\mathrm{a}\mathrm{\_}\mathrm{w}}$. Each output is subjected to global average pooling and global maximum pooling, yielding six tensors of shape (C, 1, 1, 1), representing the global average pooling and global maximum pooling tensors for each dimension. These tensors are then processed through a shared PCEC for channel compression and recovery mapping. Subsequently, the resulting vectors are pairwise summed to produce three learning vectors ${\mathrm{x}}_{\mathrm{c}\mathrm{a}\mathrm{\_}\mathrm{d}\mathrm{\_}\mathrm{o}\mathrm{u}\mathrm{t}},{\mathrm{x}}_{\mathrm{c}\mathrm{a}\mathrm{\_}\mathrm{h}\mathrm{\_}\mathrm{o}\mathrm{u}\mathrm{t}},{\mathrm{x}}_{\mathrm{c}\mathrm{a}\mathrm{\_}\mathrm{w}\mathrm{\_}\mathrm{o}\mathrm{u}\mathrm{t}}$, which are then concatenated. The concatenated result is passed through a fully connected layer to generate channel importance information. Finally, the output is activated by a Sigmoid function to produce a probability vector, where the magnitude of the values represents the importance of the channels, thereby reducing the influence of irrelevant channels.

Multi-dimensional decoupled spatial attention module

The PC segmentation algorithm aims to achieve precise segmentation of the pancreas and its tumor regions in abdominal CT images. During the decoding phase, the gradual restoration of feature maps is crucial for the accuracy of the results. For 3D medical images, learning spatial structures is particularly important. Existing spatial attention models (SAM) (Woo et al., 2018) generate spatial attention maps by computing the average and maximum values along the channel dimension to highlight important spatial regions. However, SAM tends to overlook the relationships between dimensions in 3D medical images when calculating spatial attention, making it difficult to independently capture spatial information along the axial, coronal, and sagittal planes.

To address this issue, this study designs a MDSPA, structure shown in Fig. 4, aiming to independently capture spatial information from each dimension and aggregate it into a complete 3D spatial attention map to enhance the regions of interest. The RSA module first fuses high-level and low-level semantic features to concurrently capture both global context and fine-grained details - particularly critical for addressing tumor size variations. It then performs three parallel convolutions to independently extract spatial relationships across the three dimensions, explicitly modeling the inter-slice continuity that conventional 2D approaches struggle to capture. Following feature concatenation and aggregation, the module learns a spatial attention map that dynamically enhances key regions (such as blurred boundaries and micro-lesions) while suppressing irrelevant tissue interference, thereby providing implicit spatial guidance during the decoding phase. The process concludes with a convolution operation and residual connection to alleviate gradient vanishing in deep networks. The mathematical expression for the MDSPA is as follows:

(13) $$x_{spa\mathrm{\_}d},x_{spa\mathrm{\_}h},x_{spa\mathrm{\_}w}=Con{v}_{5,1,1}\left(x_{in}\right),Con{v}_{1,5,1}\left(x_{in}\right),Con{v}_{1,1,5}\left(x_{in}\right)$$

(14) $$y_{spa}=x_{in}\times Sigmod\left(Con{v}_{3,3,3}\left(x_{spa\mathrm{\_}d}+x_{spa\mathrm{\_}h}+x_{spa\mathrm{\_}w}\right)\right).$$Here, ${\mathrm{x}}_{\mathrm{i}\mathrm{n}}$ represents the input feature map, Conv denotes convolution, with subscripts indicating the kernel size. Sigmod represents the activation function. ${\mathrm{y}}_{\mathrm{s}\mathrm{p}\mathrm{a}}$ is the output of multi-dimensional spatial aggregation.

The input ${\mathrm{x}}_{\mathrm{i}\mathrm{n}}$ undergoes three parallel convolutional computations to extract spatial information along the three dimensions, and the channels are compressed (compression ratio r = 4) to reduce interference from irrelevant channels, resulting in three outputs: $x_{spa\mathrm{\_}d}$, $x_{spa\mathrm{\_}h}$, and $x_{spa\mathrm{\_}w}$. Subsequently, the three feature maps are aggregated through addition, and a 3 × 3 × 3 convolution is applied to map the channels back to their original size, ensuring spatial diversity and smoothness. The result is then mapped to a probability map using the Sigmod function and applied to ${\mathrm{x}}_{\mathrm{i}\mathrm{n}}$ to enhance the regions of interest, implicitly guiding the model to generate segmentation results. Through this design, the multi-dimensional decoupled spatial attention module can effectively capture spatial information across dimensions in 3D medical images, enhancing the model’s focus on regions of interest and thereby improving the accuracy of PC segmentation.

Datasets

To validate the effectiveness of the proposed method, we employed two publicly available datasets: Medical Segmentation Decathlon Pancreas Tumour (MSDPT) and National Institutes of Health Pancreas (NIHP).

MSDPT dataset

The MSDPT dataset (Simpson et al., 2019) is a benchmark dataset for the pancreatic cancer segmentation task in the Medical Segmentation Decathlon challenge and is widely used in pancreas and tumor segmentation research. This dataset, provided by the Amber Simpson Cancer Center in the United States, includes a training set of 281 CT scans and a test set of 139 CT scans. The number of CT slices per scan ranges from 37 to 751, with an axial plane resolution of 512 × 512 pixels and a slice spacing ranging from 0.7 to 7.5 mm. The dataset contains three labels: background, pancreas, and tumor. The tumor category encompasses various pathological types, including intraductal mucinous neoplasms, pancreatic neuroendocrine tumors, and pancreatic ductal adenocarcinomas. Its inclusion of diverse tumor types and multi-center data makes it particularly suitable for evaluating model robustness. The dataset is publicly available at: http://medicaldecathlon.com/.

NIHP dataset

The NIHP dataset, provided by the National Institutes of Health Clinical Center, consists of 82 abdominal contrast-enhanced 3D CT scans. All scans were performed during the portal venous phase, approximately 70 s after intravenous contrast injection. The subjects included 53 males and 27 females, of whom 17 were healthy kidney donors, and the remaining 65 were individuals without significant abdominal diseases or pancreatic cancer lesions. The CT scan resolution is 512 × 512 pixels, with pixel size and slice thickness varying depending on the scanning equipment. The slice thickness ranges from 1.5 to 2.5 mm. Data acquisition was performed using Philips and Siemens MDCT scanners (120 kVp tube voltage). The pancreas was manually segmented slice-by-slice by a medical student and subsequently verified and corrected by an experienced radiologist to ensure annotation accuracy. The standardized acquisition protocol and rigorous annotation process establish it as a reliable reference for normal pancreatic anatomy. This dataset is accessible through The Cancer Imaging Archive (TCIA) at: https://www.cancerimagingarchive.net/collection/pancreas-ct/ or https://doi.org/10.7937/K9/TCIA.2016.tNB1kqBU.

Experimental environment and parameters

The experimental code is implemented in Python 3.10 and runs on an Ubuntu 22.04 environment. The hardware configuration includes an AMD Ryzen 7 5800X 8-Core Processor (3.80 GHz) CPU and an NVIDIA GTX 3060 (12G) GPU. The deep learning framework used is PyTorch 1.12.0, and the models are constructed with the assistance of the Monai 0.8.0 library. All experiments are conducted on the same device to ensure fairness. The implementation code for MDMU-Net is available at: https://github.com/SerendipityInTheWorld/MDMU_Net or Zenodo: https://doi.org/10.5281/zenodo.15714416.

The experimental parameters are set as follows: the batch size is set to 1, and the maximum number of iterations is 40,000. The loss function used is Dice Cross-Entropy Loss (DiceCEloss). To prevent overfitting, an early stopping mechanism is employed, where training is terminated early if the Dice coefficient on the validation set does not improve for 15 consecutive epochs, and the best-performing model on the validation set is saved for final testing. The AdamW optimizer is used, with an initial learning rate of 0.0001. All experiments employ a 5-fold cross-validation method to evaluate model performance.

Data preprocessing and augmentation

This study primarily uses the MSDPT dataset for training and testing, with the NIHP dataset serving as supplementary validation to assess the model’s generalization ability. Since the test set labels are not publicly available, all data splits are performed randomly based on the training set. The experiments employ a 5-fold cross-validation method, where the training set is randomly divided into five parts, with four parts used for training and one part for testing, repeated five times. The validation set is constructed by randomly sampling 1/8 of the allocated training data in each cross-validation.

During the data preprocessing stage, the training data is first resampled to unify the pixel spacing, optimizing the network training effectiveness. The resampled pixel spacing for MSDPT and NIHP datasets is [1.2, 1.2, 2.0] and [0.93, 0.93, 1.0], respectively. Next, the intensity values of the medical images are remapped to the range [0, 1] (the original intensity range for MSDPT and NIHP is [−145, 275]). Subsequently, foreground cropping is performed, and positive and negative samples are randomly cropped based on the labels, with a crop size of 96 × 96 × 96, to balance the ratio of positive and negative samples.

To enhance the model’s generalization ability and robustness, data augmentation operations are applied to the training data, including:

• (1)

  Random intensity adjustment: With a probability of 0.5, a random offset within the range [−0.1, 0.1] is added to the intensity value of each pixel.

• (2)

  Affine Transformation: Each image undergoes mandatory rotation (about the Z-axis, ranging from 0 to π/30 radians) and uniform scaling (with scaling factors varying up to ±10% in each spatial dimension).

• (3)

  For the validation and test sets, only resampling, intensity remapping, and foreground cropping are performed to maintain data consistency and authenticity. All preprocessing and data augmentation operations are implemented using the Monai framework.

Evaluation metrics

This study focuses on the segmentation of the pancreas and its tumors and employs the following evaluation metrics: Dice Similarity Coefficient (DSC), Jaccard Index (JC), sensitivity, specificity, and 95% Hausdorff distance (HD95). For the NIH dataset, the primary evaluation metrics are DSC, JC, and HD95.

• (1)

  DSC

• The DSC is one of the most commonly used evaluation metrics in medical image segmentation. It measures the overlap between the ground truth (GT) and the prediction (PR). The formula for DSC is: (15) $$DSC\left(A,B\right)={\displaystyle \frac{2\left|A\cap B\right|}{\left|A\right|+\left|B\right|}.}$$

• (2)

  JC

• The JC represents the ratio of the intersection to the union of two sets. The formula for JC is: (16) $$JC\left(A,\mathrm{B}\right)={\displaystyle \frac{\left|A\cap B\right|}{\left|A\cup B\right|}.}$$

• For both DSC and JC, A represents the ground truth, and B represents the prediction.

• (3)

  Sensitivity

• Sensitivity (also known as recall) measures the proportion of actual positive cases that are correctly identified as positive. The formula for sensitivity is: (17) $$Sensitivity={\displaystyle \frac{TP}{TP+FN}.}$$

• (4)

  Specificity

• Specificity measures the proportion of actual negative cases that are correctly identified as negative. The formula for specificity is: (18) $$Specificity={\displaystyle \frac{TN}{TN+FP}.}$$

• For sensitivity and specificity: TP (true positives): The number of samples that are actually positive and correctly identified as positive. FN (false negatives): The number of samples that are actually positive but incorrectly identified as negative. TN (true negatives): The number of samples that are actually negative and correctly identified as negative. FP (false positives): The number of samples that are actually negative but incorrectly identified as positive.

• (5)

  95% Hausdorff Distance (HD95)

• The Hausdorff Distance measures the maximum distance between two point sets. HD95 is a variant that considers the 95th percentile of these distances, thereby reducing the impact of outliers. The formula for HD95 is: (19) $$H{D}_{95}\left(A,B\right)=95thpercentileof\left(max\left({sup}_{a\in A}{inf}_{b\in B}\parallel a-b\parallel ,{sup}_{b\in B}{inf}_{a\in A}\parallel a-b\parallel \right)\right)$$

• where $\parallel \mathrm{a}-\mathrm{b}\parallel$ represents the Euclidean distance between points a and b, sup denotes the supremum (the maximum of all possible distances), and inf denotes the infimum (the minimum of all possible distances).

Ablation analysis

To evaluate the effectiveness of our proposed MDMU-Net, we conducted comprehensive ablation experiments. Using 3D U-Net as the baseline, we designed five ablation experiments: (1) 3D U-Net, (2) 3D U-Net + Res, which retains the original structure but replaces skip connections and decoder parts with residual blocks, (3) 3D U-Net + MDDMS + Res, (4) 3D U-Net + MDDMS + MDCA + Res, and (5) MDM-UNet.

The experimental results (as shown in Table 1) indicate that the model’s performance in both pancreas and tumor segmentation tasks significantly improves with the gradual addition of modules.

On the MSDPT dataset, the baseline model (3D U-Net only) achieves a pancreas DSC of 0.6606, a tumor DSC of 0.2112, a pancreas HD95 of 14.0012, and a tumor HD95 of 63.1915. After adding residual connections (res), the pancreas DSC shows little change, while the tumor DSC increases to 0.2635. However, the tumor HD95 significantly rises to 88.6984 ± 18.9174, indicating a decline in boundary segmentation accuracy. Further introducing the multi-scale feature fusion (MDDMS) module increases the pancreas DSC to 0.6804 and the tumor DSC to 0.2979, but the tumor HD95 further rises to 91.5002, suggesting poor stability in tumor boundary segmentation. After incorporating the multi-dimensional decoupled channel attention (MDCA) module, the pancreas DSC further improves to 0.6899, and the tumor DSC increases to 0.3017, while the tumor HD95 decreases to 76.3987, indicating that the channel attention module has a certain optimization effect on tumor segmentation performance. Finally, with the introduction of the spatial attention (MDSPA) module, the pancreas DSC reaches its highest value of 0.7108, the tumor DSC improves to 0.3312, and the tumor HD95 significantly decreases to 49.5939, demonstrating the significant role of the MDSPA module in enhancing tumor segmentation accuracy and boundary segmentation performance.

On the NIHP dataset, the baseline model achieves a pancreas DSC of 0.6738 and a pancreas HD95 of 37.76. With the gradual addition of modules, the pancreas DSC progressively increases to 0.7709, and the pancreas HD95 decreases to 15.3269, further validating the effectiveness of the module combination.

Overall, the MDDMS and MDSPA modules contribute the most to the improvement in pancreas and tumor DSC, while the channel attention and spatial attention modules significantly optimize the tumor HD95. The synergistic effects between the modules notably enhance the model’s segmentation performance in complex scenarios.

Model performance analysis

In this section, we compare our MDMU-Net with seven state-of-the-art segmentation methods on the MSDPT and NIHP datasets. The CNN-based models include: 3D U-Net (Çiçek et al., 2016) with its classic encoder-decoder architecture, Attention U-Net (Oktay et al., 2018) featuring attention gates, SegResNet (Myronenko, 2019) combining residual connections with multi-scale fusion, and DynUNet (Futrega et al., 2022) with dynamic architecture adjustment. The Transformer-based (Vaswani et al., 2017) approaches comprise: UNETR (Hatamizadeh et al., 2022b) integrating Transformers into the U-Net framework, nnFormer (Zhou et al., 2023) blending CNNs and Transformers, Swin UNETR (Hatamizadeh et al., 2022a) employing a Swin Transformer encoder, and 3DUX-Net using large-kernel depthwise convolutions to mimic Transformers.

The experimental results on the MSDPT dataset (Table 2) demonstrate that MDMU-Net achieves remarkable efficiency with only 26.97M parameters and 84.837G FLOPs. Specifically, compared to AttentionUNet (23.63M) with similar parameter counts, it improves tumor Dice by 0.0351 while reducing computations by 160.768G; when compared to UNETR (82.550G) with comparable computation costs, it uses 65.81M fewer parameters. This efficiency advantage originates from our depthwise convolution multi-dimensional decoupling design, which effectively eliminates redundant computations. The model delivers outstanding segmentation performance, achieving DSC scores of 0.7108 (pancreas) and 0.3312 (tumor), significantly surpassing competing methods. As shown in Table 3, its superior boundary segmentation accuracy is further validated by an HD95 score of 28.3127. These improvements stem from the synergistic combination of multi-dimensional decoupled multi-scale feature extraction modules and attention mechanisms, which precisely capture multi-scale features and demonstrate particular efficacy in small target segmentation.

Further comprehensive metric analysis (Table 3) reveals that MDMU-Net achieves optimal performance in Dice (0.5210), IoU (0.4174), and sensitivity (0.5521), with standard deviations lower than most comparative models, indicating more stable segmentation performance. The HD95 metric is 7.99 mm lower than the best comparative model (3DUX-Net), further confirming significant improvements in boundary segmentation accuracy. Notably, although SegResNet has the lowest parameter count (1.18M), its overall performance (Dice 0.4732) shows a significant gap compared to MDMU-Net, demonstrating that simply reducing parameters compromises model capability.

To validate the method’s generalization ability, experiments on the NIHP segmentation dataset (Table 4) show that MDMU-Net surpasses the current best method (DynUNet) in both pancreas segmentation Dice (0.7709) and IoU (0.6357), with average metric improvements of 6.4% and 9.0% respectively. Particularly in boundary accuracy, the HD95 (15.3267) is 6.387 mm lower than AttentionUNet, with the smallest standard deviation, indicating strong adaptability to different data distributions. The background class metric (Dice 0.9996) remains comparable to other models, showing that the method significantly improves small target segmentation while maintaining stability in large target segmentation.

We also present visualization results of multiple segmentations on CT slices (Figs. 5 and 6). For Sample2 in Fig. 5, the proposed MDMU-Net can still identify and accurately segment the tumor region. As shown in Sample3 in Fig. 6, the model also accurately segmented the pancreatic organ. However, in the pancreatic tumor segmentation of Sample5 in Fig. 5, the model shows slight deviations in localizing tumor boundaries, with some small tumor areas being missed. Although the model demonstrates effective segmentation capability for tumors and pancreatic organs in most samples, in Sample5, constrained by the complexity of tumor morphology in medical imaging (such as micro-lesions and blurred boundaries), the model exhibits insufficient sensitivity to fine structures during shallow feature extraction, or loses some detailed information due to down-sampling in deep networks, resulting in boundary deviations and missed segmentation. Furthermore, through the visualization analysis of the feature maps from the last layer of MDMU-Net (Fig. 7), we observe that the model exhibits high attention to the pancreatic region (yellow areas) and effectively identifies small-sized tumor targets.

Additionally, we use the paired t-test to calculate the p-values for the DSC metric to evaluate statistical significance. As shown in Tables 5 and 6, the p-values for MDMU-Net on the DSC metric are all less than 0.05, indicating that the performance improvement is statistically significant. The experimental results demonstrate that the proposed improvements in MDMU-Net are feasible and provide a new solution for 3D medical image segmentation.