Comparative analysis of automatic segmentation of esophageal cancer using 3D Res-UNet on conventional and 40-keV virtual mono-energetic CT Images: a retrospective study

Patient enrollment

This retrospective study was approved by the institutional review board of Zhongshan Hospital affiliated to Xiamen University, and informed consent was waived. The Ethical Approval number was XMZSYY-AF-SC-12-03. 259 Patients with histologically confirmed EC by biopsy or surgery, who underwent enhanced CT with SDCT from May 2019 to December 2020 were selected for this study. Exclusion criteria were: (a) chemotherapy or radiotherapy or other anticancer treatments before the IQon CT scans; (b) a history of other malignancies; and (c) incomplete medical records. After excluding 99 cases, 160 patients were finally enrolled in this study and randomized divided into the training dataset (104 patients), validation dataset (26 patients) and test dataset (30 patients). Figure 1 depicts the process used to select the images.

CT image acquisition

Chest CT was performed in both the arterial and venous phases on a SDCT (IQon Spectral CT, Philips Healthcare, Best, The Netherlands). All the patients were in the supine position, with both arms raised and breathing calmly; while holding the breath, scanning range was from the epiglottis to the costophrenic angle to ensure coverage of all esophageal and lung tissues. Before initiating contrast-enhanced scanning, 70 ml contrast medium (Ultravist injection at a concentration of 300 mg/ml; Bayer, Leverkusen, Germany) was injected through the elbow vein at a rate of 2.5 ml/s using a power injector. After the injection, 20–30 ml of normal saline was injected for flushing at a rate of 3 ml/s. Arterial and venous phase images were acquired at 25s and 60s. The following scanning parameters were used: detector collimation, 64 × 0.625 mm; tube voltage, 120 kV; tube current, 180–250 mA; rotation speed, 0.33s/rot; helical pitch, 0.671; matrix, 512 × 512.

Image reconstruction

Spectral base image (SBI) datasets were reconstructed, with a spectral level of three, a slice thickness of one mm, a slice increment of one mm, standard kernel (B) with a window width of 400 HU & a level of 40 HU, and sharp kernel (YB) with a window width of −1400 HU & a level of −600 HU.

All arterial phase images were transferred to the dedicated workstation (IntelliSpace V9, Philips Healthcare) for further analysis. Both conventional images (CI) and virtual mono-energetic images at 40 keV (VMI_40 keV) were derived from SBI.

Esophageal cancer annotation

An open-source software (3D slicer, version 4.11, https://www.slicer.org/) was used to delineate EC contours manually. All the tumor contours were drawn on each slice of VMI_40 kev by a radiologist with 8 years of experience and peer-reviewed by a senior radiologist with 15 years of experience. The adjacent airway, lymph nodes, heart, vascular structures and thoracic vertebrae were avoided during contour drawing. Dice similarity coefficient (DSC) was performed to assess consistency between the two observers for EC segmentation.

Image pre-processing

Image pre-processing was performed as follows. (1) All voxel sizes were reset to 1 mm × 1 mm × 1 mm by linear interpolation, and matrices were defined as n ×150 ×180 (slices ×columns ×rows) by Simple ITK (version 2.1, https://simpleitk.org/) just for the inclusion of the EC and a small amount of adjacent tissues, to reduce the calculation burden and graphic card’s memory storage. (2) The ScaleIntensityRanged function in MONAI (version 0.6, https://monai.io/) was used to rescale the density values of our images and all density values in our images which ranged from −50 HU (minimum) to 400 HU (maximum), were normalized and artificially scaled to a range of 0 to 1 using this function. (3) Before sending the images into a convolutional neural network, all images in the training dataset were randomly cropped to size of 96 × 96 × 96 mm³ and data augmentation was achieved in the training dataset by image flipping and rotation using the RandCropByPosNegLabeld and RandAffined functions in MONAI.

Automatic segmentation model and training

In this study, 3D Res-UNet, the classical U-net scheme combined with a residual connection unit (RCU), was used for the automatic segmentation task. Five residual units were added to the automatic segmentation model in this study, The model uses a kernel size of (3, 3, 3) and varying number of channels (16, 32, 64, 128, 256) to build the layers of the encoder and decoder. It uses a stride of two to downsample the spatial dimensions of the encoder and residual units with batch normalization to improve the model’s robustness. The architecture of the automatic segmentation model is shown in Fig. 2

The Adam optimizer was used to train the automatic segmentation model with the Dice loss function as the cost function. The batch size was set to eight, the learning rate was set to 0.0001. The model was trained for 600 epochs. The curves depicting the training process of the networks are presented in both Fig. 3.

The model was built using PyTorch (version 1.8, https://pytorch.org/) and MONAI (version 0.6, https://monai.io/) and trained on a Linux workstation (Ubuntu version 20.04) with one NVIDIA GeForce GTX3090 GPU with 24 GB memory (NVIDIA, Santa Clara, VA, USA).

Evaluation of segmentation performance in the test dataset

After training the model, we examined its performance in predicting the ROIs in the test dataset. Segmentation accuracy was evaluated using the DSC as follows. (1) A statistical measure of spatial overlap also defined as DSC was derived as 2TP/(FP+2TP+FN), where TP, FP and FN are the numbers of true positive, false positive and false negative detections, respectively. (2) Intersection of union (IOU) was defined as TP/(FP+TP+FN), which evaluates the numbers of TP and FN detections. (3) Average Symmetric Surface Distance (ASSD) was defined as ASSD(A,B) = $\frac{1}{S\left(A\right)+S\left(B\right)}\left({\sum }_{sA\in S\left(A\right)}d\left(sA,S\left(B\right)\right)+{\sum }_{s_B\in S\left(B\right)}d\left(sB,S\left(A\right)\right)\right)$, where S(A) denotes the set of surface voxels of A. The shortest distance of an arbitrary voxel ν to S(A) was defined as: $d\left(\nu ,\mathrm{S}\left(A\right)\right)=\underset{s_A\in S\left(A\right)}{min∥\nu -}sA∥$, where ∥. ∥ denotes the Euclidean distance. (4) 95% Hausdorff Distance (HD_95) was defined as H(A,B) = max(h(A,B), h(B,A)), a measure that describes the 95th percentile of all distances between points in image A and the nearest point in image B, where h(A,B) = max(a ∈A)min(b ∈B) ∥a-b ∥ and h(B,A) = max(b ∈B)min(a ∈A) ∥b-a ∥ and ∥b-a ∥ is the Euclidean distance.

Statistical analysis

Statistical analyses were performed with R (version 4.0; R Core Team, 2020). Patients’ age in the training plus validation datasets versus test dataset were compared with two-tailed Independent t-test. The sex, histopathological features, differentiation grade and T stage of EC were compared by the two-sided Fisher’s exact test. The independent sample rank-sum test was used for comparing tumor size. DSC, IOU, ASSD and HD_95 in the test dataset with VMI_40 keV and CI as input data were compared by the paired Wilcoxon signed-rank test. P < 0.05 was considered statistically significant.

Patient characteristics

A total of 160 patients were included in the analysis. They were aged from 39 to 88 years (mean of 64 years). Table 1 lists clinical and demographic features in the training and validation and test datasets. The results showed no statistically significant differences in clinical and demographic characteristics between the training and validation and test datasets. The histopathological types were squamous cell carcinoma, adenocarcinoma and small cell carcinoma, most of which were well differentiated or moderately differentiated. Tumor sizes ranged from 4.16 cm³ to 142.82 cm³ (median of 24.97 cm³). The DSC for the EC segmentation between the two observers was 0.898 ± 0.036.

Model performance

3D Res-UNet was applied to automatically segment the target volumes of EC on CT images. Segmentation results for a representative patient for VMI_40 keV and CI are shown in Fig. 4. The median results of DSC, IOU, ASSD and HD_95 statistics for the segmentation indexes VMI_40 keV and CI in the test dataset are shown in Table 2, while Fig. 5 shows the distribution of the results. We trained and evaluated 3D Res-UNet on VMI_40 keV and CI, separately. There were statistically significant differences in DSC, IOU, ASSD and HD_95 between the models with VMI_40 keV and CI. The results of VMI_40 keV were slightly better for the CI with the automatic segmentation model in DSC, IOU, ASSD and HD_95. According to the good agreement threshold of the DSC (DSC ≥0.7) (Zijdenbos et al., 1994; Bartko, 1991), automatic segmentation results for the EC of VMI_40 keV and CI were consistent with manual segmentation results.