Machine learning-driven PET-CT and clinical pathology model for predicting mediastinal lymph node metastasis in non-small cell lung cancer: a retrospective cohort study

Patients

This retrospective cohort included consecutive patients who underwent surgical treatment and were pathologically diagnosed with NSCLC in the First Hospital of Jilin University from January 2017 to December 2023. The dates for accessing data for research purposes were from December 2023 to August 2024. Inclusion criteria were as follows: (1) 18F-FDG PET-CT examination was performed within 2 weeks before surgery; (2) receiving surgical treatment and systematic lymph node dissection; (3) postoperative pathology confirmation of NSCLC. Exclusion criteria were: (1) neoadjuvant therapy; (2) patients with history of other malignant tumors; (3) presence of distant metastasis.

Within the study period, a total of 1,149 patients with stage II-III NSCLC who underwent surgical treatment and mediastinal lymph node dissection were identified. Among them, a total of 403 patients underwent 18F-FDG PET-CT examination two weeks before surgery. Five patients had received preoperative adjuvant therapy, and eight patients had a history of other malignant tumors. No patient had distant metastasis. After applying the exclusion criteria, 13 patients were excluded and 390 patients were finally included in the analysis (Fig. 1). The study protocol was reviewed and approved by the Institutional Review Board/Ethics Committee of the first Bethune Hospital of Jilin University (approval No. 2024-1125), which waived the requirement for written informed consent from patients. All methods were carried out in accordance with relevant guidelines and regulations.

PET-CT scan

All patients underwent the same PET-CT examination. A Biograph 16HR PET-CT scanner (Siemens Biograph 16HR) was used in the study. Before imaging, patients were fasted for more than 6 h, and their blood glucose levels needed to be below 8 mmol/L. 18F-FDG (Sumitomo HM-12, pH 4-8, radioactive purity > 95%, radioactive activity concentration > 370 MBq/ml) was then injected intravenously at a dose of 3.7 MBq/kg. Image acquisition began after 1 h. CT images were scanned from the skull base to the proximal thighs (tube voltage 120 kV, tube current 50 mAs, rotation speed 0.5 s/r, field of view (FOV) 812 mm × 812 mm, 512 × 512 matrix, slice thickness 4 mm). Low-dose CT was used for attenuation correction, and CT images were reconstructed using standard B30f soft tissue. Therefore, positron emission tomography (PET) images were scanned (eight window levels, 2.5 min each, field of view 812 mm × 812 mm, 144 × 144 matrix, slice thickness 4 mm) from the top of the head to the mid-thigh. PET images were reconstructed iteratively in 3D mode using ordered subset expectation maximization (two iterations, 24 subsets, and Gaussian filtering). Both PET and CT images were acquired using a slice thickness of 4 mm during scanning. Subsequently, all images were reconstructed with a slice thickness of 1.25 mm and an increment of 1.0 mm to improve spatial resolution and facilitate volumetric analysis. This process ensured consistent voxel geometry for quantitative feature extraction.

Image processing

The target lesions in this study were the primary tumor and the dissected mediastinal lymph node stations (stations 2R, 3a, 4R, 4L, 5, 6, 7, 8, 9). On the fused PET-CT images, regions of interest (ROIs) were manually delineated by selecting the cross section with the largest maximum standardized uptake value (SUVmax) centered on the lesion and lymph node region. PET images were reconstructed iteratively in 3D mode to generate volumes of interest (VOIs). SUVmax ≥ 40% was used as the standardized uptake value (SUV) threshold to determine the final contour edge of the target. The imaging features were recorded. The imaging features were recorded after ROIs were delineated on fused PET-CT images. Quantitative parameters, including SUVmax, mean standardized uptake value (SUVmean), peak Standardized Uptake Value (SUVpeak), maximum Standardized Uptake Value normalized to Lean Body Mass (SULmax), mean Standardized Uptake Value normalized to Lean Body Mass (SULmean), MTV, and TLG, were automatically extracted using the Medex imaging system, while qualitative CT features (e.g., spiculation, lobulation, bronchial cutoff, pleural indentation, and calcification) were assessed visually by two experienced nuclear medicine physicians. VOIs were reviewed and corrected by two nuclear medicine physicians, each with approximately 15 years of independent clinical experience, who were blinded to pathological and clinical information, and any discrepancies were resolved by consultation. The dissected mediastinal lymph node area was measured according to the lymph node map of coronal, sagittal and transverse CT published by the International Association for the Study of Lung Cancer (IASLC) in 2009.

Pathological diagnosis

All patients underwent lobectomy or partial pneumonectomy with systematic hilar (N1) and mediastinal lymph node (N2) dissection within 2 weeks after 18F-FDG PET-CT examination. The pathological status of mediastinal lymph nodes involved in this study was determined by postoperative pathological N (pN) staging according to the postoperative pathological results. Pathological findings were determined by two senior pathologists, each with approximately 15 years of professional diagnostic experience, and any disagreements were resolved by consultation.

Feature extraction

After data preprocessing, feature selection was conducted as follows. Pathological features: The pathological status (metastatic/non-metastatic) of each station of resected mediastinal lymph nodes, histological type of primary tumor, microhistological type, maximum diameter of tumor, necrosis, nerve, blood vessel, visceral pleural invasion or dissemination along the airway, TNM staging were extracted from the pathological information system of the Department of Pathology of the First Hospital of Jilin University. TNM staging was determined according to the eighth edition of the staging system.

Imaging features: Medex imaging system was used to extract the location of lung cancer (divided into central type and peripheral type according to the inner 1/3 and outer 2/3 of the lung field), the location of tumor, the type of lung nodule (pure ground glass nodule, mixed density and implementation nodule). There were spiculation sign, lobulation sign, bubble sign, cavity, calcification, bronchial truncation sign, pleural indentation sign and pleural effusion. Whether the boundary of the lesion was clear and whether it was complicated with other pulmonary diseases (Chronic Obstructive Pulmonary Disease (COPD), pneumonia, bronchiectasis, obstructive atelectasis). Imaging parameters included the maximum CT attenuation value of the primary lesion and each resected mediastinal lymph node station (CTmax, expressed in Hounsfield units, HU), mean CT attenuation value (CTavg, HU), SUVmax, Minimum Standardized Uptake Value (SUVmin), average standardized uptake value (SUVavg), SUVpeak, Standard Deviation of Standardized Uptake Value (SUVsd), SULmax, Minimum Standardized Uptake Value normalized to Lean Body Mass (SULmin), Average Standardized Uptake Value normalized to Lean Body Mass (SULavg), Peak Standardized Uptake Value normalized to Lean Body Mass (SULpeak), Standard Deviation of Standardized Uptake Value normalized to Lean Body Mass (SULsd), MTV, and TLG. The clinical stage (cN) of the lymph node and the short diameter S(mm) of the lymph node.

Clinical features: Gender, age, smoking, drinking, hypertension, type 2 diabetes, and coronary heart disease were extracted from the HALO clinical record system of the First Hospital of Jilin University. Clinical laboratory indicators included D-dimer (mg/L) and erythrocyte sedimentation rate (mm/1 h). Tumor markers included cytokeratin 19 fragment (Cyfra21-1, ng/mL), ProGRP (pg/mL), CA724 (U/mL), CEA (ng/mL), CA242 (U/mL), CA125 (U/mL), neuron-specific enolase (NSE, ng/mL), AFP (ng/mL), SCC-Ag (ng/mL), CA19-9 (U/mL), and CA153 (U/mL). Pulmonary function was represented by FEV₁ (L).

Statistical analysis

All data from January 2017 to December 2023 were analyzed using SPSS (version 27; IBM, Armonk, NY, USA) and performed the analysis. Statistical analyses were performed using SPSS version 27 (IBM Corp., Armonk, NY, USA). Continuous variables were tested for normality. Normally distributed data were expressed as mean ± SD and compared using the t-test, while non-normal data were shown as median (IQR) and compared using the Mann–Whitney U test. Categorical variables were compared using the ${\chi }^2$ test or Fisher’s exact test, and their utilization rates were calculated accordingly. Correlations between variables were analyzed using Spearman’s rank correlation. A two-sided P < 0.05 was considered statistically significant. Some patients in this retrospective study did not have complete data available for all clinical or imaging variables. To minimize bias, variables with less than 10% missingness were imputed using the median (for continuous variables) or mode (for categorical variables). Variables with more than 20% missing data were excluded from model development. A multiple imputation approach (five imputations using chained equations) was performed as a sensitivity analysis to confirm the robustness of the results. Cases with missing dependent variables were excluded from model training and evaluation.

Feature selection

Random forest based Out-of-Bag (OOB) error estimation method was used to select clinical features, pathological features and imaging features. The Bootstrap sampling technique was used to construct the training set by extracting samples from the original dataset, while the unselected samples were used as out-of-pocket data to evaluate the generalization performance of the model. By analyzing the out-of-pocket error changes before and after each feature was added to the model, we were able to quantify the impact of each feature on the prediction performance of the model and thus identify key features. Shapley Additive Explanations (SHAP) values were introduced to account for the contribution of features to the model output. The SHAP method assigns each feature a contribution score based on game theory, which helps us to understand the importance of the feature and its relationship with the target variable. A 90% cumulative contribution threshold was set, and the top n most influential features were selected for inclusion in the machine-learning algorithm. In total, 41 imaging features were initially extracted from both primary tumors and mediastinal lymph nodes. To prevent model overfitting and improve interpretability, features were ranked by their mean decrease in the OOB error and SHAP importance values. Features contributing cumulatively to 90% of the total model variance were retained, resulting in the selection of 20 imaging features. This threshold was empirically determined to balance model simplicity and predictive accuracy. The retained features included metabolic indices (e.g., SUVmax, MTV, TLG), anatomical measurements (e.g., lymph node short-axis diameter, CTmax, CTavg), and qualitative CT signs (e.g., spiculation and pleural indentation). These features were subsequently combined with the top-ranked clinical and pathological variables identified through the same selection process to form the final input set for model construction. SHAP analysis was performed separately for clinical, pathological, and imaging feature sets to interpret domain-specific feature contributions. In addition, a combined SHAP analysis was conducted for the integrated TLPC model to visualize the overall feature importance across domains.

All machine-learning analyses were implemented in Python 3.8 (Python Software Foundation) within a reproducible computing environment. The core dependent libraries and their versions were as follows: numpy 1.24.3, pandas 2.1.0, scikit-learn 1.3.0, xgboost 1.7.5, shap 0.43.0, and matplotlib 3.8.0.

A fixed random seed (2024) was applied throughout data processing, feature selection, and model training to ensure consistent results across runs. All model training, validation, and visualization procedures were executed using these fixed package versions to enable full reproducibility of the analytical pipeline.

Select the optimal machine learning classifier

Supervised learning algorithms were implemented to construct and evaluate predictive models. The performance of various machine learning algorithms was further evaluated in order to identify the optimal predictive model. The explored algorithms include Logistic Regression, Classification and Regression Tree (CART), SVM, gradient boosting decision tree (GBDT), Random Forest, multi-layer perceptron (MLP), extreme gradient boosting tree (XGBoost) and k-nearest neighbor algorithm (KNN). To ensure the robustness of the model, the dataset was divided into a training set (n = 410), a test set (n = 308), and a validation set (n = 308) using a stratified random sampling strategy based on the target variable (presence or absence of MLNM). This ensured that the proportion of positive and negative cases was consistent across all subsets, making each set representative of the overall dataset and preventing sampling bias. During the training process, each algorithm was subjected to k-fold cross-validation to assess its stability and generalization performance through repeated partitioning of the training and validation sets. Cross-validation provides reliable performance estimates for each model, avoiding the risk of model overfitting or underfitting. In addition, the area under the receiver operating characteristic (ROC) curve (AUC) was employed as the primary metric for model evaluation. The AUC value can not only comprehensively measure the performance of classifiers, but also provide an objective basis for the comparison between different algorithms. By comparing the AUC values of each model, the algorithm with the best performance and strongest stability was selected as the core model for predicting MLNM.

Model tuning and hyperparameter optimization

To ensure fair and reproducible model comparison, all machine-learning classifiers were trained and tuned using standardized pipelines implemented in Python 3.8 (scikit-learn 1.3.0, xgboost 1.7.5). The full cohort was first randomly split into a training set, a held-out internal test set, and an external validation set (as described in the ‘Statistical analysis’ section). Within the training set, a nested cross-validation framework was employed to optimize hyperparameters and avoid overfitting or information leakage. Specifically, the training set was internally divided into five folds: an inner loop for hyperparameter optimization using Randomized Search CV, and an outer loop for unbiased model evaluation.

Each algorithm was tuned within a literature-based search space as follows: Logistic Regression: penalty (L1/L2), C (10⁻⁴–10²), solver (liblinear/saga); CART: max_depth (2–20), min_samples_split (2–20), criterion (gini/entropy); Random Forest: n_estimators (200–1,500), max_depth (5–30 or None), max_features (sqrt/log2), bootstrap (True/False); GBDT/XGBoost: n_estimators (100–1,000), learning_rate (0.01–0.3), max_depth (3–10), subsample (0.6–1.0), colsample_bytree (0.6–1.0), min_child_weight (1–10), gamma (0–5); SVM with radial basis function kernel: C (10⁻³–10³), γ (10⁻⁴–10⁰); KNN: n_neighbors (3–51, step 2), weights (uniform/distance); MLP: hidden_layer_sizes ∈ {(64,), (128,), (64,32)}, activation (relu/tanh), learning_rate_init (10⁻⁴–10⁻³), max_iter (500), early_stopping (True).

The optimal hyperparameter combination for each algorithm was selected according to the mean AUC across the inner folds. After tuning, each model was retrained on the full training set with the best parameters and then evaluated on the held-out internal test and independent validation sets. Model probabilities were calibrated using Platt scaling or isotonic regression according to the lowest cross-validated Brier score.

For reproducibility, a fixed random seed (2024) was applied to all procedures. Core software packages included numpy 1.24.3, pandas 2.1.0, matplotlib 3.8.0, and shap 0.43.0.

Development and validation of the prediction model

To predict MLNM, three prediction models including TLPC, Tumor-Pathology-Clinical (TPC) and Lymph-Pathology-Clinical (LPC) were established in the test set according to the selected clinical factors, tumor PET-CT imaging features, lymphatic PET-CT imaging features and the combination of the above features. The three prediction models were further validated in the validation set. Decision curve analysis (DCA) was used to evaluate the clinical utility of a model by measuring the net benefits at different thresholds. To compare the clinical utility of eight machine learning algorithms in establishing prediction models. Accuracy, precision, recall and F1-score were used to quantify the diagnostic effect of the prediction model. The traditional standard (SUVmax > 2.5 or S > 1 cm) prediction model was used for comparison. The AUC was used to quantify the diagnostic performance of the prediction model, and the DeLong test was used to evaluate the difference in AUC between each model. All P-values were derived from DeLong’s test comparing AUCs between models, with statistical significance defined as P < 0.05. Given that multiple models and feature sets were compared (eight classifiers across three feature combinations), a correction for multiple comparisons was applied to control the family-wise error rat. The P-values obtained from DeLong’s tests were adjusted using the Benjamini–Hochberg false discovery rate (FDR) method, with an adjusted significance threshold of q < 0.05. This approach reduces the risk of type I error inflation while maintaining statistical power in comparative model evaluation. The confusion matrix was used to visually present the classification effect of the model, including statistics for true positive, false positive, true negative, and false negative.

Characteristics of the patients

This study included a total of 390 patients with NSCLC, among whom 241 patients (61.79%) were classified as pN0-1, while 149 patients (38.21%) were classified as pN2 (Table 1). A total of 1,026 mediastinal lymph node stations were dissected, with 204 stations (19.88%) showing metastasis and 822 stations (80.12%) exhibiting no metastasis. The clinical, pathological, and imaging characteristics were analyzed and evaluated in the training set (n = 433), test set (n = 288), and validation set (n = 308). To improve methodological clarity and readability, Table 1 was divided into two parts: one summarizing clinical and pathological characteristics, and the other summarizing imaging and radiomic features. Continuous variables were rounded to one decimal place, and all abbreviations were defined in table footnotes for consistency and transparency.

Figure 2 presents a heatmap of the Spearman correlation analysis, illustrating the relationships among tumor and lymph node metabolic characteristics, imaging features, clinical-pathological characteristics, and various clinical factors. Figure 2A shows strong correlations among metabolic parameters (SUVmax, SUVavg, MTV, and TLG) with Spearman coefficients ranging from 0.71 to 0.96. Figure 2B demonstrates moderate correlations between lymph node and primary tumor imaging features (0.1–0.6). Figure 2C depicts associations among clinical variables such as smoking history, forced expiratory volume in 1 s (FEV1), CEA, and SCC-Ag, revealing an inverse relationship between smoking and FEV1 and weak correlations between tumor markers and traditional clinical indicators.

Cross-domain correlations showed that several metabolic parameters (SUVmax, MTV, TLG) were moderately associated with systemic clinical markers (e.g., CEA, Cyfra21-1), suggesting biologically plausible links between tumor metabolism and systemic inflammation. To minimize the impact of potential multi-collinearity, tree-based models (XGBoost, Random Forest) intrinsically handled redundant predictors through random feature sampling, and additional variance inflation factor (VIF) screening confirmed that no variables exceeded VIF > 5. These correlations guided subsequent feature selection and informed the integration of multimodal variables in model construction.

Results of feature selection

This study extracted 41 imaging features from primary tumors and mediastinal lymph nodes, selecting the top 20 features based on their contribution for the machine learning model. These key features include CT values of lymph nodes (maximum and average), TLG, MTV, and SUV metabolic parameters (such as SUVmax and SUVpeak). Additionally, anatomical features such as spiculated signs and lymph node short diameter (Smm) were incorporated, as they significantly influence MLNM prediction. From the clinical and pathological characteristics, 26 indicators were extracted, ultimately identifying 15 important features, including CEA, CA19-9, tumor pathology type, FEV1, AFP, and Cyfra211. These features exhibited high relevance in MLNM prediction. SHAP analysis in Fig. 3A indicates that metabolic features such as TLG, SUVmax, and MTV contribute the most to the model, highlighting that metabolic activity and tumor burden are crucial predictors of MLNM. The short diameter of lymph nodes (Smm), as an anatomical feature, also shows significant contribution, further supporting its value in morphological prediction. Figure 3B demonstrates that CEA, CA19-9, and tumor histological type are the primary clinical and pathological driving factors, with FEV1, as a lung function indicator, also impacting the model’s results. The independence and importance of these features are clearly defined through SHAP values, providing direction for future model optimization and clinical application.

Performance of machine learning algorithms

In the training set, five-fold cross-validation was used to evaluate the performance of various machine learning algorithms. The results indicated that XGBoost performed the best, achieving an AUC of 0.90. In the test set, the AUCs for each algorithm were as follows: logistic regression (0.83), SVM (0.83), GBDT (0.88), Random Forest (0.86), CART (0.77), MLP (0.87), and KNN (0.78). Notably, XGBoost had the highest AUC in both cross-validation and test sets at 0.90, significantly outperforming other models (see Table 2). The clinical decision curve further validated the advantages of XGBoost. Within the threshold range of 0 to 1, XGBoost demonstrated high net benefits across all possible thresholds, clearly surpassing other algorithms, indicating the model’s potential for clinical application. Figures 4 and 5 present the ROC curves and clinical decision curves for each machine learning algorithm in the training and validation sets. XGBoost excelled across multiple evaluation metrics, including AUC, accuracy, recall, and F1-score, confirming its stability and superiority in predicting MLNM. Therefore, based on these evaluation results, XGBoost was selected as the optimal classifier for MLNM prediction, and its use is recommended in future clinical applications and further research.

To comprehensively compare the effectiveness of the TPC, LPC, and TLPC models in predicting MLNM, evaluations were first conducted on the test set (see Fig. 6 and Table 3). The results indicated that the integrated TLPC model achieved the highest performance, with an AUC of 0.90 (95% CI [0.883–0.957]), specificity of 0.84, and sensitivity of 0.96 (P = 0.0069). In contrast, the LPC model achieved an AUC of 0.78 (95% CI [0.713–0.751]), with a specificity of 0.73 and sensitivity of 0.84 (P = 0.0269). The TPC model had an AUC of only 0.67 (95% CI [0.647–0.703]), with a specificity of 0.46 and sensitivity of 0.56 (P = 0.7037), demonstrating significantly inferior performance.

In the validation set, the TLPC model similarly exhibited superior performance, with an AUC of 0.87 (95% CI [0.874–0.953]), specificity of 0.76, and sensitivity of 0.93 (P = 0.0087), again outperforming the other models. The LPC model recorded an AUC of 0.68 (95% CI [0.652–0.715]), with a specificity of 0.64 and sensitivity of 0.78 (P = 0.0314). The TPC model had an AUC of 0.65 (95% CI [0.623–0.685]), specificity of 0.44, and sensitivity of 0.55 (P = 0.7514), which was still suboptimal. Compared to traditional diagnostic methods (AUC of 0.71, 95% CI [0.673–0.732], specificity of 0.70, sensitivity of 0.74), the TLPC model demonstrated a significant advantage in predicting MLNM (Fig. 7 and Table 3). The TLPC model not only dominated in AUC values but also showed higher specificity and sensitivity, confirming its potential for clinical application. This indicates that the integrated model can predict MLNM more comprehensively and accurately. It highlights that combining metabolic and anatomical features of the primary tumor and lymph nodes, along with pathological and clinical characteristics, significantly enhances the predictive performance of the model.