Machine learning and radiomics for predicting therapeutic efficacy in newly diagnosed sputum-negative pulmonary tuberculosis: a retrospective study

Participants

This retrospective study was conducted at Beijing Chest Hospital, Capital Medical University, and was approved by the Ethics Committee of the hospital (No. BJXK-2021-KY-06). Informed consent from patients was waived by the ethics committees of Beijing Chest Hospital, Capital Medical University (No. BJXK-2021-KY-06), considering the retrospective nature of the study and the utilization of anonymized data, while ensuring patient privacy and confidentiality. All patient data were handled in strict accordance with relevant privacy regulations, guidelines, and the principles outlined in the Declaration of Helsinki, ensuring confidentiality and anonymity throughout the study.

Retrospective data were collected from Hospital 1 and Hospital 2 between 2020 and 2023, including chest CT scans before treatment and laboratory information from patients with newly diagnosed sputum-negative but BALF-positive pulmonary tuberculosis. Missing data for variables such as ESR (erythrocyte sedimentation rate), CRP (C-reactive protein), and time from symptom onset to CT scan were imputed using the median. Two professional radiologists assessed the changes in lung lesions on CT images taken within 1 month before treatment and after intensive therapy, and in combination with the clinical evaluation by physicians, the patients were classified into progression and improvement groups. According to the inclusion and exclusion criteria, a total of 178 patients were enrolled from Hospital 1 as the training cohort, of which 119 improved and 59 progressed. A total of 77 patients were enrolled from Hospital 2 as the testing cohort, of which 51 improved and 26 progressed. The patient recruitment flowchart is shown in Fig. 1. All datasets analyzed in this study, including radiomic features, clinical variables, and corresponding outcome labels, are provided in the Supplementary Materials.

Inclusion criteria: (1) negative sputum smear and culture results, but tuberculosis confirmed by BALF microscopy or polymerase chain reaction (PCR) testing; (2) patients with drug-susceptible tuberculosis were confirmed by drug susceptibility testing (DST) using the proportion method on Löwenstein–Jensen medium or molecular testing (GeneXpert MTB/RIF assay for rifampicin resistance), and all received standard anti-tuberculosis therapy for drug-susceptible TB; (3) newly diagnosed pulmonary tuberculosis; (4) CT scans conducted within 1 month before treatment and after 2 months of standard therapy.

Exclusion criteria: (1) irregular or interrupted treatment (Hospital 1 = 35, Hospital 2 = 21); (2) history of lung surgery, trauma, or other diseases such as HIV (Hospital 1 = 21, Hospital 2 = 18); (3) images with artifacts or incomplete clinical information (Hospital 1 = 26, Hospital 2 = 16).

In imaging assessment, the evaluation is primarily based on factors such as the size, extent, density, and presence of cavities in the lesions. If new tuberculosis lesions appear on imaging or if pre-existing lesions significantly enlarge, accompanied by new pulmonary consolidation or cavity formation, this suggests imaging progression. Conversely, if the size of pre-existing lesions notably decreases or if the pulmonary inflammatory response significantly diminishes, accompanied by a density change in the lesions (e.g., from high-density to low-density areas), this indicates imaging improvement. In clinical assessment, the evaluation is primarily based on clinical symptoms (such as cough, sputum production, and fever), laboratory indicators (including ESR, CRP, serum globulin), sputum smear, and culture results. The final classification is determined through a comprehensive analysis of symptoms, microbiological findings, and imaging.

“Improvement” is only defined when symptoms, microbiology, and imaging all suggest clinical deterioration; otherwise, the condition is classified as “Progression”.

CT image acquisition

All chest CT scans were performed using a Revolution CT or LightSpeed VCT (GE Healthcare). The scanning covered the area from the thoracic inlet to both adrenal glands, with the patient holding their breath after a deep inspiration. The scanning parameters were as follows: tube voltage, 120 kV; automatic tube current modulation; detector collimation, 64 × 0.625 mm; rotation time, 500 ms; and pitch, 1.375. The image reconstruction parameters were: slice thickness, 1.25 mm; increment, 1.25 mm; field of view, 15 cm; and matrix, 512 ×512.

The CT images used in this study were acquired from two scanners (Revolution CT or LightSpeed VCT). To minimize the impact of scanner variability on imaging features, all images underwent standardized pre-processing, including resampling to a uniform voxel size and intensity normalization, to reduce biases arising from differences in resolution and scanning parameters. Moreover, the radiomic features extracted in this study—including texture, shape and first-order statistical features—have previously been validated for good cross-scanner reproducibility (Jha et al., 2021; Zhao et al., 2016). During model training, performance remained consistent across data from different scanners, indicating that scanner variability had a limited impact on feature performance and model predictive capability.

Evaluation of subjective CT findings

The primary CT signs were analyzed separately by two radiologists with rich experience in TB diagnosis, who were blinded to the final therapeutic efficacy results. The primary signs CT signs of PTB included (a) nodule, (b) TIB sign, (c) consolidation, (d) cavity, (e) fibrostripe (shown in Fig. 2).

CT signs segmentation and radiomics feature extraction

The CT images were imported into ITK-SNAP (version 3.8) for manual segmentation of tuberculosis lesions. Two radiologists with over 10 years of experience in thoracic imaging independently delineated the lesion regions of interest (ROIs) on axial CT images, including both main lesions and satellite nodules visible in the lung window. To assess segmentation reproducibility, 20% of cases were randomly selected and re-segmented by a second reader, and the intraclass correlation coefficient (ICC) was calculated for all extracted features, only features with ICC > 0.75 were retained for subsequent analysis.

To reduce the variability of radiomic features, the dataset was homogenized before feature extraction. All CT images on the Shukun technology platform were resampled to a pixel spacing of 1.0 mm × 1.0 mm × 1.0 mm and discretized with a bin width of 25 Hounsfield units (HU). Additionally, to mitigate differences in image quality and noise between different scans and ensure the reproducibility of results, z-score normalization was applied during CT image preprocessing. A total of 1,874 features were extracted from each ROI, encompassing first-order features, shape-based features, and texture features derived from gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), gray-level size-zone matrix (GLSZM), gray-level dependence matrix (GLDM), and neighboring gray-tone difference matrix (NGTDM).

Feature selection and model construction

All extracted features were standardized using z-score normalization before feature selection. To ensure that only the most contributive features for predicting treatment response are retained, variables with a very strong correlation are excluded. A correlation coefficient between 0.8 and 1 indicates a very strong correlation. Pearson correlation analysis was applied to remove highly collinear features (r > 0.8). A least absolute shrinkage and selection operator (LASSO) logistic regression algorithm with ten-fold cross-validation was then applied to identify the most valuable features (those with non-zero coefficients) by selecting the optimal regularization parameter, alpha.

All machine learning models were optimized for hyperparameters using grid search combined with ten-fold cross-validation on the training set to select the best parameter combination and prevent overfitting (Material S1). We selected three representative machine learning models—logistic regression (LR), random forest (RF), and support vector machine (SVM)—which correspond to three distinct algorithm types: linear models, tree-based ensemble methods, and kernel-based discriminative models. These models have been extensively validated in medical radiomics and clinical prediction studies, offering a balance between feature interpretability, the ability to capture nonlinear relationships, and generalizability. Considering the sample size and feature dimensionality in this study, overly complex models such as deep learning may risk overfitting, while overly simple models might fail to capture nonlinear associations between imaging and clinical features. By choosing these three models, we aimed to control overfitting, assess the robustness of different algorithms, and provide a scientific basis for future model optimization.

Based on the selected radiomic features, three models were trained on the training dataset, with the 3-month imaging and clinical follow-up results as the labels. The training cohort was randomly split into two parts with an 8:2 ratio, and the three models were trained and validated using ten-fold cross-validation on these subsets, followed by testing on an external test cohort. After tuning, the finalized models were evaluated on both the independent test cohort and an internal validation cohort to assess generalizability and detect potential overfitting. The flow chart of this study is shown in Fig. 3.

Model performance may be influenced by various factors, including feature selection and quantity, feature preprocessing and scaling, sample size, class imbalance, data quality and noise, as well as model-specific hyperparameters such as regularization strength, tree depth, or learning rate.

Statistics

All data were analyzed using SPSS 25 (IBM Corp., Armonk, NY, USA) and R software 4.2.2 (https://www.r-project.org). Continuous variables are described as mean ± standard deviation, and the kappa test was used to assess the inter-reader agreement on key CT findings. The differences in continuous variables were analyzed using an independent two-sample t-test, categorical variables were compared between groups using the chi-square test or Fisher’s exact test. To account for multiple comparisons, p-values were adjusted using the Bonferroni/FDR correction. The predictive performance of the models was quantified by the area under the receiver operating characteristic (ROC) curve (AUC) in both training and testing cohorts. Sensitivity, specificity, accuracy, and F1 score were calculated based on the confusion matrix. The DeLong test was used to compare the AUCs of the three models across both cohorts and to assess potential overfitting. Calibration curves were drawn to determine the accuracy of the model predictions. Differences were considered statistically significant at P < 0.05.

Clinical characteristics

Table 1 summarizes the clinical data, key CT findings and treatment regime of the patients. Except for the TIB sign, no significant differences were observed between the progression and improvement groups. The TIB sign was more common in the improvement group (P < 0.05). The inter-observer agreement for all key CT findings was excellent (k > 0.7, all). Distribution and proportion of key CT signs in the training, testing and validation cohorts were comparable (Material S2).

Feature selection

A total of 1,874 radiomic features were initially extracted from CT images. After removing highly correlated features (Pearson r ≥ 0.8), LASSO regression using ten-fold cross-validation identified 113 non-zero coefficient features associated with tuberculosis progression (Figs. 4 and 5, Material S3), including multiple first-order, shape, and texture features. These features were subsequently used to construct LR, RF, and SVM models. LASSO coefficients and RF feature importance scores showed a weak correlation (r = 0.362, P = 0.000, Material S3).

Model performance

Among the three models, the RF model achieved the highest predictive performance in both the training (AUC = 0.996, F1 = 0.982) and testing cohorts (AUC = 0.824, F1 = 0.832), with superior calibration compared to LR and SVM (Fig. 6, Material S3 and Table 2). However, there were no significant differences between the models (P > 0.05). Internal validation on a 42-case set further confirmed strong generalization (AUC = 0.949, Fig. 7).