A novel nomogram to predict the risk of requiring mechanical ventilation in patients with sepsis within 48 hours of admission: a retrospective analysis

Patient definition, inclusion, and exclusion criteria

Data for 2,044 patients with sepsis admitted to the Dongyang People’s Hospital between October 2011 and October 2023 were collected and analyzed. For inclusion in the study, the patients had to meet the diagnostic criteria of sepsis 3.0: with an infection that has caused the SOFA score to increase by two points. The exclusion criteria were: 1. patients younger than 18 years old, 2. patients with respiratory failure upon admission (patients with oxygenation index less than 300 mmHg and pulse oxygen saturation (SPO₂) less than 90% upon admission), 3. patients who did not finish treatment and those who met the indications for mechanical ventilation but refused to have this intervention, and 4. patients who underwent emergency surgery.

This study was approved by the Ethics Committee of Dongyang People’s Hospital (approval number 2024-YX-096). The data was anonymously analyzed after removing personal identification information. The study was carried out in accordance with the principles of the Declaration of Helsinki and its amendments.

Retrospective data collection from medical records

Patients’ information, including gender, age, and medical history, including hypertension, type 2 diabetes mellitus, cerebral infarction, chronic lung disease (including chronic obstructive pulmonary disease (COPD)), interstitial pneumonia, and chronic pulmonary fibrosis, chronic heart disease (including cardiac dysfunction with New York Heart Association (NYHA) II grade and above), chronic liver disease (cirrhosis), chronic kidney disease (chronic renal insufficiency and nephrotic syndrome), leukemia, and malignancy status, and other serum parameters status, such as high-sensitivity C-reactive protein (HS-CRP), alanine transaminase, triglyceride, total bilirubin, creatinine, lactic acid (LA), pro-brain natriuretic peptide (PRO-BNP), cholinesterase, prothrombin time (PT), D-dimer, potassium, sodium, magnesium, calcium, hemoglobin (HB), albumin, and globulin levels, as well as white blood cell (WBC) and platelet counts, were extracted from patient records. Vital signs upon admission, including SPO₂, temperature, mean arterial pressure (MAP), heart rate (HR), respiratory rate, the Glasgow coma score (GCS), and infection sites, such as intracranial, lung, digestive tract, biliary tract, and urinary tract, were also recorded. The indications for the need for mechanical ventilation were as follows: 1. after active interventions such as drug oxygen inhalation, high-flow oxygen therapy, and non-invasive ventilator, the patient still exhibited obvious dyspnea, chest tightness, and shortness of breath, with a respiratory rate of >35 times/min, oxygen partial pressure of less than 50 mmHg or oxygenation index of <200; 2. respiratory depression occurred, and the number of breaths was less than 8/min; 3. patients who developed severe consciousness disorders, including lethargy and coma; and 4. a progressive increase in the partial pressure of carbon dioxide, accompanied by a pH of 7.20 or lower (mechanical ventilation was evaluated by physicians).

Variable screening and predictive model construction

Partially continuous variables were converted to categorical variables based on clinical normal ranges to facilitate clinical interpretation. The data were randomly divided into the modeling group and the validation group at a ratio of 7:3. Following univariate analysis of the modeling group data, multiple collinear analysis (indicated by the variance inflation factor, VIF) was conducted to confirm that no collinearities existed between any two included variables (Hsieh & Lavori, 2000). Then, a linear relationship between the variables and logitP was confirmed using the BoxTidwell function (P ≥ 0.05). The criteria for the above analysis were provided in our previous work (Wang, Chen & Wang, 2023). Finally, a multi-variable logistic regression analysis was performed, followed by a stepwise regression analysis to identify independent risk factors and establish a predictive nomogram.

Accuracy and reliability of the nomogram

A receiver operating characteristic (ROC) curve, calibration graph, and decline curve analysis (DCA) were used to evaluate the discriminatory power, goodness of fit, and clinical accuracy of the model, respectively. An area under the curve (AUC) of >0.75 indicated good discrimination ability (Silva et al., 2015). The cut-off value was determined based on the maximal Youden indexes in the ROC. The sensitivity, specificity, prediction accuracy (ACC), negative predictive value (NPV), and positive predictive value (PPV) of the model were also determined. A calibration graph was used to evaluate the consistency between predicted and observed probabilities. A P-value greater than 0.05 in the Hosmer-Lemeshow test indicated an acceptable goodness of fit of the model (Niu et al., 2017). Finally, the model was considered to exert good clinical validity when the model’s curve was far from the two extreme curves (Nattino, Finazzi & Bertolini, 2016).

Comparing the accuracy of the nomogram and other models

Two models were constructed based on SOFA scores and NEWSs to compare the predictive power of the established nomogram. Several machine-learning models were also constructed in the modeling group and validated in the validation group. The methods for the machine-learning models included C5.0, XGBOOT, and SVM. An integrated model was also established using the stacking method (Jiang et al., 2022b). The DeLong test was performed to compare the discriminatory power of our nomogram with those of other models.

Statistical analysis

Normally distributed continual variables were expressed as $\overline{X}$ ± s, and differences between corresponding groups were analyzed using the two independent-samples t-test method. Non-normally distributed continual data were expressed as the median (quartiles), and differences between corresponding groups were analyzed using the Mann-Whitney U test. Count data were expressed as rates and percentages, and differences were compared using the χ² test. All statistical analyses were performed using R (software version 4.1.2) containing packages as described previously (Wang, Chen & Wang, 2023). Statistical significance was set at P < 0.05.

Baseline data of the enrolled participants

A total of 2,336 patients with sepsis admitted to the Dongyang People’s Hospital between 2011 and 2023 were recruited in this study. After excluding 150 patients with respiratory failure at admission, one patient under 18 years old, 131 patients with acute abdominal surgery, and 10 patients who discontinued treatment, 2,044 patients were included in the final analysis. Of the 2,044, 1,431 patients were enrolled in the modeling group, and 613 patients were included in the validation group (Fig. 1). Among them, 108 patients were intubated and placed on a ventilator within 48 h after admission (108/2,044; 5.3%). Of the included patients, 25% had lung infections, 10% patients had bile duct infections, 17% had urinary duct infections, 17% had gastrointestinal infections, and 31% had infections in other anatomic sites or unknown sites. No significant difference was observed in any variable between the modeling group and the validation group (P > 0.05, Table 1).

Independent factors that predict the need for mechanical ventilation in patients with sepsis

Univariate analysis showed that 11 variables, including triglyceride, creatinine, LA, PRO-BNP, and cholinesterase levels, PT, D-dimer level, platelet count, albumin level, breathing rate, and lung infection (Table 2), were associated with the need for mechanical ventilation in patients with sepsis after admission (P < 0.001). The VIF values for each variable were less than 5, indicating no multicollinearity (Table S1). A linear relationship existed between each continuous variable and logitp (Table S2). The multi-variable logistic regression and stepwise regression analyses showed that six variables, including LA, PRO-BNP, albumin, and D-dimer levels, as well as PT and lung infection, were independent risk factors that predicted the probability of the need for mechanical ventilation in patients with sepsis within 48 h after admission (Table 3).

Establishment of the nomogram

The nomogram was developed by combining six variables. Each variable was assigned a specific score by vertically mapping the input variables to the top score line. The risk for requiring mechanical ventilation could be obtained by mapping the summed score to the risk line (Fig. 2).

The discriminatiory power, goodness of fit, and clinical accuracy of the model for data in the training set

The AUC value of the model at the 95% confidence interval (CI) was 0.820 [range 0.7689–0.870], demonstrating its good discriminatory power (Fig. 3A). The optimal cut-off score was 0.055, while the specificity score was 0.795 (95% CI [0.772–0.816]), and the sensitivity score was 0.743 (95% CI [0.643–0.843]). The ACC value at the 95% CI was 0.792 [range 0.792–0.793], the PPV value at the 95% CI was 0.157 [range 0.118–0.196], and the NPV value at the 95% CI was 0.984 [range 0.976–0.991]. The P-value of the calibration graph was 0.695, with a Brier value of 0.043 and a slope of 1, indicating a good fit (Fig. 3B). In the DCA curve, the curve of the model was very far from the two extreme curves, indicating that the model had good clinical benefits (Fig. 3C).

Validation of the model using patients in the validation group

The AUC value for patients in the validation group at the 95% CI was 0.837 [range 0.776–0.898] (Fig. 4A). The optimal cut-off value was 0.043, with a specificity score of 0.642 (95% CI [0.604–0.684]) and a sensitivity score of 0.895 (95% CI [0.790– 0.974]). The ACC value at the 95% CI was 0.657 [range 0.657–0.658], the PPV value at the 95% CI was 0.142 [range 0.098–0.186], and the NPV value at the 95% CI was 0.989 [range 0.979–1]. The P-value for the calibration graph was 0.693, with a Brier value of 0.043 and a slope of 1, indicating a good fit (Fig. 4B). The curve of the model was far from the two extreme curves, demonstrating good discriminatory power, calibration, and clinical accuracy (Fig. 4C).

Comparison of the nomogram and the models established based on SOFA scores and NEWSs

The AUC value of the predictive model established in the modeling group based on SOFA scores was 0.696 (95% CI [0.631–0.761]), while that established based on the NEWSs was 0.626 (95% CI [0.560–0.690]). Further analysis revealed that the discriminatiory power of our nomogram was higher than that of the model developed based on SOFA scores and NEWSs (the P-values in DeLong’s test were 0.001 and <0.001, respectively (Table 4, Fig. 5A). The AUC value of the model established using SOFA scores in the validation group was 0.664 (95% CI [0.565–0.763]), and the AUC value of the model established using NEWSs was 0.595 (95% CI [0.504–0.687]), significantly lower than that of the logistic model (Fig. 5B).

Comparison with other machine-learning models

The AUCs of machine-learning models in the training model varied with the method, including 0.999 (95% CI [0.998−1.000]) for C5.0, 0.820 (95% CI [0.764−0.876]) for SVM, and 0.994 (95% CI [0.987−1.000]) for XGBoost. The integrated model developed by combining the three machine-learning models was 0.996 (95% CI [0.994−0.999], Fig. S1A). The predicted lines were not close to the ideal line (Fig. S1B), indicating the bad fit of these four methods.

The AUC of the C5.0 model in the validation group was 0.809 (95% CI [0.729–0.889], Fig. 6A), and that of SVM was 0.744 (95% CI [0.654–0.835]). The AUC value of the model developed using XGBoost was 0.867 (95% CI [0.812–0.923]), and the AUC value of the integrated model developed by combining the three machine-learning models was 0.849 (95% CI [0.783–0.908]). The calibration graph indicated that the calibration of each machine-learning model was relatively poor (Fig. 6B). In the validation group, the AUC value of the logistic model was comparable to that developed by Xgboost and the ensemble but higher than that developed using C5.0, SVM, SOFA scores, and NEWSs (Table 4).