Choice of respiratory therapy for COVID-19 patients with acute hypoxemic respiratory failure: a retrospective case series study

Research ethics

This study was conducted in accordance with the guidelines of the Declaration of Helsinki as a retrospective observational study accompanying the Kyoto Prefectural University of Medicine (KPUM) COVID-19 Registry Study (ERB-C-1810, approved by the Institutional Review Board of KPUM on 3 September 3, 2020). This study evaluated only adult COVID-19 patients. Written informed consent was obtained from all subjects and/or their legal guardian(s). The informed consent form consists of two parts: the information sheet that outlines the nature of the project, activities involved, timeframe, expectations of both the researcher and participants, data collection and storage methods, how the data will be used, and whether there will be any risk or benefit to the participants, and the consent certificate on which investigators obtained a signature from the participant. All methods were performed in accordance with the relevant guidelines and regulations.

Target patients and the choices of respiratory therapies

The KPUM Hospital is a nationally accredited first-class infectious disease-designated hospital in Kyoto Prefecture, Japan. This hospital has been performing inpatient treatment for COVID-19 patients with severe respiratory failure, mainly via referral requests from other medical institutions in Kyoto Prefecture to control centers in Kyoto Prefecture. From April 2020 to September 2021, 188 patients diagnosed as COVID-19-positive were hospitalized (Fig. 1). Of these, 112 were mildly ill patients who did not require advanced oxygen therapy. Of the 76 patients who required high-flow oxygen therapy, after excluding 14 patients who had already been hospitalized and were receiving MV and three patients who did not receive MV because of palliative care, 59 patients who started HFNC therapy immediately after admission were the subjects of this study. As a result of their choice of respiratory management, patients who were indicated for MV and managed with MV were classified into the MV group, and those who were successfully managed with HFNC were classified into the HFNC group. HFNC therapy was started using Optiflow (Fisher & Paykel Healthcare, Auckland, New Zealand) to maintain a respiratory rate of less than 30 breaths per minute by adjusting oxygen flow and oxygen concentration. The indication for MV was empirically determined by the attending physicians in charge of the patient with reference to the patient’s age, comorbidities, oxygenation assessment, and chest CT images. The major criteria were: hypoxemic respiratory failure with SpO₂ <90% or a ratio of the partial pressure of arterial oxygen to FiO₂ of <200 despite receiving the maximal FiO₂ possible with HFNC; hypercapnic respiratory failure accompanied by blood pH <7.3; respiratory rate >30 breaths per minute; and hypotension (systolic blood pressure <90 mmHg) despite catecholamine and/or fluid administration. The following additional data were collected at admission: age, gender, weight, height, body mass index (BMI), comorbidities (hypertension, diabetes, lung disease, heart disease, cerebrovascular disease), blood clinical laboratory data, pneumonia severity index (Fine et al., 1997), and Charlson comorbidity index (Charlson et al., 1987).

ROX index

The oxygen flow rate was adjusted according to the patient’s body condition, and the concentration was adjusted so that SpO₂ was maintained at ≥ 95% at rest. The ROX index was then calculated approximately 6 h after admission.

Chest CT analysis

All patients underwent CT before transfer to our hospital or immediately after admission. 3D Slicer software (ver.4.11, https://www.slicer.org/) was used to calculate the ratio of LIV by chest CT image analysis (Balbi et al., 2021; Cattabriga et al., 2020; Digumarthy et al., 2019). According to each Hounsfield unit value, the segmented lung images were color-coded using one mm-volume reconstructions. The LIVs were calculated and expressed as percentages. Chest CT images of the HFNC and MV groups were analyzed using 3D Slicer to determine the volume of the normal lung range and the ratio of the LIV (Lanza et al., 2020).

Kernel density graph

Open-source Python (ver. 3.8; https://www.python.org) with the Seaborn (https://seaborn.pydata.org) library was used for graphing.

Estimation of the cut-off points based on the physicians’ decisions

If the choice of respiratory therapy by the attending physician was considered a problem regarding the accuracy of the two-group classification, the choice was replaced with an analysis of sensitivity and specificity by plotting the percentages of MV cases with a ROX index >ROX cut-off (true positive ratio, or sensitivity) curve and the percentages of HFNC cases with a ROX index <ROX cut-off (true negative ratio, or specificity) curve vs cut-off values to identify a threshold for MV use by physicians. The cut-off value that the attending physicians subconsciously selected was considered the crossover point where the proportion of false negatives (HFNC cases with a ROX index ≤ cut-off) equaled the proportion of false positives (MV cases with a ROX index >cut-off) (Greiner, Sohr & Göbel, 1995; Yoshida et al., 2017).

We assumed that LIV directly captured the severity of lung injury; therefore, the crossover point value was obtained by superimposing the LIV cut-off value (= 35.5) for false positives and false negatives over those for the cut-off of the ROX index alone and the cut-off that varied with the ROX index. In the ROX alone judgment, it was possible to explore the effect of LIV on effects that were classified as false. Accordingly, we calculated the proportion of HFNC cases with LIV ≤ 35.5 and ROX index ≤ ROX cut-off, and the percentage of MV cases with LIV >35.5 and ROX index >ROX cut-off.

Multiple binomial logistic regression analysis

R programming language (R Core Team, 2022) was used for the multiple binomial logistic regression analysis (MLRA) to calculate the decision border (cut-off) for the classification of HFNC or MV groups. In R function glm (response∼epredictor, family = binomial (link = “logit”), dependent variable data), using a series of attributes for each patient as the dependent variable data, we described the choice of respiratory therapy, which is a predictor variable, in MV (1), HFNC (0), as binary. For example, in the classification of HFNC or MV groups using the ROX index and/or LIV as the dependent variables, logit() can be expressed as

1. ROX index alone: logit(p_i) = SCORE = ln(p_i/(1 − p_i)) = β₀+ β ₁ × [ROX index], where p_i is the probability of the event and β₀ is the intercept from the linear regression equation; β₁ is a parameter. In this case, the decision borderline for the classification is described as [ROX index] = −β₀/ β₁, where p_i = 0.5, and SCORE = 0 is the cut-off value.

2. LIV alone: logit(p_i) = SCORE = ln(p_i/(1 − p_i)) = β₀ + β₂ × [LIV], where p_i is the probability of the event, and β₀ is the intercept from the linear regression equation; β₂ is a parameter. In this case, the decision borderline for the classification is described as [LIV] = −β₀/ β₂, where p_i = 0.5 and SCORE = 0 is the cut-off value.

3. ROX index and LIV: logit(p_i) = SCORE = ln(p_i/(1 − p_i)) = β₀ + β₁ × [ROX index] + β₂ × [LIV], where p_i isthe probability of the event and β₀ is the intercept from the linear regression equation; β₁ and β₂ are parameters. In this case, the decision borderline for the classification is described as [LIV] = −β₁/β₂ × [ROX index] − β₀/β₂, where p_i = 0.5 and SCORE = 0 is the cut-off value.

Receiver operating characteristic (ROC) analysis

Under the above-described definitions of the binomial classifications associated with the HFNC or MV groups and three different cut-off lines (ROX index = 6.1, LIV = 35.5, and LIV = 4. 26 × (ROX index) + 7.89), we performed a ROC analysis to evaluate the reliability of classification by each cut-off value. In addition to the sensitivity: TP/(TP + FN), the specificity: TN/(FP + TN), ROC curve, and area under the summary ROC curve (AUC), we calculated accuracy: (TP + TN)/[Total (n = 59)]; positive likelihood ratio (PLR): sensitivity/(1 − specificity); negative likelihood ratio (NLR): (1 − sensitivity)/specificity; and diagnostic odds ratio (DOR): PLR/NLR.

Other statistical analyses

SPSS (ver. 27; IBM Corp., Armonk, NY, USA) was used for the unpaired t-test to compare group means, and Microsoft Excel was used for the χ² test for comparisons between groups. Data are shown as mean ± standard deviation (sd).

Patients’ therapeutic backgrounds

The first wave of the COVID-19 pandemic started in April 2020 in Kyoto, and five pandemic waves occurred by September 2021. From December 2020, more active use of HFNC was promoted at our facility, and as a result, the number of patients who underwent HFNC or MV management increased gradually until September 2021, the end of the study period (Fig. S1). During the 18-month study period, among the hospitalized patients, three were under HFNC therapy and were not candidates for MV therapy; these patients were excluded from this study per the hospital’s code of ethics. Fourteen patients had been mechanically ventilated under tracheal intubation by the time they were transferred to our hospital. The remaining 59 patients were the target of further analysis in this study. Of these patients, within 2.8 ± 3.6 days, 24 were indicated for MV and were changed to MV management under tracheal intubation. We compared the primary data of the 35 patients who were successfully treated with HFNC (HFNC group) and the 24 patients who required MV (MV group). Note that the number of ventilators available at our hospital could have been the upper limit of the number of patients in the MV group. However, in the study period during the COVID-19 pandemic, the number of ventilated patients never reached the upper limit of the number of ventilators available at our hospital. No statistically significant difference was detected for gender, age, body weight, height, and BMI (Table 1). Regarding the presence or absence of underlying disease, no statistically significant difference was detected, except for chronic kidney disease, which was higher in the MV group (Table S1). The primary treatment comprised antiviral drugs, such as favipiravir or remdesivir, anti-immunotherapy, mainly with dexamethasone, and anticoagulant therapy with heparin. There was no significant difference in drug therapy between the two groups (Table S2). A multidisciplinary conference was held by the attending physician and infectious disease specialist, infectious disease control team, and intensive care specialist, and baricitinib, tocilizumab, and steroid pulse therapy were given as additional anti-immunotherapies when needed. No patients were treated with monoclonal antibodies and none were vaccinated.

The patients’ blood laboratory test data showed significantly higher lactate dehydrogenase concentrations at admission in the MV group compared with the HFNC group (Table 1). The mean ROX index value in the HFNC group was significantly higher compared with the MV group (Table 1). Regarding the analysis of chest CT images by 3D Slicer, the LIVs and their proportions were significantly higher in the MV group compared with the HFNC group (Table 1) Fig. 2 and Video Clip A–F show the analysis of the chest CT images of six cases with different pneumonia severity according to 3D Slicer. The period from onset to admission to our hospital and from onset to intervention with HFNC were significantly longer in the HFNC group than those in the MV group (Table 1).

As a clinical outcome in both groups, the length of hospital stay was significantly longer in the MV group compared with the HFNC group (Table S3). Patients in the HFNC group were intubated and transferred to the MV group if their respiratory status deteriorated. Therefore, no deaths occurred in the HFNC group; however, four patients died in the MV group (Table S3). In the HFNC group, HFNC was performed for an average of 7.1 ± 10.3 (range: 1–62) days. In the MV group, the average period from HFNC to MV was 2.8 ± 3.6 (range: 0–16) days, and this was followed by 15.2 ± 23.6 (range 2–97) days of MV (Table S3).

The various cut-off levels of the ROX index and the clinical outcomes

As stated, no patients died among the 35 patients who received HFNC until their recovery (because patients who were initially receiving HFNC but who were later intubated owing to worsening respiratory status were subsequently assigned to the MV group). In contrast, four of the 24 patients in the MV group died. The ROX index values of these patients were 9.8, 7.3, 5.4, and 3.0, respectively, suggesting that the ROX index of half of the patients who died was higher than the reported cut-off values of the ROX index, which range from 2.7 to 5.99 (Prakash et al., 2021).

The MV group comprised seven patients with ROX index values ≥ 6 and five had LIV values ≥ 35.5%, indicating severe lung injury. Conversely, two of the 34 survivors in the HFNC group had a ROX index of ≤ 5. Therefore, the attending physicians selected respiratory therapy (HFNC or MV) without being bound only by the ROX index. When the cut-off value of the ROX index varied from four to seven in increments of 0.1, we calculated the proportion of HFNC patients whose ROX index was ≤ the cut-off value and the proportion of MV patients whose ROX index was >the cut-off value. The proportion of HFNC patients with ROX index values ≤ the cut-off value and the proportion of MV patients with ROX index values >the cut-off value crossed over at 25%, where the cut-off value of the ROX index was approximately 6.2 (Fig. 3A).

Next, we calculated the percentage of HFNC patients with LIV values ≤ 35.5 (we explained this LIV cut-off value in the next section) and ROX index ≤ the cut-off value and the percentage of MV patients with LIV values >35.5 and ROX index >the cut-off value. The percentage lines of both HFNC and MV patients crossed over at 17%, where the cut-off value of the ROX index was approximately 6.1 (Fig. 3B). These results mean that the judging criteria for the cut-off value of the ROX index by the attending physician was approximately 6.1–6.2, which is slightly higher than the reported cut-off value of the ROX index (2.7–5.99) (Prakash et al., 2021). Thus, by adding LIV = 35.5 as a cut-off to ROX index = 6.1, the crossover point, at which the proportion of false positives in the HFNC group matched the proportion of false negatives in the MV group, decreased from 25% to 17% under the ROX index cut-off of about 6.1. This finding suggests that the addition of LIV to the treatment decision contributed to reducing false positives in HFNC cases (HFNC cases with ROX index ≤ ROX cut-off) and false negatives in MV cases (MV cases with ROX index >ROX cut-off), and the accuracy (1 − (false positives + false negatives)/total cases) increased in the sensitivity and specificity analyses.

MLRA of the indications for HFNC and MV

MLRA was performed using the five factors involved in the decision to initiate MV management: the period from onset to admission to our hospital, the period from onset to the initiation of HFNC, laboratory examination data (lactate dehydrogenase concentration), a lung injury parameter (LIV) from chest CT imaging, and the ROX index (Table 2). Note that we did not include characteristics related to history and underlying diseases for the MLRA because these diagnostic criteria are ambiguous (Table S1). Covariates with p-values ≥ 0.05 were excluded from the regression analysis (Table 2). As a result, the results for the ROX index (odds ratio, 0.32; 95% CI [0.13–0.77]; p = 0.012) and LIV on chest CT images (odds ratio, 1.25; 95% CI [1.06–1.46]; p = 0.008) were significant. Note that the pairs plot shows significantly different distributions for the ROX index and LIV when the patients were divided into two groups (MV group and HFNC group) (Fig. S2). Next, MLRA was repeated using only the ROX index and LIV. Optimal cut-off values for the ROX index and LIV were then determined for the two management groups (38 patients who underwent MV and 35 patients who were treated with HFNC alone). As a result, when using the ROX index alone as a cut-off value, the boundary score (SCORE) for classifying patients selected for HFNC or MV was calculated as SCORE = ln(p_i/(1 − p_i)) = 4.21 − 0.69 × [ROX index], and [ROX index] = 6.1 when p_i = 0.5, indicating SCORE = 0. Therefore, the cut-off value of the ROX index was 6.1. When using LIV alone as a cut-off value, the boundary score (SCORE) for classifying patients selected for HFNC or MV was calculated as SCORE = ln(p_i/(1 − p_i)) = −8.09 + 0.23 × [LIV], and [LIV] = 35.5 when p_i = 0.5, indicating SCORE = 0. Therefore, the cut-off value of LIV was 35.5%. Finally, when using both the ROX index and LIV as cut-off values, the boundary score (SCORE) for classifying patients selected for HFNC or MV was calculated as SCORE = ln(p_i/(1 −p_i)) = −1.50 − 0.81 × [ROX index] + 0.19 × [LIV]. Therefore, the decision borderline was calculated as [LIV] = 4.26 × [ROX index] + 7.89 when p_i = 0.5, indicating SCORE = 0.

We plotted all 59 patients by ROX index and LIV values with color codes demonstrating HFNC or MV (cases of transition from HFNC to MV), and drew the distribution density as a kernel density estimation (KDE) plot (Fig. 4A). The KDE plot indicated that higher patient density was associated with more concentrated patient distribution. Next, the KDE plot was drawn separately for the HFNC and MV groups, namely 35 patients who were treated with HFNC alone and 24 patients who underwent MV (Fig. 4B). As shown in Fig. 4A, the highest density in the center of the KDE plot indicates the crossing point of three decision borderlines, and, as shown in Fig. 4B, the KDE plots, which were drawn separately for the HFNC and MV groups, overlapped noticeably near the crossing point of the three decision borderlines. These findings suggested that the physicians’ classification of HFNC or MV was problematic for patients whose indices were close to the statistically calculated cut-off values in this study. Considering that no patients in the HFNC group died, the association between a ROX index <6.1 and the prediction of mortality outcome with MV treatment must be related to the validity of the physicians’ clinical decisions, which is a limitation of this study.

Cut-off by ROX index and/or LIV for the classification of HFNC or MV

With 6.1 as the cut-off for the ROX index, 18 (75.0%) of the 24 patients managed with MV were classified as the severe group, and 32 (80.0%) of the 35 patients managed with HFNC were classified as the mild group (Table. S4). In contrast, when the LIV cut-off was 35.5%, 18 (75.0%) of the 24 patients managed with MV were classified as the severe group, and 31 (88.6%) of the 35 patients managed with HFNC were classified as the mild group (Table. S4). As shown in Fig. 3, compared with the vertical cut-off line with a ROX index of 6.1 alone, the cut-off line by MLRA SCORE LIV = 4.26 × (ROX index) + 7.89 was tilted in the positive direction of the ROX index and the LIV axes. When using the SCORE cut-off, 19 (79.2%) of the 24 the patients managed with MV were classified as the severe group, and 32 (91.4%) of the 35 patients managed with HFNC were classified as the mild group (Table. S4). Patients above the SCORE are more likely to require MV, even if the ROX index is ≥ 6.1.

Sensitivity, specificity, accuracy, DOR, and the areas under the ROC curves were compared for three decision boundaries: ROX index alone, LIV alone, and ROX index and LIV combined (Fig. 5 and Table 3). As a result, in all binomial classification confidence indices, the decision boundary by the combination of ROX index and LIV showed better values (AUC: 0.94, sensitivity: 0.79, specificity: 0.91, accuracy: 0.86, DOR: 41) compared with values obtained from the ROX index alone (AUC: 0.83, sensitivity: 0.75, specificity: 0.80, accuracy: 0.78, DOR: 12) and LIV alone (AUC: 0.89, sensitivity: 0.75, specificity: 0.89, accuracy: 0.83, DOR: 23).

These findings suggest, in terms of the accuracy rate, that classification by the MLRA SCORE cut-off line was better than that by the cut-off of the ROX index alone or LIV alone.

This MLRA analysis excluded gender, age, and BMI from the main factors influencing the need for MV, as stated. However, there are many reports in which these factors are involved in the aggravation of COVID-19. Therefore, we confirmed whether these factors affected the need for MV and whether they affected the grouping according to the three cut-off lines. The results showed that only the number of cases with BMI >25 and BMI ≤ 25 showed statistically significant uneven distribution between HFNC-positive in mild cases and MV-positive in severe cases classified by the cut-off line of LIV alone, as shown in Table S5 . Gender and age (≥ 65 years, <65 years) did not significantly affect the use of HFNC and MV in our patient cohort.