Knowing the likelihood of failure of noninvasive positive pressure ventilation (NPPV) in patients with exacerbation of chronic obstructive pulmonary disease (COPD) could indicate the best choice between NPPV and endotracheal intubation instituted earlier. For this purpose, two risk charts were designed (at admission and after 2 h of NPPV) that included all relevant measurable clinical prognostic indicators derived from a population representing the patients seen routinely in clinical practice.
Risk stratification of NPPV failure was assessed in 1,033 consecutive patients admitted to experienced hospital units, including two intensive care units, six respiratory intermediate care units, and five general wards. NPPV was successful in 797 patients.
Patients with a Glasgow Coma Score <11, acute physiology and chronic health evaluation (APACHE) II ≥29, respiratory rate ≥30 breaths·min−1 and pH at admission <7.25 have a predicted risk of failure >70%. A pH <7.25 after 2 h greatly increases the risk (>90%). The risk charts were validated on an independent group of 145 consecutive COPD patients treated with NPPV due to an acute ventilatory failure episode. To identify patients with a probability of failure >50%, the sensitivity and specificity were 33% and 96.7% on admission and 52.9% and 94.1% after 2 h of NPPV, respectively.
The prediction chart, based on data from the current study, can function as a simple tool to predict the risk of failure of noninvasive positive pressure ventilation and thus improve clinical management of patients tailoring medical intervention.
- Chronic obstructive pulmonary disease
- endotracheal intubation
- noninvasive ventilation
- respiratory failure
Noninvasive positive pressure ventilation (NPPV), in patients with exacerbations of chronic obstructive pulmonary disease (COPD) and respiratory acidosis, reduces the intubation rate and mortality 1–9. Operated by well trained teams, NPPV is effective and safe in both intensive care settings 10 and general respiratory wards 8. A randomised, clinical trial showed that NPPV also reduces mortality in COPD patients in the intensive care unit (ICU) within the inclusion criteria for intubation 6. Nevertheless, two recent consensus guidelines on NPPV in acute respiratory failure (ARF) recommend that NPPV should not be used as a substitute for endotracheal intubation and invasive ventilation when the latter is clearly more appropriate 11, 12.
The likelihood of failure of NPPV is crucial in deciding if and when to apply this ventilatory technique. However, attempts to predict the success of NPPV have only been made in single-centre studies that included a limited number of patients 13–15. Furthermore, results from randomised, controlled studies could be affected significantly by the research setting, e.g. strict selection criteria of patients or a unique care location. This limits the possibility of generalising the results to the daily clinical practice of general hospitals 16.
The purpose of the current study was to assess the risk of NPPV failure in a large unselected population admitted to different hospital units with expertise in the application of NPPV and to build a risk chart of failure of NPPV to be used in hospitals.
From December 1998 to March 2000 prospective data was collected from 1,033 consecutive patients affected by COPD exacerbation and respiratory acidosis that were treated by NPPV in addition to standard medical therapy (oxygen supplementation, systemic corticosteroids, inhaled bronchodilators, antibiotics, and diuretics if needed). These patients were admitted to two ICU, five general wards and six respiratory intermediate intensive care units (RIICU) where NPPV is the first-line intervention for such patients 17. The admission criteria to different units were based on the need for an appropriate level of care, but the personnel in each unit were well trained in the use of NPPV, with ≥3 yrs experience. The various units used different types of ventilators, either intensive care ventilators (Servo 300; Siemens Elema, Solna, Sweden. Puritan Bennet 7200; Puritain Bennett Co., Overland Park KS, USA) or less sophisticated ventilators (BiPAP Respironics; Respironics Inc., Murrysville, PA, USA. O'nyx Mallinckrodt; Mallinckrodt Inc, Minneapolis, MN, USA. Helia Saem; Saime, Savigny-le-Temple, France) via face and/or nasal mask. Every patient who received NPPV because of hypercapnic respiratory failure and respiratory acidosis due to exacerbation of COPD was included in the current observational study. Post-operative patients and patients receiving NPPV for weaning were excluded. The definition of COPD exacerbation was in accordance with that of the American Thoracic Society 18. NPPV failure was defined as the need for endotracheal intubation. Pre-determined criteria for endotracheal intubation included: 1) worsening of pH and carbon dioxide tension in arterial blood (Pa,CO2) in spite of correct NPPV administration (e.g. pH ≤0.04 and Pa,CO2≥0.8 kPa (≥6 mmHg)); 2) the need to protect the airways (coma or seizure disorders) or to manage copious secretions; 3) haemodynamic instability (heart rate <50 beats·min−1 with loss of alertness, and/or systolic blood pressure <70 mmHg); and 4) agitation and inability to tolerate the mask.
The following data were collected for every patient. 1) General demographic information (age, sex, weight and height) and clinical data. Severity of illness was assessed by acute physiology and chronic health evaluation (APACHE) II score 19, and neurological status was assessed by the Glasgow Coma Scale (GCS) 20. 2) Data relative to the institution of NPPV, including cause of exacerbation, arterial blood gases (before beginning NPPV, after 2 h and at discharge), respiratory rate (RR), cardiac frequency, length of stay in hospital, and total hours of ventilation. 3) Data on comorbidities, previous use of home mechanical ventilation and long-term oxygen therapy.
Variable definition and statistical analysis
The outcome variable was defined as failure of NPPV due to invasive ventilation or death.
After descriptive analysis, univariate analysis was performed using the unpaired t-test to examine differences in continuous variables. Multivariable logistic regression was used to analyse the independent effect of each variable on the outcome (treatment failure) and the strategy used to determine which variables should be included in the multivariable analysis was based on whether the variable was of direct interest or merely a confounder. Variables of interest were included, whilst confounders that showed little or no effect, or variables related to a very small number of patients, were eliminated. After the elimination of these confounders, factors of interest were eliminated if their effect was not statistically significant at the 5% level (except pH between 7.25–7.29 at admission due to the interest of pneumologists for this category of pH and to allow comparability between the two charts). This strategy may, of course, be considered rather arbitrary, as statistical significance at the 5% level does not necessarily imply that the variable will be a useful predictor. Also, variables such as body weight, which for many patients was not measured, were excluded as these missing values reduce the effective sample size, and, if they were not available for a new patient, it would not help in the prediction of the outcome of treatment. However, this procedure was considered more efficient in the search for true prognostic variables than an automatic step up or step down elimination procedure. All tests and p-values are two-tailed.
Two charts of failure risk were built from the final predictive models obtained using logistic regression; they refer to the proportion predicted to fail with NPPV treatment at admission and after 2 h of NPPV. The proportions predicted to fail were calculated using the odds ratios (OR) derived from the models. The predicted risks of failure were graphically depicted in risk charts with changing colours showing the increasing level of risk, similar to the risk charts recently proposed by the European Societies of Cardiovascular Disease Prevention 21–25. To graphically graduate risk intensity, four different colours were used according to the quartile of predicted risk probabilities. The aim was for the predicted risk of failure to be used as a rule to decide whether or not to intubate. To explore this possibility, arbitrary probabilities of failure equal to 25%, 50% and 75% were selected as possible cut-off values and sensitivity, specificity and percentage of correctly classified patients were calculated for each cut-off value.
The chart of risk for failure of NPPV was evaluated in an independent sample of 145 consecutive patients admitted during a 3-month period to three units (one general ward, one RIICU, and one ICU) to be treated with NPPV, due to an acute episode of COPD exacerbation.
Accuracy and generalisability
Accuracy, defined as the degree to which predictions match outcomes, and generalisability, defined as the ability to provide accurate predictions in different samples of patients 26, represent two important aspects of prognostic assessment and are essential if prognostic evaluation is aimed to improve clinical management of patients. Inaccuracy can be attributable either to errors in calibration (the predicted probabilities may result in a too high or too low) or to errors in discrimination (relative ranking of individuals may be out of order).
Concerning accuracy, the calibration of the model was assessed by evaluating expected to observed outcomes across deciles of risk, and discrimination was measured as the area under the receiver operating characteristic (ROC) curve (C statistic) 27.
Concerning generalisability, the predictive performance of the model was assessed through external validation, using an independent sample and comparing expected to observed outcomes across deciles of risk, and in totals and measuring the area under the ROC curve 27.
A total of 1,033 patients were recruited in the 13 centres. All the patients were treated with NPPV for the management of ARF due to COPD exacerbation. The main demographic and physiological characteristics at the start of NPPV are shown in table 1⇓.
Table 2⇓ summarises the outcome data in the study population. NPPV was performed successfully in 797 patients (77.1%) until the normalisation of arterial blood gases. Among the 236 patients who failed, 185 patients were intubated (17.9% of the total and 78.4% of those for whom NPPV failed) and 51 died without intubation due to a previous “do-not-intubate” order. Among the 185 intubated patients, 94 patients successfully completed the treatment and 91 patients died. In total, 142 patients died (13.7% of the total and 60.2% of those for whom NPPV had failed). The intubation rates varied among the centres from 0 (zero intubated out of 72) to 72.2% (39 intubated out of 54). However, the rates were highest in the ICU at 50% and lower in general wards (13.6%) and in RIICU (15.7%).
Table 3⇓ shows the means of variables which may be associated with a successful outcome. Successful patients were younger, less severe cases and had better values of Pa,CO2, pH, RR, and oxygen arterial tension/inspiratory oxygen fraction.
Table 4⇓ shows the final multivariable models including all the statistically significant predictors of NPPV failure. The variables measured on admission that were found to significantly increase the probability of NPPV failure were: APACHE II score ≥29 (OR = 3.30, p<0.001), GCS between 12–14 (OR = 2.29, p<0.001) or GCS ≤11 (OR = 4.40, p<0.001), pH <7.25 (OR = 1.97 p<0.05), and RR between 30–34 breaths·min−1 (OR = 1.83, p<0.05) or >35 breaths·min−1 (OR = 2.66, p<0.001). To allow comparison between admission and after 2 h, it was only possible to examine 594 patients out of 1,033 due to missing values, either for early failure or a lack of blood gas assessment. After 2 h of NPPV, the main factor influencing the outcome was the pH value: if pH <7.25 the OR for failure is 21.02 (p<0.0001), whereas if pH after 2 h is between 7.25–7.30, the OR is 2.92 (p<0.005). Also, all the other variables remained statistically significant: APACHE II score ≥29 (OR = 4.79, p<0.001), GCS between 12–14 (OR = 1.93, p<0.05) or GCS ≤11 (OR = 5.16, p<0.001), RR between 30–34 breaths·min−1 (OR = 2.67, p<0.001) or >35 breaths·min−1 (OR = 4.95, p<0.001). These models were then used to predict the risk of NPPV failure. Accuracy of the predictions was good as there were no statistically significant differences between expected and observed failures when compared across deciles of risk (at admission: Chi-squared = 3.46, degrees of freedom (dof) = 8, p = 0.9023; after 2 h: Chi-squared = 1.33, dof = 9, p = 0.9982). The final models had a high predictive capability, as illustrated by the values of the C statistic: 0.81 at admission (95% confidence interval (CI): 0.78– 0.84) and 0.88 after 2 h (95% CI: 0.85–0.90). The C statistic, which is equal to the area under the ROC curve, is a measure of the ability of the model to discriminate between patients who failed or succeeded with the NPPV treatment. The comparison of the two ROC curves was statistically significant (p<0.001), implying that the ROC curve after 2 h of NPPV is more efficient for prediction (fig. 1⇓).
The risk chart at admission (fig. 2⇓) shows, in the top right corner, the lowest predicted probabilities of NPPV failing. These probabilities increase moving across the table and reach the highest peak in the bottom left corner. In detail, patients with pH >7.30, and APACHE II <29 have the lowest predicted probabilities of NPPV failing (∼10%), probabilities which increase when RR increases and GCS decreases. In contrast, patients with pH <7.25, APACHE II ≥29 and GCS <11 have a predicted risk of failure >70%.
The risk chart after 2 h of NPPV is interpreted similarly (fig. 3⇓), but the predicted percentages of failure increase and are ∼90% when pH is between 7.25–7.29, APACHE II is ≥29 and GCS ≥11. If pH is <7.25 after 2 h of ventilation, this greatly increases the risk of failure, especially if APACHE II is ≥29.
The generalisability of these models was assessed through an external validation using an independent sample of 145 patients, and the results were encouraging. There were no statistically significant differences between expected and observed failures when compared across deciles of risk and in totals at admission (Chi-squared = 2.96, dof = 8, p = 0.9371) and after 2 h (Chi-squared = 3.53, dof = 9, p = 0.9397), and the C statistic was 0.71 at admission (95% CI: 0.63–0.79) and 0.83 after 2 h (95% CI: 0.76–0.89). Again, the comparison of the two ROC curves was statistically significant (p<0.001, fig. 4⇓).
To explore the possibility of using the predicted risk of failure as a rule to decide whether or not to intubate, the following probability of failure was arbitrarily chosen: 25%, 50% and 75% as possible cut-off values and the corresponding scenarios are illustrated in table 5⇓. Sensitivity decreases when risk of failure increases, whilst specificity increases when risk of failure increases. The percentage of patients correctly classified is quite good and reaches its maximum at a cut-off value of 50% (82 and 85%, respectively at admission and after 2 h).
Prediction of success/failure of NPPV in the “real world”
The present study evaluated a large unselected group of patients with acute exacerbation of COPD and respiratory acidosis treated with noninvasive ventilation in different care settings (ICUs, respiratory intermediate units and general wards; all were experienced care settings). The current authors used these data to assess the risk of NPPV failure and built two risk charts for admission and after 2 h of ventilation. These charts could be used to determine the possibility of failure or success of NPPV in patients with acute decompensate COPD and to help the clinical decision-making process. In particular, given that the Gold Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines 28 suggested an initial trial of NPPV for most patients anyway, the decision to continue or not continue with NPPV can be greatly helped by the risk chart of failure tailored after 2 h of noninvasive ventilation.
The mortality rate in the current study is similar to that observed in the most quoted controlled trials on NPPV in carefully selected patients with acute exacerbation of COPD 1–3, 5. The present study showed also that outside a specialised research setting, in the everyday clinical environment, NPPV can be effectively used in all COPD patients, such as those with pneumonia or with a lower GCS who might otherwise be excluded in a restricted clinical trial.
A series of randomised trials on noninvasive ventilation in acute exacerbations of COPD was published showing the effectiveness of NPPV, but in single ward settings. Nevertheless, the possibility to predict at hospital admission the chance of success of NPPV in patients with exacerbation of COPD should be based on a patient population with a variable degree of severity of illness. Anton et al. 15 studied only 44 patients in a single centre and derived a regression equation using six parameters, including the baseline forced expiratory volume in one second and Pa,CO2 in a stable condition, that may not be available at all or not obtainable from older records at the time of admission. In contrast to Anton et al. 15, the present authors preferred not to use change variables as they are more likely to indicate the rapidity with which the patient improves (and consequently the length of stay in the hospital) rather than the failure/success of NPPV. Moreover, given that, in practice, the two values are often not available for every patient, the inclusion of changes in the risk chart would have reduced its usefulness for prediction, as well as increasing the sampling error due to the reduction in sample size. Indeed, there was already a reduction in sample size from admission to 2 h in the present study, and to have included the changes in the levels would have caused the sample size at 2 h to have been even further reduced.
Data from a multicentre study performed in British respiratory general wards showed that if pH and/or Pa,CO2 improved after 1–4 h, successful NPPV was probable 9.
In the large population of unselected patients in the present study, on admission, severity of illness and neurological impairment, respectively (assessed by APACHE II score and GCS), were found to influence significantly the probability of NPPV failure together with RR and pH. Clear differences were found between the patients who succeeded and those who failed, with respect to the predictor variables observed at admission and after 2 h of treatment. In particular, after 2 h of NPPV, the main factor influencing the outcome was the pH value. Since pH seems to be a very important variable in predicting failure of NPPV and there is much discussion about which is the correct cut off to choose, the current authors evaluated in both models a pH <7.25 and a pH ≥7.25 but <7.30, even if the latter was not statistically significant at admission. However, given that the decision to intubate is based on clinical judgement, and this judgement is also probably based on the values of predictor variables, there is a danger of falling into a circular argument. For example, patients with pH <7.2 tend to be judged to be in need of intubation and, therefore, pH <7.2 significantly predicts intubation. However, in the current analysis the cut-off values of the predictor variables were decided a priori, which should reduce this effect.
The presence of pneumonia, at admission and after 2 h, did not significantly influence the probability of NPPV failure in the patients of the present study. Ambrosino et al. 13 reported a high failure rate in nine COPD patients with pneumonia in a retrospective single centre study, but this was not confirmed by either a recent meta-analysis 8 or a prospective, randomised controlled study on NPPV in community-acquired pneumonia 29. In agreement with the current study, previous studies have reported that severity of illness scores 2, 14, 30 and pH 1, 2, 5, 31, 32 at enrolment are independent risk factors for NPPV failure in patients with exacerbations of COPD. A reduction in RR with NPPV has been shown in a number of studies, with larger falls generally being associated with a successful outcome 2, 3, 13, 14. Clearly, it is not only clinical status that can improve the outcome of NPPV e.g. intolerance of the mask can lead to failure of NPPV 10. However, this cannot be predicted for individual patients before starting NPPV; the patient should try NPPV for a minimum of 1–2 h, adjusting the mask and changing the interface. It should be noted that the present study population included a fairly high percentage of COPD patients, mechanically ventilated at home. This is more typical for Italy than other European Countries, and it could have some impact on the generalisability of results as these patients are “experts” at NPPV.
Applicability of predictive risk charts
The charts of risk for NPPV failure showed a good generalisability assessed on an independent sample of patients with acute exacerbation of COPD. Calculation of the area under the ROC curves showed a high accuracy of the prediction, and the rate of the patients correctly classified is >75% at each cut-off value arbitrarily chosen. Since prediction improves after 2 h of NPPV and the GOLD guidelines suggest an initial trial of NPPV for most patients anyway, the decision to continue or not continue with NPPV can be greatly helped by a risk chart of failure tailored after 2 h of noninvasive ventilation. This discussion should also focus on which cut-off value is the best choice i.e. above which risk of failure it is suggested to intubate. At one extreme, the risk of failure can be considered to be ≥0%, which implies that all the patients will fail with NPPV treatment and will be intubated, with a sensitivity of 100% and a specificity of 0%. At the other extreme, the risk of failure can be considered to be equal to 100%, which implies that all the patients will succeed with NPPV treatment and nobody will be intubated, with a sensitivity of 0% and a specificity of 100%. Clearly, these two options are only hypothetical; more realistic ones are depicted in table 5⇓. The risk of failure equal to 25% appears as a good option, as sensitivity and specificity are both at high levels, particularly after 2 h. However, a risk of failure equal to 25% can be considered as a low risk and several people will be indicated for intubation without really needing it (false positive). If the risk of failure of NPPV equals 75%, fewer people will be indicated for intubation, but some people who really needed it could be missed (false negative). Thus, perhaps a risk of failure equal to 50% could be a useful compromise for whoever wants to start using the predictive chart of risk, keeping in mind that this cut-off value could be changed if clinical experience suggests it.
Furthermore, it would be interesting to assess if both risk charts (admission and after 2 h) are useful in the clinical practice. In this case, the risk charts will be used consecutively, which will maximise specificity, but lower sensitivity. One ends up surer that positive test results represent failure of NPPV, but there is an increased risk of missing people who really need intubation.
Indeed, the GOLD guidelines suggest trying NPPV initially on all patients and only after 2 h of NPPV can a decision then be made whether to continue or not. In this case, only the risk chart of failure tailored after 2 h of noninvasive ventilation can be used, but it should greatly help the decision.
The efficacy of noninvasive positive pressure ventilation in acute exacerbation of chronic obstructive pulmonary disease is so well documented that international guidelines 28 recommend it as the first choice treatment of acute respiratory failure with respiratory acidosis. Nevertheless, given that noninvasive positive pressure ventilation is used in a variety of care settings, it may be important to know the likelihood of failure to decide upon the best choice between noninvasive positive pressure ventilation and endotracheal intubation instituted earlier. The prediction charts are based on data collected by the present authors and take into consideration all relevant clinical prognostic indicators and are derived from a population representing the patients seen routinely in clinical practice. Thus, the current authors think they could greatly help the decision on clinical management of the patient. Using the chart, it is possible to predict “a priori” the probability of noninvasive positive pressure ventilation failure and reduce the useless and prolonged use of noninvasive positive pressure ventilation in patients with respiratory acidosis due to chronic obstructive pulmonary disease exacerbation.
The authors would like to thank the Eleonora Lorillard Spencer Cenci Foundation for its contribution to the research.
The following physicians participated in the Italian NPPV study group: A. Bellone, R. Della Porta, A. LoCoco, M. Demsar, G. Gadaleta, M. Moretti, P. Parigi, G. Pinelli, M. Piattella, A. Potena, A. Rossi, F. Turati, A. Vianello, M. Vitacca.
- Received July 19, 2004.
- Accepted November 8, 2004.
- © ERS Journals Ltd