Abstract
Early recognition of patients at high risk of acute lung injury (ALI) is critical for successful enrolment of patients in prevention strategies for this devastating syndrome. We aimed to develop and prospectively validate an ALI prediction score in a population-based sample of patients at risk.
In a retrospective derivation cohort, predisposing conditions for ALI were identified at the time of hospital admission. The score was calculated based on the results of logistic regression analysis. Prospective validation was performed in an independent cohort of patients at risk identified at the time of hospital admission.
In a derivation cohort of 409 patients with ALI risk factors, the lung injury prediction score discriminated patients who developed ALI from those who did not with an area under the curve (AUC) of 0.84 (95% CI 0.80–0.89; Hosmer–Lemeshow p = 0.60). The performance was similar in a prospective validation cohort of 463 patients at risk of ALI (AUC 0.84, 95% CI 0.77–0.91; Hosmer–Lemeshow p = 0.88).
ALI prediction scores identify patients at high risk for ALI before intensive care unit admission. If externally validated, this model will serve to define the population of patients at high risk for ALI in whom future mechanistic studies and ALI prevention trials will be conducted.
Acute lung injury (ALI) and its more severe form acute respiratory distress syndrome (ARDS) are examples of critical care syndromes with limited treatment options once the condition is fully established. Pre-clinical studies support a “two-hit” model of ALI/ARDS development whereby exposure to pertinent risk factors modify the development and expression of ALI/ARDS in an already susceptible host with predisposing conditions 1. The condition usually develops in patients with underlying risk factors (pneumonia, severe sepsis, trauma and aspiration) 2, 3 but is modified by different patients characteristics including genetic predisposition 4, as well as certain medical interventions (adverse ventilator settings and transfusion of alloimunised plasma) 5, 6. Animal models provide compelling evidence in support of oxidative stress, lung deformation, loss of compartmentalisation of inflammation and intravascular coagulation as the pathogenic mechanisms involved in the development of ALI/ARDS 7–14. However, many treatments targeting these mechanisms have failed to improve patient outcomes despite compelling pre-clinical data. It is probable that inadequate or delayed recognition and treatment of patients at risk of the full-blown syndrome have obscured the therapeutic window 15–19. The recent National Institute of Health workshop 20 prioritised the development of strategies to perform ALI/ARDS prevention trials.
Importantly, epidemiological data suggest that ALI/ARDS is rarely present at the time of hospital admission. Rather, ALI/ARDS appears to develop over a period of hours to days in this subset of patients at risk 21–23. Unfortunately, clinical studies are usually performed in the intensive care unit (ICU) setting, enrolling patients with established ALI/ARDS who are beyond the therapeutic window of potential prevention strategies. This delayed enrolment prevents adequate study of patients at risk.
A significant challenge with early enrolment of patients at risk of ALI/ARDS into prevention trials is the fact that the majority of patients with predisposing conditions never develop ALI/ARDS and are never admitted to the ICU. This makes the enrolment of unselected patients into ALI/ARDS prevention studies neither feasible nor efficient 21. The likelihood of ALI development depends not only on specific risk factors (from 5% with elective cardiopulmonary bypass 24 to 40% in patients with septic shock 25), but also on the presence of specific risk modifiers. These include alcohol abuse 24–27, smoking 24, 27, hypoalbuminemia 28, 29, tachypnoea 25, 28, oxygen supplementation 30, chemotherapy 25, 31 and diabetes mellitus 28, 32.
To facilitate the enrolment of patients into future mechanistic and outcome studies, we aimed to develop and validate an ALI/ARDS prediction model which incorporates risk factors and risk modifiers that are present at the time of hospital admission, before the development of ALI. This model will serve to define populations of patients at high risk of ALI, and guide the interpretation of results and the satisfactory enrolment of patients into specific groupings for putative therapeutic interventions.
MATERIALS AND METHODS
This was an observational cohort study designed to develop and validate a numerical index which accurately estimates the probability of developing ALI/ARDS. The study was approved by the Institutional Review Board and all patients provided consent to the use of their medical records for research. The retrospective derivation cohort included Olmsted County (MN, USA) residents admitted to an adult ICU in Rochester (MN, USA) from January to December 2006. Exclusion criteria included age <18 yrs, pregnancy and a previous hospital admission during the study period. The prospective validation cohort included Olmsted County residents with risk factors (see below) for ALI/ARDS at the time of hospital admission admitted from November 2008 to May 2009. Exclusion criteria for the validation cohort included age <18 yrs, pregnancy and a previous admission during the study period. All patients were cared for at a single academic medical centre as it is the only hospital system with advanced ICU capabilities in this geographical area. Trained investigators abstracted data from the electronic medical records of patients from both cohorts and confirmed presence of specific ALI/ARDS risk factors according to standardised definitions.
Outcome variable
The primary outcome of interest in this study was the development of ALI/ARDS during the hospital stay. Standard American-European Consensus Conference 33 criteria were used for determination of ALI/ARDS occurrence. Patients with possible ALI/ARDS were first identified with an electronic alert system (ALI sniffer). This system utilises a Microsoft SQL-based integrative database, ICU DataMart, where data are populated within 1 h of entry into the electronic medical record (EMR). An automatic alert was created if a patient had both the qualifying arterial oxygen tension/inspiratory oxygen fraction ratio on arterial blood gas analysis and a qualifying chest radiograph report; trigger words include “bilateral” and “infiltrate” or “oedema”. This system has been validated in previous publications and has excellent sensitivity for identifying ALI/ARDS 34. Records of alerted patients were independently reviewed by two trained investigators who underwent a structured ALI/ARDS tutorial prior to reviewing the EMR in order to confirm presence of ALI/ARDS. Interobserver agreement was measured using Kappa values with disagreements solved by consensus.
Predictor variables
For model derivation, risk factors independently associated with development or prevention of ALI/ARDS in previously published studies were evaluated. These variables had to be recorded during the first 6 h of admission to the hospital to be considered to be present. Standardised definitions were used to identify risk factors (high risk trauma 28, 35–37, high-risk surgery 24, 38–40, aspiration 28, 35, 38, 41, sepsis 21, 35, 36, 38, shock 21, 42–44, pneumonia 21, 28, 38, 45 and pancreatitis 1, 21, 46–50) and risk modifiers (alcohol abuse 24–27, smoking 24, 27, hypoalbuminemia 28, 29, tachypnoea 25, 28, oxygen supplementation 30, chemotheorapy 25, 31 and diabetes mellitus 28, 32). The validation cohort included 467 Olmsted County patients who were admitted to hospital wards (ICU and non-ICU), excluding 1-day surgical procedures, cardiac observation, paediatric and maternity wards. Participants had to have at least one predisposing condition to be included in this validation cohort. The variables needed to generate the Lung Injury Prediction Study (LIPS) score were collected prospectively by trained study coordinators from the data recorded in the EMR during the first 6 h of admission.
Statistical analyses
14 previously reported independent predictors of ALI/ARDS were included in the model derivation. Seven of these predictors were considered predisposing conditions (high-risk trauma, high-risk surgery, sepsis, shock, pneumonia, aspiration and pancreatitis). The seven remaining predictors were considered modifier conditions (respiratory rate >30 breaths·min−1, alcohol abuse, hypoalbuminemia, oxygen supplementation, chemotherapy, diabetes mellitus and smoking history). The relative weight assigned to each LIPS covariate was quantified according to the β-coefficients from logistic regression analysis in the derivation cohort. Consideration was given to the magnitude of effects reported in previous studies showing an independent association between a specific factor and ALI/ARDS. In order to assess the discriminatory power of our rule, the area under the receiver-operating curve (AUC) of the prediction scale was determined. The threshold score providing the best combination of sensitivity and specificity was determined by AUC analysis of the retrospective cohort. We evaluated the model for lack of fit with the Hosmer–Lemeshow statistic.
RESULTS
The characteristics of both retrospective (ICU) and prospective (hospital) cohorts are presented in table 1.
Retrospective derivation cohort
Of 409 patients with at least one predisposing condition at the time of hospital admission, out of 1,707 Olmsted County admissions who required ICU care during the study period, 68 (17%) developed ALI/ARDS. Tables 2 and 3 provide univariate and multivariate comparisons of specific risk factors and risk modifiers in the derivation cohort. Corresponding LIPS points are shown in table 3 as well.
The LIPS model discriminated well between patients who did and did not develop ALI (AUC 0.84, 95% CI 0.80–0.89) and was well calibrated (Hosmer–Lemeshow p = 0.60) (fig. 1a).
a) Receiver operating characteristic curve for acute lung injury (ALI) development in the derivation sample. The area under the receiver operating characteristic curve was 0.84 (95% CI 0.80–0.89) and was well calibrated (Hosmer–Lemeshow p = 0.601). b) Receiver operating characteristic curve for ALI development in the validation sample. The area under the receiver operating characteristic curve was 0.84 (95% CI 0.77–0.91) and was well calibrated (Hosmer–Lemeshow p = 0.881).
Prospective validation cohort
The validation cohort included 467 patients with at least one predisposing condition for ALI/ARDS identified at the time of hospital admission; out of 2,600 Olmsted County admissions screened during the study period. The incidence of ALI/ARDS was 7%. Performance of the LIPS was similar to that observed in the retrospective derivation cohort with an AUC of 0.84 (95% CI 0.77–0.91). The model was well calibrated (Hosmer–Lemeshow p = 0.88) (fig. 1b).
Table 4 describes the performance of the score in both cohorts. Despite similar characteristics, positive predictive value was lower in the validation cohort due to a lower incidence of ALI/ARDS in a hospital (rather than ICU) based cohort.
DISCUSSION
In this population-based cohort we developed a prediction model for identifying patients at high risk of ALI/ARDS at the time of hospital admission. The model showed good discrimination and calibration in both the derivation and validation cohorts.
Importantly, when comparing the validation cohort (hospitalised patients regardless of ICU disposition at the time of admission) to the derivation cohort (only ICU patients), the proportion of patients with risk factors who developed ALI was markedly reduced. Similar results were recently published by Ferguson et al. 21 where only 7% of hospitalised patients with sepsis, 2% with pancreatitis, 10% of patients with pneumonia and 15% of patients with witnessed aspiration developed ALI. Indeed, the majority of patients with predisposing conditions never develop ALI/ARDS and are never admitted to the ICU 21. This makes the enrolment of unselected patients into ALI/ARDS prevention studies neither feasible nor efficient without a method for identifying those who are at high risk. The failure to take into account multiple triggers that influence ALI/ARDS development has probably led to the discarding of a number of potentially important therapeutic advances in ARDS which may prove to be effective in specific and highly characterised groups of patients, particularly if applied early in the course of illness.
Our study has confirmed many but not all of the previously published ALI risk factors. In spite of a striking increase of incidence of ALI/ARDS in the elderly in the study by Rubenfield et al. 3, age did not predict ALI/ARDS development in our derivation cohort and, therefore, was not taken into account in the final model.
While some previous studies reported the increased risk of ALI/ARDS in the elderly 5, 27, 51, other studies have not confirmed this association 52–54. It could be argued that elderly patients seem to have an increased incidence of ALI/ARDS as they tend to have more sepsis, pneumonia and aspiration, and require more medical interventions. However, in patients admitted to the hospital with a risk factor (pneumonia or sepsis), age does not seem to increase the risk of ALI/ARDS development. Indeed, recent work implies that incidence of ALI/ARDS due to community-acquired pneumonia is lower in patients aged ≥85 yrs 52.
Risk prediction models have been increasingly used to identify high-risk patients who may benefit from specific interventions. While their accuracy and precision are often limited, the models developed for severe pneumonia 55, 56 and perioperative cardiovascular complications 57 have been extensively used in both clinical practice and research. This study is the first attempt to develop a similar risk-prediction tool for the development of ALI/ARDS in patients at risk.
The overall performance of the LIPS tool is modest and it is probable that the model could be refined by: 1) separating specific predisposing conditions (refining the high-risk surgery according to a surgery type: cardiovascular, thoracic or acute abdomen); 2) utilising sophisticated analytic methods such as neural network analysis and recursive partitioning; and 3) adding additional pertinent variables as their association with ALI/ARDS are described.
Nonetheless, the LIPS model efficiently discriminates the patients who have a small chance of developing ALI/ARDS (good specificity), while maintaining appropriate sensitivity as a screening tool. Through the early and accurate identification of patients at high risk of ALI/ARDS at the time of hospitalisation, the model will allow timely and efficient enrolment of patients into future ALI/ARDS mechanistic studies and prevention trials. If externally validated, this tool may also be used in clinical practice to alert providers of patients who are at particular risk of ALI/ARDS.
The LIPS model utilises variables that are clearly defined and routinely available in the medical record. It does not require testing beyond the standard of care and is not restricted to an ICU population. It identifies patients early, at the time of hospital admission, and is validated for hospitalised patients irrespective of their required intensity of care at the time of admission. The population-based sample increases generalisability by avoiding the referral bias often found in tertiary academic medical centres. However, the most important limitation of our study is the fact that, although population based, both cohorts come from a single institution with specific practice patterns, in addition to a suburban homogenous population. The small sample size poses a significant potential for over fitting the logistic regression model, and further refinement and validation is needed prior to clinical use of this tool. The US Critical Illness and Injury Trials Group are currently testing the external validity of the LIPS (NCT00889772).
In conclusion, we have developed and validated an efficient and effective prediction tool for evaluating the risk of ALI/ARDS at the time of hospital admission. As the majority of patients with predisposing conditions never develop ALI and are never admitted to the ICU, our prediction model can facilitate the timely and efficient enrolment of patients into mechanistic and outcome studies, as well as future ALI prevention trials. Since ALI patients represent an aetiologically diverse group, focus should be on defining subgroups that could benefit from particular target therapies. Nevertheless, multicenter validation is required before large-scale screening projects are performed.
Footnotes
Clinical Trial
This study is registered at ClincalTrials.gov with identifier number NCT00889772.
Statement of Interest
None declared.
- Received March 8, 2010.
- Accepted June 2, 2010.
- ©ERS 2011
REFERENCES
