Abstract
The aim of this study was to develop and validate a new method: a classification and regression tree (CART) based on easily accessible measures to predict mortality in patients with stable chronic obstructive pulmonary disease (COPD).
This was a prospective study of two independent prospective cohorts: a derivation cohort with 611 recruited patients and a validation cohort with 348 patients, all followed for 5 yrs. CART analysis was used to predict 5-yr mortality risk using the following covariates from the derivation cohort: age, % predicted forced expiratory volume in 1 s (FEV1), dyspnoea, physical activity, general health and number of hospital admissions for COPD exacerbations in the previous 2 yrs.
Age (≥75 or <75 yrs) provided the first branch of the COPD-CART. The highest mortality risk (0.74) was seen in patients >75 yrs of age with higher levels of dyspnoea and FEV1 <50% pred. Patients with the lowest risk of 5-yr mortality (0.04) were <55 yrs of age with FEV1 >35% pred and one or no recent hospitalisations for COPD exacerbations.
A simple decision tree that uses variables commonly gathered by physicians can provide a quick assessment of the severity of the disease, as measured by the risk of 5-yr mortality.
Determining the likelihood that a patient with chronic obstructive pulmonary disease (COPD) will experience an adverse event, such as hospitalisation or death, can help guide a clinician in establishing a hierarchy of risk that is useful for designing a therapeutic strategy. Until 2004, when Celli et al. [1] published the BODE (body mass index (BMI), obstruction, dyspnoea and exercise) index, the way to determine the severity and prognosis of patients with COPD was based largely on the forced expiratory volume in 1 s (FEV1). Since then, other multidimensional scales have been developed. These new scales aim to improve the prognostic ability of the BODE index while simultaneously making them easier to use in daily clinical practice [2–4].
The BODE index and similar instruments are typically created using linear and additive models. An alternative approach is to use a classification and regression tree (CART) [5]. Creating a CART centres around statistically optimal clustering of patients based on predictors that produce maximum separation among the subgroups and minimum variability within these subgroups with respect to the outcome. The result is a representation, in tree form, of a decision rule with a hierarchical sequential structure that can easily be applied in clinical practice. CART analysis is gaining widespread popularity in the health sciences as a screening method for variables, for summarising large datasets, or as a means of devising prediction rules for rapid and repeated evaluation [6, 7].
To date, CART analysis has not been applied to gauging the severity of COPD. The aim of our study was to develop and validate a CART to predict mortality in patients with stable COPD.
Methods
We surveyed all patients previously diagnosed with COPD who regularly visited five outpatient clinics affiliated with a teaching hospital in the interior district of Biscay, Spain with a catchment area of 300,000 rural and urban inhabitants. We prospectively assembled two cohorts of patients with stable COPD. The derivation cohort was recruited from February 1998 to February 1999. The validation cohort was recruited from January 2003 to January 2004. Patients in each cohort were followed for 5 yrs.
Patients were consecutively included in one of the cohorts if they had been diagnosed with COPD for ≥6 months, had been under treatment for ≥6 months and had been clinically stable (no increase in respiratory symptoms or changes in treatment) for the 6 weeks prior to inclusion. Other inclusion criteria were FEV1 <80% predicted, with a FEV1/forced vital capacity ratio of <70%, and a bronchodilation test with FEV1 change of <200 mL and <15% of the baseline value. Patients were not eligible for the study if they had been diagnosed with asthma, had extensive residual pulmonary tuberculosis, or a neoplastic process, or were suffering from any problem that could prevent effective collaboration.
Variables evaluated for the development of the COPD-CART were those previously included in the BODE index and in two of its offshoots, the HADO (health, activity, dyspnoea and obstruction) score [2] and the ADO (age, dyspnoea and airway obstruction) index [3]: dyspnoea, FEV1 % pred, level of physical activity (PA), overall health status, age, BMI and 6-min walking test (6MWT). We also evaluated the impact of the number of COPD-related admissions to the hospital in the prior 2 yrs.
Patients’ perceptions of their dyspnoea were categorised using a four-degree scale adapted from that of Fletcher et al. [8] (degree 1: “dyspnoea only with intense and strenuous exercise”; degree 2: “capable of walking at the same pace as other people my age on the level”; degree 3: “capable of walking on the level at my own speed without dyspnoea but incapable of walking at the same pace as people my own age”; and degree 4: “dyspnoea after walking slowly for 100 m” or “dyspnoea when resting or after slight effort, such as getting dressed”).
Spirometry was conducted following Spanish Respiratory Society criteria [9]. Predicted values were those of the European Community for Steel and Coal [10]. COPD severity was defined according to the four levels established by the American Thoracic Society (ATS) (FEV1 <35, 35–49, 50–64 or ≥65% pred) [11].
In the baseline personal interview, patients were asked about the types of PA they performed. Special emphasis was placed on walking, with questions about time spent walking and distance covered. The level of PA was defined as the time patients spent walking during their leisure time, and was classified as very low (does not leave the house, life is limited to the bed or armchair or to doing some domestic chores, or leaves the house but walks <100 m), low (engaging in light physical activity, such as walking, for <2 h·week−1), medium (engaging in light physical activity, such as walking, for 2–4 h·week−1) or high (engaging in light physical activity, such as walking, for >4 h·week−1), as in previous studies [12, 13].
The 6MWT was carried out according to ATS guidelines [14]. Two tests were conducted with a 30-min break in between; the best result was selected.
BMI was calculated by dividing weight in kilograms by the square of height in metres.
Health-related quality of life (HRQoL) was assessed using the St George’s Respiratory Questionnaire (SGRQ), an instrument specifically designed to assess HRQoL in patients with COPD [15]. We used the version that has been translated into and validated in Spanish [16].
Overall health was assessed by means of a single question: “In general, how would you characterise your health?” The four possible answers were bad, fair, good and very good or excellent.
The study was approved by the research committee of our hospital (Hospital de Galdakao-Usansolo, Galdakao, Spain). Patients provided informed consent to take part in the study.
Main outcomes
Vital status was initially determined by telephone calls made to patients or their next of kin. All reported deaths and dates of deaths were confirmed by reviewing medical reports, examining the hospital database and public death registries, or both. Deaths were considered confirmed if the record matched the subject for name, sex and date of birth. In all cases, the cause of death was based on the hospital reports and public death registries.
Statistical analysis
Comparisons between the derivation and the validation sample were performed using Pearson’s Chi-squared test and the unpaired t-test. Univariate analysis between mortality and the covariates was performed using Pearson’s Chi-squared test. Logistic regression was applied to obtain the area under the receiver operating characteristic curve (AUC) for each covariate, with 5-yr mortality as the dependent variable [17].
CART analysis was used to predict mortality risk in a 5-yr period with the following covariates: age, FEV1, dyspnoea, physical activity, general health and number of hospital admissions in the previous 2 yrs. CART-COPD was calculated by applying a recursive partitioning algorithm to the data. The algorithm tests for statistically optimal clustering of patients based on predictors that produce maximum separation among the clusters and minimum variability within these clusters with respect to the outcome. The predictor causing the largest separation was situated at the top of the tree, followed by the predictor causing the next largest separation and so on. Splitting continues until the subgroups reach a minimum size or until no improvement can be obtained. Risk of death was the outcome of interest. Therefore, patients were clustered, depending upon the values of the predictors, on groups with similar predicted risk of death. The minimal and maximal scores are the minimal and maximal predicted risk of death by the CART model.
To test the predictive ability of the CART model created from the derivation cohort, we applied it to patients in the validation cohort. The predictive ability of the CART was evaluated by sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and AUC. These measures were also calculated in the validation sample for the BODE, HADO and ADO indices. Various cut-off points were used to estimate the sensitivity, specificity, PPV, NPV and AUC changes according to the selected cut-off point for each instrument. Those cut-off points were chosen as they were some of those originally considered by the authors of the scores, as they complemented previous studies.
To test the calibration, we applied the Hosmer–Lemeshow goodness-of-fit test. As the risk of death predicted by the CART-COPD was subject to prediction error, predicted risk of death at each node was estimated by a random value at the 95% confidence interval of the predicted risk of death given by the CART-COPD at that node. Therefore, the estimated risk of death for each patient by the CART-COPD was compared to the observed mortality rate by the Hosmer–Lemeshow test. Calibration was also evaluated for the HADO, BODE and ADO scores by the Hosmer–Lemeshow test. The AUCs of the three severity scores were compared to the AUC of the CART model. ANOVA was performed to compare BODE, HADO, ADO and SGRQ across CART quartiles of mortality risk. The Scheffe test was performed for multiple comparisons.
All effects were considered statistically significant at an α-value of 0.05. Data analyses were performed using SAS for Windows version 9.2 (SAS Institute Inc., Cary, NC, USA) [18] and R version 2.11.1 (R Development Core Team, Vienna, Austria) [19].
RESULTS
The derivation cohort included 611 patients; the validation cohort included 348 patients. Statistically significant differences between the cohorts were observed for sex, age, dyspnoea, FEV1 % pred and previous hospital admissions for COPD exacerbations in the preceding 2 yrs. In general, patients in the validation cohort had slightly more severe COPD than those in the derivation cohort (table 1).
Associations between health variables and 5-yr mortality were evaluated in a univariate analysis (table 2). All of the variables except sex were statistically significantly related to mortality. The associations were highly significant except for BMI, which was just within the limit of significance.
Based on the results of the univariate analysis, we included the following variables in the COPD-CART: dyspnoea, FEV1 % pred, level of PA, overall health status, age and hospital admissions for COPD exacerbations in the preceding 2 yrs. All the variables were also selected as statistically significant by CART, except for overall health status. The CART created with data from the derivation cohort is shown in figure 1. Patients aged ≥75 yrs had a high and increasing risk of mortality depending on their level of dyspnoea, FEV1 % pred and hospital admissions for COPD. The exception in this age group was patients with a moderate-to-high level of PA, who had a moderate risk of death (0.12). Patients aged <75 yrs followed various paths. Those with severe flow limitation (FEV1 <35% pred) had a significant risk of death, except patients with only moderate-to-mild dyspnoea (degree 1 or 2) and no hospitalisations in the previous 2 yrs for COPD exacerbations. Patients with FEV1 of 35–64% pred and two or more prior admissions for COPD exacerbations had a high risk of mortality depending on their level of PA (0.27 and 0.65 for high/moderate and low/very low PA, respectively), but when FEV1 was ≥65% pred, the risk of death was lower (0.14). For those with no or one previous hospitalisation for COPD exacerbations, the risk depended on age: patients <55 yrs had the lowest mortality risk (0.04), while those between 55 and 74 yrs of age with FEV1 >64% pred also had low mortality risk (0.07). Among patients with FEV1 between 35 and 64% pred, mortality risk depended on the level of PA, ranging from a mortality risk of 0.1 among those with a high level of PA to 0.31 among those with a low level of PA.
Decision tree. FEV1: forced expiratory volume in 1 s; % pred: % predicted; PH: prior hospitalisations; PA: physical activity.
Figure 2 presents the AUC for the BODE, ADO and HADO scores as well as for the COPD-CART decision tree. The CART had the highest AUC (0.74), although the four values were not statistically different.
Receiver operating characteristic curves for the classification and regression tree (CART), and the HADO (health, activity, dyspnoea and obstruction), BODE (body mass index, obstruction, dyspnoea and exercise) and ADO (age, dyspnoea and obstruction) scores. AUC: area under the curve.
We compared the sensitivity, specificity and AUC of the three previously published COPD severity indices with the COPD-CART using three different cut-off points for each (table 3). The COPD-CART provided acceptable sensitivity or specificity, depending on the cut-off point chosen, and was similar to or better than the other indices. AUCs for the different cut-off points were 0.66–0.67 for the COPD-CART, which was slightly superior to the other rules in most cases.
To validate the COPD-CART, we compared its quartiles of severity with the three other indices and the total SGRQ score (table 4). Statistically significant differences were observed between the COPD-CART quartiles in the BODE and ADO scores, while differences in the HADO scores were observed between the two most severe categories compared with all the others. With the SGRQ scores, differences were found between the two more severe categories and the two more moderate.
DISCUSSION
A CART built from five easily obtained parameters in clinical practice provides an accurate decision model, expressed as a tree, for 5-yr mortality among patients with stable COPD. Patient age (≥75 or <75 yrs) provided the first branch. After age, in order of importance in the next branch, came FEV1 % pred and dyspnoea status, followed by the number of admissions for COPD exacerbations in the previous 2 yrs and level of PA. Such a decision tree may help physicians estimate prognoses for their COPD patients and potentially influence clinical decision making.
In the COPD-CART, age established a distinct cut-off point (age <75 versus ≥75 yrs) to give significantly different probabilities of mortality. This is consistent with previous studies that have established age as a predictor of mortality [20–23]. In the ADO index, for example, age was included with FEV1 % pred and dyspnoea as an important predictive variable [3]. In our cohort of 611 COPD patients, age was the second most important predictor of all-cause mortality and the third most important predictor for respiratory mortality [22].
Guidelines for the management of COPD [24] highlight the importance of FEV1 in the diagnosis, treatment and follow-up of patients with COPD. Several studies have established that the level of airway obstruction is related to disease prognosis [20–22]. It is no surprise, then, that FEV1 was an important predictor of mortality and provided a branch of the COPD-CART.
In a study by Nishimura et al. [23], dyspnoea proved to be a more effective predictor of mortality than FEV1 % pred among COPD patients. This finding was not replicated by our group; we found that dyspnoea was a more powerful predictor than FEV1 % pred for all-cause mortality but not for respiratory mortality, while FEV1 % pred was a stronger predictor than dyspnoea for respiratory mortality [22]. Thus, it appears that both FEV1 % pred and dyspnoea are good predictors, but for different types of mortality. The importance of these measurements is underscored by the fact that FEV1 and dyspnoea are the only two factors included in all four multidimensional prognostic scores [1–4].
In relation to the choice of previous hospital admission, Soler-Cataluña et al. [25] demonstrated that the risk of mortality among patients with COPD increased with the frequency of severe exacerbations requiring hospital admission. When they replaced 6MWT in the BODE index with severe exacerbations, the new BODEx index had similar prognostic capacity for overall mortality (C statistic 0.74, 95% CI 0.65–0.83) compared with the original BODE index (0.75, 95% CI 0.66–0.84) [4].
Physical activity has previously been shown to influence mortality among patients with COPD. In a population-based cohort, Garcia-Aymerich et al. [12] showed that even slight PA (equivalent to walking or bicycling for 2 h·week−1) reduced the risk of respiratory mortality among patients with COPD. Our results also highlighted the importance in some cases of the level of PA in terms of mortality risk. Physical activity is also a component of the HADO score [2].
Our decision tree was built by recursive partitioning using CART. CART has some advantages over the other developed scores, some statistical and others in the way it can be used. Among the main statistical advantages are that CART is a nonparametric technique. Therefore, this method does not require specification of the nature of the relationship between predictors and outcome. Tree-based models allow complex interactions between the independent variables, which must be specified a priori in standard linear models. Interpretations of complex interactions are clear and often more easily understood than in other model constructions. The CART algorithm will itself identify the most significant variables and eliminate nonsignificant ones. CART can easily handle outliers and is less dependent on missing values in the independent variables. The simplicity of using the CART is also worth highlighting: the user merely has to follow the branches relevant to the patient under study. It provides a global vision not only of what can be expected for a particular patient but also what can be expected if the patient’s status changes. Among the disadvantages of the CART are that the clusters created may not necessarily agree with established diagnostic combinations or have any rational clinical meaning. However, scales developed using logistic regression models (as for the BODE, HADO and ADO indices) may use continuous variables; all relevant variables are considered in the scores and they will have more power, as they always use the whole database.
Other authors have used this kind of recursive partitioning-based model to develop clinical practice guidelines for high-dose chemotherapy and bone marrow transplant in chronic myelogenous leukaemia, compared to ANOVA [26] and logistic regression [27]. McConnochie et al. [28] compared classification trees to logistic regression for developing prediction rules and concluded that the use of both techniques is complementary. The three previously developed scores for COPD severity to which we compared the COPD-CART were built using logistic regression.
The ability of the COPD-CART to predict mortality, measured by AUC, was quite similar to that of the BODE index, HADO score, ADO index and total SGRQ score, supporting the validity of the COPD-CART.
Sensitivities for the COPD-CART for mortality risk ranged from 47.3% to 87.5%, depending on the cut-off points chosen, and specificities ranged from 83.9% to 47.5%. This compares favourably with the sensitivities and specificities of the BODE, ADO and HADO indices, and as it had quite similar results and, even better, prognostic ability, our tool seems to be at least similar to most of the previous tools.
Besides the previously cited strengths, our study, with two large samples of patients with stable COPD, which allowed us to validate our proposal, outlines the importance of some well-known variables (age, FEV1 and dyspnoea) but also of others (PA and previous hospitalisations) not included in previous rules. These latter variables make a difference, in terms of mortality, in some cases.
Several limitations of our study should be noted. The decision tree was derived from and validated in patients from a single hospital, which could limit the generalisability of these results. For example, both cohorts were predominately male, reflecting the history of smoking patterns in northern Spain, where smoking was traditionally restricted to males.
Another question is whether the CART results might be applied to other cohorts in lower-risk settings, such as primary care. The fact that in our two cohorts, ≥50% of patients had a FEV1 >50% pred means that an important proportion of the patient sample was very similar to patients managed in primary care.
PA is a variable that is sometimes difficult to measure and at risk of measurement error. The effect of measurement error for PA was checked, with a <5% risk of significant misclassification and a statistically nonsignificant change of mean predicted risk of death (p=0.27).
The AUC of the COPD-CART is merely moderate, similar to those of other COPD indices. Thus, other variables are needed to improve the predictive capacity of the decision tree. One possibility is BMI <20 kg·m−2, which has been shown to constitute an independent predictor of COPD-related mortality [29]. In our study, some other variables that were statistically nonsignificant in the univariate analysis, but considered in other COPD prediction rules, were considered: BMI and the general health of the patient. None of them entered any branch of the developed trees. The main limitation of decision tree models is that including higher-order interactions without considering the main effects could lead to spurious relationships between predictors and overestimate the effect of some predictors. This is sometimes referred to as estimation bias [30]. However, our study validates the predictive capacity of the COPD-CART index in a different sample, which minimises that bias.
The utility of a clinical decision rule that is easily applied in daily practice cannot be understated. The main advantage of a decision tree is its simple representation. Some investigators believe that a decision tree represents how clinicians think, starting with the most important characteristic, followed by another characteristic depending on the answer of the first, and so on [24]. The main value of this predictive tree is that it can easily establish a prognosis without clinicians having to memorise the scores of different variables. The COPD-CART decision tree, which employs measures generally gathered by physicians in the evaluation of COPD patients, provides a quick assessment of the severity of the disease, as measured by the risk of premature death, and by patient-perceived HRQoL, as measured by the SGRQ. The ability of this instrument to predict 5-yr mortality is as good as, or better than, that of other existing indices, but with greater ease of use in routine clinical practice. However, the COPD-CART must be tested in other settings and other populations, and other measures should be evaluated for inclusion to improve its predictive capacity, before it could become a widely used instrument.
Footnotes
Support Statement
This study was partially supported by grants from the Fondo de Investigación Sanitaria (PI97/0326 and PI020510), Departamento de Sanidad del Gobierno Vasco 200111002 and the Research Committee of the Hospital Galdakao-Usansolo.
Statement of Interest
None declared.
- Received December 8, 2010.
- Accepted April 24, 2011.
- ©ERS 2011
REFERENCES
