Abstract
The recent Towards a Revolution in COPD Health (TORCH) randomised trial replicated the findings of previous trials in chronic obstructive pulmonary disease (COPD) on the apparent effectiveness of inhaled corticosteroids (ICS) in reducing exacerbation rates, but not so for mortality.
In the present article, the authors review methodological issues in the TORCH and previous trials, such as patients already receiving ICS before randomisation and the absence of follow-up after study drug discontinuation, using data from two trials.
First, among previous ICS users in the Canadian Optimal Therapy of COPD Trial, the hazard ratio of the first exacerbation with ICS relative to bronchodilators was 0.71 (95% confidence interval (CI) 0.53–0.96), while among those not using ICS prior to randomisation, it was 1.11 (95% CI 0.69–1.79). Secondly, the rate ratio of exacerbations with ICS was 0.78 (95% CI 0.61–0.99) prior to drug discontinuation during follow-up and 1.23 (95% CI 0.78–1.95) thereafter. Finally, a 2×2 factorial analysis of the TORCH data found a rate ratio of mortality for the salmeterol component to be 0.83 (95% CI 0.74–0.95), while for the fluticasone component it was 1.00 (95% CI 0.89–1.13).
In conclusion, after proper consideration of the various methodological shortcomings in the design and analysis of randomised trials, the effectiveness of inhaled corticosteroids in treating chronic obstructive pulmonary disease remains doubtful, while the benefit observed with combination therapy may be due exclusively to the beneficial effects of the long-acting bronchodilator alone.
- Biases
- chronic obstructive pulmonary disease
- data analysis
- drug effectiveness
- inhaled corticosteroids
- study design methods
The recently published Towards a Revolution in COPD Health (TORCH) trial 1 has provided the field of chronic obstructive pulmonary disease (COPD) therapeutics with a landmark study that has undoubtedly advanced present knowledge about the potential benefits of inhaled corticosteroids (ICSs) and long-acting bronchodilators in the treatment of this disease. The TORCH study also illustrates several methodological issues pertinent to the proper conduct of randomised trials in COPD. While this major study answers important questions as to the role of ICSs in potentially preventing mortality in COPD, major questions related to the role of ICSs in preventing exacerbations of COPD remain unanswered.
Major randomised controlled trials that have evaluated the effectiveness of ICSs in the treatment of COPD over the last 10 yrs have reported results that are often contradictory and that seem paradoxical 2, 3. Generally, these studies have found that ICSs had either no effect, or only minor beneficial effects, on decline in lung function as measured by forced expiratory volume in one second, the primary outcome measure. Conversely, most of these trials have observed significant reductions in COPD exacerbation rates associated with ICS use; a meta-analysis of these trials suggested a clinically significant reduction in exacerbations of ∼30% 4. Much of this apparent benefit reported in the meta-analysis was the result of an incorrect statistical approach to the analysis of these exacerbations 5. Indeed, many of the previously performed trials used statistically flawed, unweighted analyses of exacerbations, and therefore failed to incorporate correction of overdispersion into their analyses. Two subsequent trials that used the proper statistical approach demonstrated a modest reduction, albeit nonsignificant, in the rate of COPD exacerbations 6, 7. In addition to a reduction in exacerbations, a pooled analysis of data from seven randomised trials involving >5,000 patients reported a significant 27% reduction in all-cause mortality associated with ICS use 8.
Thus, paradoxically, ICSs would do little to improve lung function, itself strongly related to mortality 9 and probably to frequency of exacerbations 10, but could prevent significant numbers of COPD exacerbations and deaths from all causes. While various physiological explanations, such as effects on systemic inflammation 11, have been put forward to explain a benefit on exacerbations and mortality in the face of no benefit on lung function, possible methodological explanations for these discrepant results have received little attention.
In the present article, the authors review several key aspects of the design and statistical analysis of COPD intervention trials, including the TORCH trial, and illustrate these issues using data from the randomised Canadian Optimal Therapy of COPD Trial (Optimal trial).
DATA FOR ILLUSTRATION
The Canadian Optimal trial was a three-arm randomised trial of 449 patients with moderate or severe COPD, all given a base therapy of tiotropium and allocated randomly to receive, in addition, placebo (n = 156), salmeterol (n = 148) or fluticasone/salmeterol (n = 145) 12. The primary outcome was a COPD exacerbation requiring treatment with oral or intravenous steroids and/or antibiotics within the year after randomisation. Outcomes were recorded over the entire 1-yr follow-up period, regardless of whether patients had discontinued the study medications. A patient was considered to have experienced a new COPD exacerbation if they had not taken oral steroids and antibiotics for at least 14 days following their previous exacerbation.
Since the issue currently being discussed involves the potential benefit of ICSs, the present authors combined the 304 patients randomised to tiotropium with or without salmeterol (the “bronchodilator group”), and compared them with those randomised to the two bronchodilators plus fluticasone (the “ICS group”). This regrouping is also justified by the similar results obtained whether tiotropium was combined with placebo or with salmeterol 12. Information on the dates of exacerbations, the use of ICSs prior to randomisation and the date of discontinuation of study medications during follow-up was used to illustrate the methodological issues relevant to COPD trials.
METHODOLOGICAL ISSUES
Most randomised trials to assess the effectiveness of ICSs in COPD conducted to date have had several unique features and limitations, as follows: 1) many patients were already receiving ICSs before randomisation; 2) analyses did not look specifically beyond the first exacerbation; 3) the trials stopped following the patients who discontinued the study drugs before the end of the observation period. Furthermore, several trials, including the TORCH study did not fully exploit the data available from their unique 2×2 factorial design. These issues are described herein, and their impact on the results and interpretation of trials of therapy in COPD is demonstrated.
Previous therapy
The various trials of ICSs in COPD have been conducted, at least in part, among patients already using these medications before randomisation. This is an exceptional situation for randomised trials. The proportion of patients who were previous users of these drugs in the placebo group varied from 26% in the study by Szafranski et al. 6 to 51% in the TORCH trial, and reached as much as 77% in the Optimal trial. In all these trials, the patients were, of course, required to stop their ICSs at the time of randomisation. This unusual situation creates a unique challenge in interpretation. Indeed, among the patients who did not previously use ICSs, the randomisation will lead to the desired comparison of patients initiating treatment with ICS with similar patients who do not. Conversely, among the patients who were previous regular users of ICSs, the randomisation will in fact provide a comparison of patients who continue to use ICS (patients allocated to ICS treatment) with patients who discontinue ICS (patients allocated to placebo); figure 1⇓ depicts this phenomenon. Thus, among previous users of ICSs, the randomised trial will in fact evaluate the effect of stopping the use of ICS, albeit perhaps a different ICS and at a different dose, rather than the effect of introducing ICS as treatment.
Depiction of a typical randomised trial of chronic obstructive pulmonary disease (COPD) patients already using the study drug, namely inhaled corticosteroids (ICS), prior to randomisation, who are then randomised to either ICS or placebo.
A potentially analogous situation occurs in trials of tiotropium, when patients are obliged to stop their short-acting anticholinergic bronchodilator, ipratropium bromide, and, in some studies, also their long-acting β-agonist, at the time of randomisation so that those allocated to placebo are in fact being withdrawn from therapy with anticholinergic bronchodilators and possibly other bronchodilators 13–17.
Using the illustrative data from the Optimal trial, the patients were stratified according to their previous use of ICSs and studied from the time of randomisation to the first COPD exacerbation. Figure 2⇓ shows the time to the first exacerbation among the 335 patients who were previous users of ICSs for the randomised bronchodilator and ICS groups. It also displays the corresponding curves among the 114 patients who were naïve to ICSs at the time of randomisation; it is evident that the effects are different. Table 1⇓ provides the corresponding hazard ratios (HRs) estimated from the proportional hazards model. The HR of the time to first exacerbation for the ICS group relative to the bronchodilator group was 0.79 (95% confidence interval (CI) 0.62–1.02; p = 0.07) for the analysis including all subjects. However, among the patients who had previously used ICS, the HR was 0.71 (95% CI 0.53–0.96; p = 0.027), while for those who had not used ICS, it was 1.11 (95% CI 0.69–1.79; p = 0.68). This latter estimate, however, is based on only 30% of the subjects and thus has lower power. While its best estimated value is 1.11, it is also compatible with a value of 0.69, the lower value of the 95% CI.
Kaplan–Meier curves for the time to the first exacerbation comparing the patients randomised to either inhaled corticosteroids (ICS) or bronchodilators among a) the 335 patients who were previous users of ICS at the time of randomisation, and b) the 114 patients who were naïve to ICS at the time of randomisation (data from the Canadian Optimal Therapy of COPD Trial). –––––: inhaled ICS users; -----: bronchodilator users.
Analyses of the effect of inhaled corticosteroids(ICS) compared with bronchodilators on the time to first and second exacerbation using the Canadian Optimal Therapy of COPD Trial study data
The relationship between the effect of inhaled corticosteroids (ICSs) and prior ICS use was then investigated in all randomised controlled trials of ICSs conducted in patients with COPD that provided both the rate ratio of exacerbations and information on the prior use of ICS (table 2⇓). Figure 3⇓ shows the plot of the rate ratio of exacerbation across the studies against the rate of ICS use prior to randomisation. The corresponding generalised linear model analysis of the logarithm of the rate ratio on the rate of ICS use prior to randomisation, weighted by the inverse of its variance, resulted in a reduction of 8% in the relative rate ratio (0.92; 95% CI 0.89–0.96; p<0.001) for every 25% increase in the rate of prior ICS use.
Graph showing rate ratio of exacerbations against the proportion of users of inhaled corticosteroids (ICS) prior to randomisation in all six randomised controlled trials that compare an ICS (alone) with placebo, and that include these necessary data. –––––: fitted generalised linear model for these rate ratios, weighted by the inverse of the variance.
Randomised controlled trials comparing an inhaled corticosteroid(ICS; alone) to placebo that include data on the rate ratio of exacerbation and previous use of ICSs prior to randomisation
Beyond the first exacerbation
While all randomised trials have studied the rate of exacerbation during the time that subjects were being followed, and some trials have analysed the time to the first COPD exacerbation, none has specifically examined the pattern of COPD exacerbations subsequent to the first one. In particular, it might be asked how much of the effect of the treatment on the rate of exacerbation measured over the entire follow-up is due to its effect on the first COPD exacerbation. To investigate this question, the current authors considered the subgroup of patients with at least one COPD exacerbation in the Optimal trial. This included 87 patients in the ICS group and 186 in the bronchodilator group. The time from the end of the patient’s first COPD exacerbation to the onset of the second one was analysed, censoring the patients who did not have a second exacerbation at the last follow-up. Figure 4⇓ shows the time from the first to the second COPD exacerbation according to randomisation to the bronchodilator or ICSs groups. The corresponding HR of the second exacerbation for the ICS group relative to the bronchodilator group is 1.00 (95% CI 0.73–1.40; p = 0.96), as shown in table 1⇑. Since this analysis is by the intention-to-treat approach and was performed in the subset of the population with an exacerbation during follow-up, the interpretation is complex. However, <10% of the subjects in this analysis had discontinued their allocated treatment during follow-up but prior to their first exacerbation (the starting point for the analysis). Nevertheless, the analysis suggests that the effect of ICS on the rate of exacerbation over the study period may be dominated by its effect on first exacerbation. This contrasts with trials of ICS in combination with bronchodilators in asthma where the relative benefits increase with each exacerbation 21.
Kaplan–Meier curves for the time from the first to the second exacerbation comparing the patients randomised to either inhaled corticosteroids (ICS) or bronchodilators, among the 273 patients who had an exacerbation (data from the Canadian Optimal Therapy of COPD Trial). ––––––: ICS; -----: bronchodilators.
Extent of follow-up
Until the TORCH and Optimal trial studies, randomised trials of COPD therapy stopped patient follow-up at the time they discontinued the study drug. Thus, any outcome information after the patients were withdrawn, but before the planned end of study follow-up, was not collected. As such, the fundamental intent-to-treat analysis for such trials was not possible, since the data were truncated at the time of drug discontinuation. While the extent of this problem may be trivial in other diseases, COPD trials characteristically demonstrate very high discontinuation rates, and such dropouts often occur very early in the trial. Not following patients up until the end of a trial and conducting the data analysis only until discontinuation of study drugs will produce biased results if the reason for discontinuation is associated with the outcome and differs between treatments. The importance of this principle was clearly demonstrated by the Coronary Drug Project Research Group, which showed that patients who were not compliant with placebo, just like those who were not compliant with clofibrate, had significantly higher mortality than “properly treated” patients who took their placebo regularly 22. Thus, considering only the “compliant” follow-up time prior to discontinuation can lead to bias.
Table 2⇑ also shows that the withdrawal rates were variably different between treatment groups in the randomised controlled trials comparing an ICS with placebo in patients with COPD. The corresponding generalised linear model analysis, adjusted for prior ICS use, resulted in an increase of 3% in the rate ratio of exacerbation (1.03; 95% CI 1.01–1.06; p = 0.011) for every 10% increase in the ratio of withdrawal rates between the ICS and placebo groups. This suggests that the studies with the greatest disparity in withdrawal rates found the largest reduction in the rate of exacerbation with ICS compared with placebo.
To avoid such bias, the TORCH trial correctly followed all patients up to the end of the 3-yr trial period to ascertain mortality, its primary outcome, even after discontinuation of the study medications. This was not done, however, for the secondary outcomes including exacerbations, lung function and health status. This is of concern, since 44% of patients in the placebo group discontinued treatment, mostly during the first few months, as compared with 34% in the combination therapy group. Thus, for the TORCH trial, the intent-to-treat results for mortality are certainly valid; however, the results describing the secondary outcomes and, in particular, exacerbations, may be biased.
To investigate the potential effect of this bias, the current authors used data from the Optimal trial that aimed to follow up patients for the entire 1-yr study period. However, despite the attempt to follow up each patient, 65 of the 449 patients could not be followed up beyond drug discontinuation because of withdrawal of patient consent, while 110 of the patients who prematurely discontinued study drugs were followed up for the duration of the 12-month trial period. This dataset provides sufficient numbers of fully followed patients to evaluate this potential bias. Table 2⇑ shows the rates of COPD exacerbation when ascertained over the entire follow-up period and when restricted to the period until discontinuation of study drugs. The rate ratio of exacerbation with ICSs using the entire follow-up time is 0.83 (95% CI 0.66–1.04), while it is 0.78 (95% CI 0.61–0.99) prior to drug discontinuation and 1.23 (95% CI 0.78–1.95) thereafter.
With respect to mortality, a major discrepancy exists between a pooled analysis of seven major randomised trials, involving 5,086 patients, that found a significant 27% reduction in mortality with ICSs (HR 0.73; 95% CI 0.55–0.96; p = 0.039), and the TORCH trial, that found no reduction whatsoever with fluticasone 1, 8. Trials that comprised the pooled analysis truncated patient follow-up when they discontinued the study drug. In fact, the pooled analysis found no difference in mortality during the first 9 months of follow-up, the time period where dropouts were still rare and thus where most randomised patients were included in the mortality analysis. The apparent benefit of ICS was, in fact, only visible after 9 months. While the mortality rate in the ICS group is stable over time, the benefit is the result of two clusters of deaths in the placebo group occurring at precisely 9 and 24 months of follow-up 23. Conversely, the TORCH trial that followed all patients for 3 yrs found a HR of mortality for fluticasone relative to placebo of 1.06 (95% CI 0.89–1.27; p = 0.53) 1. This disagreement between the two large mortality studies is thus probably a direct result of the follow-up process.
Factorial analysis of TORCH data
While the TORCH study aimed to compare the combination of fluticasone and salmeterol with placebo, the study also included a fluticasone-only and a salmeterol-only arm 1. The TORCH study 1 was thus structured as a 2×2 factorial design of fluticasone (yes/no) and salmeterol (yes/no). This same factorial design was also used in other trials involving an ICS and a long-acting β-agonist 6, 7, 20. However, all these trials, including the TORCH trial, were not analysed as factorial trials, thus squandering much-needed power and denying the reader important information about the independent contribution of each component of the combination.
As mortality was the only outcome ascertained in a complete manner for an intent-to-treat analysis, the present authors used the mortality data presented in the TORCH article 1 to perform the analysis corresponding to a 2×2 factorial design, as has been previously reported by La Vecchia and Fabbri 24. A generalised linear model was used with a binomial distribution to estimate the 3-yr mortality rate ratio associated with fluticasone and salmeterol. Furthermore, an interaction term was used to assess whether there is synergy between the two drugs. The present authors found that the interaction term was not significant (p = 0.32), suggesting that the combination of fluticasone and salmeterol is not particularly more effective than the two components added independently. Table 3⇓ presents the rates and the independent effects of fluticasone and salmeterol on mortality, adjusted for each other. While the salmeterol component is associated with a significant 17% reduction in mortality (rate ratio 0.83; 95% CI 0.74–0.95; p = 0.0043), the fluticasone component provides no reduction whatsoever (rate ratio 1.00; 95% CI 0.89–1.13; p = 0.9918).
Analyses of the effect of inhaled corticosteroid(ICS) use compared with bronchodilators on the rate of exacerbation over different follow-up periods using the Canadian Optimal Therapy of COPD Trial data
Factorial analysis of Towards a Revolution in COPD Health(TORCH) data of the independent effects of fluticasone and salmeterol on the 3-yr incidence of all-cause mortality
Unfortunately, factorial analysis was invalid for the other outcomes, such as exacerbations and health status, since these were not measured throughout the entire follow-up. They are thus also subject to the bias resulting from truncating patient follow-up at the discontinuation of assigned drug therapy.
DISCUSSION
Randomised controlled trials conducted to assess the effectiveness of ICSs over the past 10 yrs have reported results that appear paradoxical. While these drugs were found to have no or only minor effects on lung function in earlier studies, they were associated with significant reductions in COPD exacerbation rates, as well as significant reductions in mortality in pooled studies. The current authors have shown that unique methodological aspects of the study design and statistical analysis of such studies at least partly explains this apparent paradox. After accounting for several characteristics of the analytical techniques used in the studies, as illustrated using the Optimal trial data, an apparent benefit of ICSs was found to disappear.
The single most important methodological concern is the rather unique situation in COPD of randomising some patients who were already being treated with ICSs before randomisation, to treatment with ICSs after randomisation. In actuality, this unusual situation creates two types of comparison. Among the patients who did not previously use ICSs (ICS naïve), the randomisation leads to a comparison of patients initiating treatment with ICS with similar patients who do not. Among previous users of ICSs, however, the randomisation will lead to a comparison of patients who continue ICS use with patients who stop their use of ICS. Thus, combining previous users with nonusers in the trials leads to a mixture of the true effect of ICSs (in ICS-naïve patients) with the effect of suddenly interrupting ICSs (previous ICS users). The current authors’ illustration using data from the Optimal trial showed that among the ICS-naïve patients at randomisation, no benefit whatsoever was observed with ICSs on the rate of exacerbation or the time to the first exacerbation. Thus, the apparent effect seen among all patients combined was exclusively due to the effects in the subgroup that had previously used ICS. In actuality, these trials therefore measure the effects of stopping ICS rather than introducing ICS. There are studies that have directly examined the effect of stopping ICS and have found either a decrease in lung function or a similar increase in exacerbations 25, 26. Such a deleterious effect of stopping ICS does not necessarily translate to a beneficial effect of initiating such therapy. This design aspect is not unique to ICS, as it was also observed in the trials of tiotropium.
Another crucial methodological issue is the design of trials in which follow-up is ended at the time patients discontinue using the study drug. As a result, outcome data after patient withdrawal are missing so that an authentic intent-to-treat analysis becomes impossible. The analysis based on data censored at drug discontinuation, equivalent to restricting the analysis to the compliant patients, introduces intricate biases that are difficult to disentangle. The TORCH trial did correctly follow all patients after discontinuation of study medications to ascertain mortality, but did not do this for exacerbations, lung function and health status outcomes. Thus, with almost one-half of patients discontinuing treatment, the results are valid for mortality, but can be biased for exacerbations and for other outcomes. The Optimal trial, which did measure outcomes to the end of follow-up, found different results for the exacerbation rates when the data were analysed over the entire follow-up compared with follow-up censored at drug discontinuation 12. Future trials will necessarily have to follow patients to the end of follow-up and continue to obtain measures of important outcomes after drug discontinuation. To minimise such bias, studies may envisage a shorter, more manageable follow-up (of 1 or 2 yrs) that will decrease the number of patients actually lost to follow-up.
Other methodological aspects of these studies can also have a bearing on the results. Many studies did not look specifically at the time to the first exacerbation and the subsequent pattern, particularly the time from the first to the second exacerbation. The use of the negative binomial model to fit the exacerbation rate in the TORCH study needs to be evaluated and compared with the straightforward Poisson model, which accounts for the between-patient variability in the exacerbation rate with an overdispersion parameter 5. Important differences between these approaches were found with the Optimal trial data 27.
Finally, the studies designed as 2×2 factorial trials to assess the effects of an ICS and a long-acting β-agonist, such as the TORCH study, did not fully exploit the data by using the corresponding 2×2 factorial data analysis. When this 2×2 analysis was performed for the TORCH study, there was no evidence of a synergistic effect from the combination of salmeterol and fluticasone; the observed benefit was due entirely to the effect of salmeterol, with no effect whatsoever attributable to the ICS component.
The randomised controlled trial is and will remain the fundamental tool with which to evaluate the benefit of chronic obstructive pulmonary disease treatments. Its proper conduct, however, including the most rigorous study design and data analysis, is essential to the production of valid results. The Towards a Revolution in COPD Health trial and the Canadian Optimal Therapy of COPD Trial have provided an important evolution in this direction, with designs that allowed a proper intent-to-treat analysis of the primary outcomes. These analyses showed no clear substantiation of the added benefit of inhaled corticosteroids in chronic obstructive pulmonary disease. The science of chronic obstructive pulmonary disease research is, however, still faced with major challenges involving a complex and serious disease requiring multiple treatments, both pharmacological and nonpharmacological, and with highly variable outcomes. The corresponding methodological complexities will have to be tackled to prevent the dissemination of studies that will bring false hope to the makers, prescribers and users of medications, such as inhaled corticosteroids. With chronic obstructive pulmonary disease being one of the major causes of morbidity and mortality worldwide, the utmost scientific rigor in the design and analysis of studies in this domain is of absolute importance.
Support statement
This research was funded by grants from the Canadian Institutes of Health Research (CIHR) and Fonds de la recherche en santé du Québec. S. Suissa is the recipient of a Distinguished Investigator award from the CIHR. The Canadian Optimal Therapy of COPD Trial database used for illustration was funded by grants from CIHR and the Ontario Thoracic Society.
Statement of interest
Statements of interest for S. Suissa, P. Ernst and S.D. Aaron can be found at www.erj.ersjournals.com/misc/statements.shtml
- Received August 1, 2007.
- Accepted January 10, 2008.
- © ERS Journals Ltd
