The placebo comparison group

One of the major criticisms with the TORCH trial is the choice of ‘placebo’ as the comparison group, which was incompatible with the study objective of ‘usual care’ as the comparison group. Indeed, over 59% of the patients in the placebo comparison group were on maintenance therapy (LABA, ICS or a long-acting anticholinergic) when they entered the study, but these treatments were discontinued at randomisation and replaced by a placebo for the 3-year follow-up. It was in fact somewhat surprising that the TORCH study results did not actually show inordinate benefits in view of the expected advantage from discontinuing the maintenance therapy of patients who visibly must have needed it and replaced it with placebo for 3 years.8

The SUMMIT trial is clear about the study question, aiming to ‘test the hypothesis that treatment with combined inhaled corticosteroids and long-acting β-agonists will reduce mortality when compared with placebo’.1 However, here again, the patients in the placebo comparison group in SUMMIT who were on maintenance therapy when they entered the study will also have them discontinued at randomisation and replaced by a placebo during follow-up. One is justified to question whether using such a comparison group is actually appropriate and whether it does not provide an inherent advantage to the treated group by giving placebo for years in replacement for pretrial maintenance therapy. These effects are presented below.

Effect of treatment discontinuation at randomisation

When study patients entering the trial must terminate their maintenance treatment at the time of randomisation, such discontinuation becomes an inherent component of the intervention when the discontinued drug is of the same class as the study drug.

This phenomenon was illustrated using data from the OPTIMAL trial that assessed the effect of adding ICS to long-acting bronchodilators in COPD patients on the incidence of exacerbation over 1 year follow-up.9 A reanalysis showed that the overall HR of exacerbation associated with ICS relative to bronchodilators was estimated as 0.79 when all subjects were combined. However, among the patients who had previously used ICS and had to discontinue, the HR was 0.71, while for those who had not used ICS and thus did not have to discontinue, it was 1.11.5 In essence, this phenomenon warns that an overall effect estimate in such a trial is comprised of two different components: the effect of discontinuing the drug among the previously treated who had to discontinue and the effect of initiating the drug among the treatment-naive users. This effect was present in the TORCH trial where over 60% of the placebo patients had to suddenly discontinue their maintenance therapy and use placebo for 3 years.

In the SUMMIT trial, three treatments are discontinued and studied, namely, ICS, LABA and the ICS–LABA combination. Although recruited patients are those that ‘the practitioner believes can be adequately managed without these medications’,1 it remains that these medications are used by over 70% of COPD patients, including the moderate ones targeted in this trial, as suggested by market data and several trials.7 ,10

In the context of the SUMMIT trial, we illustrate this phenomenon focusing specifically on the primary comparison between the ICS–LABA combination and placebo. Figure 1 depicts three different sources of discontinuation. First is the subgroup of patients who will be on an ICS–LABA combination before randomisation and who will be assigned to either the study ICS–LABA combination or placebo. This contrast will clearly provide a direct effect of the discontinuation of the combination drug, not its initiation. Figure 1 also shows that the patients who stop their ICS at randomisation and are assigned to the study ICS–LABA combination or placebo during follow-up will contribute a double effect, namely, the effect of ICS discontinuation and the effect of initiating LABA. Similarly, the subjects who were on LABA will contribute to the effect of LABA discontinuation and the effect of initiating ICS.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Depiction of the six subgroups of patients that result in various effects of drug discontinuation. Recruitment into the trial on an ICS–LABA combination, an ICS or a LABA just before randomisation and assignment to ICS–LABA or placebo will lead to effects of discontinuation. ICS, inhaled corticosteroid; LABA, long-acting β-agonist.

Figure 2 depicts the valid comparisons related to the desired effects of initiation of the ICS–LABA combination and its components. It shows that it is only among the patients who are treatment-naive of any ICS or LABA that the ICS–LABA combination can be directly compared with placebo to provide a valid estimate of the effect of the treatment.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Depiction of the pure effect of drug initiation, based on the four subgroups of patients who were treatment-naive of ICS, LABA or the ICS–LABA combination, before randomisation. ICS, inhaled corticosteroid; LABA, long-acting β-agonist.

Therefore, these depictions show that the computation of a single estimate of the effect based on comparing ICS–LABA with placebo across all four scenarios will encompass a mix of unspecific estimates of the effects of various drug discontinuations and initiations, clearly dissonant with the sought after effect of ICS–LABA.

Censoring at treatment discontinuation in follow-up

One of the important methodological contributions of the TORCH trial has been that it was among the first trials in COPD to provide a valid intent-to-treat analysis by collecting data on death, the primary outcome, after study (and treatment) discontinuation during follow-up, untainted by the reasons for discontinuing. The censoring of follow-up for such patients is unlikely to be random and thus, faced with such informative censoring, an on-treatment analysis can be biased. Indeed, a reanalysis of the TORCH data showed that withdrawals occurred earlier and were more likely in the placebo group (44%) compared with the ICS–LABA combination group (34%), as were patients previously on ICS and with a history of multiple exacerbations.11

Thus, in the SUMMIT trial, mortality will not be affected by such censoring as all deaths occurring during follow-up will be identified, irrespective of whether the patients discontinued treatment. However, such is not the case for secondary outcomes such as lung function decline and exacerbations, which are not measured after that point. Such data truncation can lead to bias if the reasons for discontinuation are associated with the outcome and differ between treatments, which can affect the exacerbation rates.12 It will also introduce bias in computing the rate of decline in lung function over time, as it is affected by the phenomenon of ‘regression to the mean’ when follow-up is discontinued.6

Thus, the censoring of follow-up at treatment discontinuation in SUMMIT will not affect mortality but could exaggerate the effects for the secondary outcomes such as lung function decline and exacerbations.

Factorial design analysis

Like TORCH, the SUMMIT trial is structured as a 2×2 factorial design of an ICS (yes/no) and a LABA (yes/no). However, despite generating data on four exposure groups, the study intends to compare only the ICS–LABA group with placebo, thus squandering much needed power. Moreover, a full 2×2 factorial analysis would quantify the independent contribution of each drug component on the outcome, which would provide valuable scientific information about these drugs. For the TORCH trial, for example, such an analysis showed that the LABA component of the combination was associated with a significant 17% reduction in mortality (rate ratio 0.83; p=0.0043), while the ICS component provided no reduction (rate ratio 1.00; p=0.99).4 ,5