Abstract
Randomised trials of COPD medications involving a “run-in” period could be biased: interpret results with caution http://ow.ly/wDR830bPuca
Most randomised trials of newer and established medications in chronic obstructive pulmonary disease (COPD) enrol patients who have already been treated, often with drugs of the same class or even with the very same drugs that are to be studied. Some trials such as UPLIFT evaluated the effect of a drug as an add-on, with patients continuing their existing treatments [1]. Other trials, such as OPTIMAL and the Salford Lung Study, discontinued the regular maintenance treatment and replaced it with a randomly allocated active treatment [2, 3]. Finally, trials such as TORCH and SUMMIT had patients discontinue their regular maintenance treatment and directly receive the randomly allocated treatment, which included placebo [4, 5]. This last approach, however, has been shown to complicate the interpretation of results, particularly under the placebo group after the discontinuation of maintenance therapy [6, 7].
To address this challenge, trials have introduced a “run-in” period to create a gap between the discontinuation of usual therapy and the initiation of the randomly allocated study drugs [8]. These run-in periods have involved the use of a placebo or an active treatment, including using one of the drugs under study. Although the run-in period has been the object of some ethical discussions [9], it is used extensively, particularly to identify generally compliant patients, although they can introduce some bias in the results [10–13].
We describe some trials in COPD that have used a run-in approach and provide a hypothetical trial with simulated data to illustrate a bias potentially introduced by run-in periods, particularly when they involve one of the study drugs.
Examples of run-in from COPD trials
Several trials of COPD treatments have involved run-in periods. These periods have been of varying length and used different treatment exposures, including placebo or active drugs, although some were not specified clearly. We describe these in examples from some of the major COPD trials published to date.
Placebo run-in
The European Respiratory Society Study on Chronic Obstructive Pulmonary Disease (EUROSCOP) trial evaluated the effect of the inhaled corticosteroid (ICS) budesonide in patients with mild COPD who continued smoking [14]. The trial introduced a 3-month period during which compliance was assessed with the use of a placebo inhaler. Of the eligible 1526 COPD patients, 84% were deemed compliant during this 3-month run-in period and were randomly assigned to budesonide or placebo.
The second study involved two identical trials of the efficacy of the phosphodiesterase-4 inhibitor roflumilast in patients with moderate-to-severe COPD already treated with the long-acting inhaled bronchodilators salmeterol or tiotropium [15]. Aside from salmeterol or tiotropium, all other respiratory drugs were discontinued at study entry, after which eligible patients had to undergo a 4-week run-in period when they received placebo tablets. Of the 1221 eligible patients, 935 (77%) were deemed compliant and randomly assigned to roflumilast or placebo and followed for 24 weeks.
Unspecific or variable active run-in
In the Inhaled Steroids in Obstructive Lung Disease in Europe (ISOLDE) trial, eligible patients had to discontinue any oral or inhaled corticosteroids before entering an 8-week run-in period with unspecified treatment [16]. Of the eligible 990 COPD patients, 751 (76%) completed the run-in period and were randomly assigned to fluticasone propionate or placebo.
In the Trial of Inhaled Steroids and Long-acting Beta-2 Agonists (TRISTAN), eligible patients had to discontinue their ICS and long-acting β2-agonist (LABA) therapies and enter a 2-week run-in during which only treatment with anticholinergics, mucolytics and theophylline was allowed [17]. Of the 1974 eligible patients recruited, 1465 (74%) who did not have an exacerbation were randomised to salmeterol and/or fluticasone and compared with placebo.
The Prevention of Exacerbations with Tiotropium in COPD (POET-COPD) trial involved a 2-week run-in period where LABAs were continued, long-acting muscarinic receptor antagonists (LAMAs) were replaced by short-acting ones, and ICS/LABA combinations were replaced by ICS monotherapy [18]. Of the 8293 eligible patients recruited, 7394 (89%) were randomised to the LAMA tiotropium or the LABA salmeterol.
Specific active run-in
In the two identical trials to compare the effectiveness of the ICS/LABA fluticasone furoate and vilanterol combination with the LABA vilanterol alone, eligible patients had to discontinue their current medications and enter a 4-week run-in during which they received an ICS/LABA combination [19]. Of the 4163 eligible patients in both trials combined, the 3255 (78%) who did not experience an exacerbation or adverse events and were compliant were randomised.
In the FLAME trial, eligible patients had to discontinue their medications and enter a 4-week run-in during which they received a LAMA [20]. Of the 4942 eligible patients, 3362 (68%) were randomised to LAMA/LABA or ICS/LABA combinations. Among the ones who did not make it to the randomisation stage, 179 had a COPD exacerbation, three died and 67 had adverse events.
Illustration
To illustrate the bias from an active run-in treatment, we simulated a hypothetical randomised trial of 4000 patients with COPD eligible to evaluate the effectiveness of an anti-inflammatory drug (AI) compared with a bronchodilator (BR) in reducing the incidence of COPD exacerbation over a 1-year follow-up (figure 1). We assumed that the patients are divided equally with half being AI responders and the other half BR responders. For simplicity, we assumed that the incidence rate of an exacerbation is 0% among AI responders who received the anti-inflammatory drug and 0% among BR responders who received the bronchodilator. For the non-responders, we assumed an incidence rate of exacerbation of 20% per month equally for the two drugs, using the exponential distribution for the time to exacerbation. We then simulated this trial with and without a run-in period.
Depiction of illustrative randomised trial comparing the effectiveness of an anti-inflammatory drug (AI) compared with a bronchodilator (BR) with 1-year follow-up, preceded by a 4-week run-in with treatment AI.
Without a run-in, the 4000 eligible patients were all randomised to receive either AI (n=2000) or BR (n=2000), with each arm expected to include 1000 AI responders and 1000 BR responders, so that the trial should result in equal incidences of exacerbation at 1 year for the AI and BR arms. Table 1 illustrates that, by the end of the 1-year follow-up under an incidence rate of 20% per month, the cumulative incidence of a COPD exacerbation is 45.5%, equal for the patients allocated to AI and to BR, resulting in a cumulative incidence ratio of 1.0 comparing AI to BR. Table 1 also shows that, for each of the AI and BR arms, the incidence of 45.5% is composed of a 0% incidence among the responders and a 90.9% incidence among the non-responders, respectively, of each drug class.
Simulated data from a hypothetical randomised trial of 4000 chronic obstructive pulmonary disease (COPD) patients
With a run-in, all 4000 eligible patients received AI for 1 month, i.e. one of the study drugs. Only patients who were exacerbation-free during the run-in were subsequently randomised, resulting in 3638 patients randomised (table 1). By the end of the 1-year follow-up under an incidence rate of 20% per month, the cumulative incidence of a COPD exacerbation is 40.9% in the AI arm compared with 50.0% in the BR arm, resulting in a cumulative incidence ratio of 0.82 comparing AI to BR. The difference in the cumulative incidence between the two arms is a result of the number of BR responders being reduced from 2000 eligible patients to 1638 who remained exacerbation-free during the AI run-in period.
Discussion
Numerous randomised trials of COPD medications have introduced a “run-in” period to bridge the gap between the abrupt discontinuation of usual treatment and the start of the study treatment. We showed that this run-in period can introduce bias by selecting out specific patients before randomisation.
The bias we illustrated arises when the drug used during the run-in is one of the drugs under study, or of the same class. In such a case, the patients who are non-responders to the run-in drug will be more likely to experience a worsening of their disease, such as having more exacerbations, and will thus be excluded from the randomisation stage. Consequently, the remaining patients available for randomisation will necessarily include more responders to the run-in drug so that, after randomisation, the study drug used in the run-in will fare better than the comparator drug as non-responders will have been differentially weeded out in the run-in. This phenomenon is particularly plausible in COPD where there is growing evidence of subgroups of patients who respond to specific drugs, such as for ICS [21, 22]. In fact, early treatment response markers in the first 2 months of pooled data from three trials comparing budesonide/formoterol with formoterol alone have been shown to predict the risk of a future COPD exacerbation among patients continuing to use the treatment [23]. Such analyses provide further evidence for the bias arising from the selective inclusion of patients in a randomised trial after a run-in with one of the study drug.
This run-in bias may have affected some of the recent trials of COPD medications. For example, in the 1-year FLAME trial comparing LABA/LAMA with ICS/LABA combinations on the incidence of exacerbations, patients had to first discontinue the use of ICS, LABA, LABA/ICS and LAMA [20]. They entered a 4-week run-in during which they received the LAMA tiotropium. Among the patients who were excluded during the tiotropium run-in and did not make it to the randomisation stage, 179 had a COPD exacerbation, three died and 67 had adverse events. The run-in is likely to have preserved more of the LAMA responders and fewer of the ICS responders, so that the randomised allocation to the LABA/LAMA or LABA/ICS combinations among the remaining patients would have favoured the patients allocated to the LAMA-containing combination. Consequently, the reported hazard ratio of a first exacerbation of 0.84 (95% CI 0.78–0.91) favouring the LABA/LAMA combination compared with the LABA/ICS combination may represent a magnification of the real effect.
Although less apparent, this bias can also arise when the run-in involves a placebo, which is generally used to identify and select for randomisation the patients who have a compliant behaviour [10, 24]. Indeed, patients who were well-controlled by a medication that was discontinued prior to entering the placebo run-in may incur a worsening of their disease and subsequently be excluded from the randomisation phase. Consequently, more non-responders to the discontinued drug will be included at randomisation, which will disfavour the study drug if it is the same, or in the same class, as the discontinued drug.
One way to minimise such run-in bias is to randomise the patients prior to the run-in period and allocate the randomly selected treatment after the run-in [25]. Such an approach would provide a comparison of the effect of the run-in medication followed by the randomly allocated drugs. Another suggested approach is to consider the patients excluded during the run-in period in the data analysis phase as a problem of missing data [26].
In conclusion, the results of randomised trials of COPD medications that involve a “run-in” period to bridge the gap between the abrupt discontinuation of usual treatment and the start of the study treatment must be interpreted with caution. Generally, such trials will favour the treatment that was also used during the run-in.
Disclosures
Supplementary Material
S. Suissa ERJ-00361-2017_Suissa
Footnotes
Conflict of interest: Disclosures can be found alongside this article at erj.ersjournals.com
- Received February 20, 2017.
- Accepted March 3, 2017.
- Copyright ©ERS 2017
References
