Skip to main content

Main menu

  • Home
  • Current issue
  • ERJ Early View
  • Past issues
  • Authors/reviewers
    • Instructions for authors
    • Submit a manuscript
    • Open access
    • COVID-19 submission information
    • Peer reviewer login
  • Alerts
  • Podcasts
  • Subscriptions
  • ERS Publications
    • European Respiratory Journal
    • ERJ Open Research
    • European Respiratory Review
    • Breathe
    • ERS Books
    • ERS publications home

User menu

  • Log in
  • Subscribe
  • Contact Us
  • My Cart

Search

  • Advanced search
  • ERS Publications
    • European Respiratory Journal
    • ERJ Open Research
    • European Respiratory Review
    • Breathe
    • ERS Books
    • ERS publications home

Login

European Respiratory Society

Advanced Search

  • Home
  • Current issue
  • ERJ Early View
  • Past issues
  • Authors/reviewers
    • Instructions for authors
    • Submit a manuscript
    • Open access
    • COVID-19 submission information
    • Peer reviewer login
  • Alerts
  • Podcasts
  • Subscriptions

Real world effects of COPD medications: a cohort study with validation against results from randomised controlled trials

Kevin Wing, Elizabeth Williamson, James R. Carpenter, Lesley Wise, Sebastian Schneeweiss, Liam Smeeth, Jennifer K. Quint, Ian Douglas
European Respiratory Journal 2021 57: 2001586; DOI: 10.1183/13993003.01586-2020
Kevin Wing
1Dept of Non-communicable Disease Epidemiology, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Elizabeth Williamson
2Dept of Medical Statistics, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Elizabeth Williamson
James R. Carpenter
2Dept of Medical Statistics, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lesley Wise
1Dept of Non-communicable Disease Epidemiology, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sebastian Schneeweiss
3Dept of Epidemiology, Harvard Medical School, Boston, MA, USA
4Division of Pharmacoepidemiology and Pharmacoeconomics, Dept of Medicine, Brigham and Women's Hospital, Boston, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Liam Smeeth
1Dept of Non-communicable Disease Epidemiology, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jennifer K. Quint
5National Heart and Lung Institute, Imperial College London, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jennifer K. Quint
Ian Douglas
1Dept of Non-communicable Disease Epidemiology, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

Abstract

Real-world data provide the potential for generating evidence on drug treatment effects in groups excluded from trials, but rigorous, validated methodology for doing so is lacking. We investigated whether non-interventional methods applied to real-world data could reproduce results from the landmark TORCH COPD trial.

We performed a historical cohort study (2000–2017) of COPD drug treatment effects in the UK Clinical Practice Research Datalink (CPRD). Two control groups were selected from CPRD by applying TORCH inclusion/exclusion criteria and 1:1 matching to TORCH participants, as follows. Control group 1: people with COPD not prescribed fluticasone propionate (FP)-salmeterol (SAL); control group 2: people with COPD prescribed SAL only. FP-SAL exposed groups were then selected from CPRD by propensity score matching to each control group. Outcomes studied were COPD exacerbations, death from any cause and pneumonia.

2652 FP-SAL exposed people were propensity score matched to 2652 FP-SAL unexposed people while 991 FP-SAL exposed people were propensity score matched to 991 SAL exposed people. Exacerbation rate ratio was comparable to TORCH for FP-SAL versus SAL (0.85, 95% CI 0.74–0.97 versus 0.88, 0.81–0.95) but not for FP-SAL versus no FP-SAL (1.30, 1.19–1.42 versus 0.75, 0.69–0.81). In addition, active comparator results were consistent with TORCH for mortality (hazard ratio 0.93, 0.65–1.32 versus 0.93, 0.77–1.13) and pneumonia (risk ratio 1.39, 1.04–1.87 versus 1.47, 1.25–1.73).

We obtained very similar results to the TORCH trial for active comparator analyses, but were unable to reproduce placebo-controlled results. Application of these validated methods for active comparator analyses to groups excluded from randomised controlled trials provides a practical way for contributing to the evidence base and supporting COPD treatment decisions.

Abstract

In COPD patients selected from real-world data based on similarity to participants of the TORCH RCT, non-interventional methods generated comparable results to the TORCH analysis of LABA-ICS versus LABA in relation to exacerbations, mortality and pneumonia https://bit.ly/33ky5D0

Introduction

The long-acting β2-agonist (LABA)–inhaled corticosteroid (ICS) combination product fluticasone propionate (FP) + salmeterol (SAL) is one of the most widely used COPD treatments. It was studied in large randomised trials (such as the landmark TORCH study [1]), but the effects of treatment in important patient groups who were either not included or underrepresented in trials are unknown. While real-world observational data has the potential to be used to carry out non-interventional studies of COPD drug treatment effects in groups excluded from trials, the use of these data for estimating treatment effectiveness is in its infancy. Demonstrating that non-interventional methods can account for the absence of treatment randomisation in real-world settings is a particular challenge. Rigorous, validated methodology is needed to translate these complex data into reliable evidence [2].

One approach is to analyse “randomised controlled trial (RCT)-analogous” cohorts from non-interventional data sources; if results are comparable to those generated by the reference RCT, this should increase confidence in the validity of the results, and in the non-interventional methods used to obtain them. The validated non-interventional methodology could then go on to be used for the analysis of treatment effects within people prescribed drugs in clinical practice who would have been excluded from (or were underrepresented in) RCTs [2].

In this study, we 1) applied trial inclusion and exclusion criteria to detect trial “eligible” participants from real-world data; 2) selected from these “eligible” participants to obtain a group who were as similar to the TORCH participants as possible by individual matching using TORCH data; and 3) applied standard observational methods to account for confounding [1].

We then assessed whether treatment effects in this real-world cohort were comparable to those measured by the TORCH trial in terms of the effect of the FP-SAL fixed combination product on 1) exacerbations; 2) mortality; and 3) pneumonia.

Materials and methods

Study design

A historical cohort study, with validation against RCT results.

Setting/data sources

This study uses individual trial data from the TORCH RCT (obtained via www.clinicalstudydatarequest.com), and non-interventional data from the UK Clinical Practice Research Datalink (CPRD) linked to Hospital Episodes Statistics (HES) data. We have described the use of CPRD in this study and the characteristics of TORCH in a previous publication (replicated in appendix 1) [2].

Diagnostic and therapeutic codelists

All diagnostic and therapeutic codelist files are available for download (https://datacompass.lshtm.ac.uk/1655/).

Selection of participants: FP-SAL exposed versus unexposed

Step 1: selection of all potentially eligible patients

An initial cohort was selected from all HES-linked patients actively registered in the CPRD between January 1, 2004 and January 1, 2017, who fulfilled the TORCH inclusion criteria (supplementary table A1-1) [1]. The date that an individual met all inclusion criteria with ≥12 months prior registration in the CPRD was the “eligible for TORCH inclusion” date.

Step 2: selection of pool of unexposed patients

Time periods during which patients were unexposed to FP-SAL that occurred on or after the eligible for TORCH inclusion date and which did not meet any of the TORCH drug exposure exclusion criteria (supplementary table A1-1) were selected (supplementary figure A1-1) [1]. The start of follow-up date (index date) for the unexposed time period was selected as a random date between the start and end of the unexposed period (supplementary figure A1-1). Individuals in CPRD were able to contribute more than one such unexposed time period to the total pool of unexposed time periods (supplementary figure A1-1). Unexposed time periods were then removed from the cohort if the patient met any of the remaining TORCH study exclusion criteria prior to the index date [1].

Step 3: selection of unexposed to FP-SAL people by 1:1 matching FP-SAL time periods to TORCH participants

Each individual TORCH participant from the TORCH RCT (obtained via www.clinicalstudydatarequest.com, as described earlier) was matched 1:1 with the closest available unexposed to FP-SAL time period on the following TORCH baseline characteristics: age, sex, body mass index (BMI), 1-year history of exacerbations requiring hospitalisation, history of cardiovascular disease and lung function (forced expiratory volume in 1 s (FEV1)) (appendix 1). An individual could only contribute one unexposed period to the final TORCH-matched unexposed cohort (supplementary figure A1-1); therefore, the output of this step was a cohort of unexposed to FP-SAL people.

Step 4: selection of exposed to FP-SAL time periods and application of TORCH exclusion criteria

All prescriptions for FP-SAL that started 1) on or after the initial eligible for TORCH inclusion date specified in Step 1 and 2) ≥4 weeks after the end of a prescription for any of the TORCH study drugs were identified. FP-SAL exposed time periods were created with the index date assigned as the start of an FP-SAL prescription. The same exclusion critiera applied to the unexposed FP-SAL time periods (Step 3) were applied. If an individual contributed time periods to both the unexposed (Step 2) and exposed (Step 4) cohorts, they were contributing different periods of their person-time to each cohort (pre-FP-SAL treatment for Step 2 versus post-FP-SAL treatment for Step 4) (supplementary figure A1-1).

Step 5: selection of comparable FP-SAL exposed participants by matching FP-SAL exposed time periods to FP-SAL unexposed people

Using the index date baseline characteristics, propensity scores for receiving FP-SAL were calculated for the (TORCH-matched) FP-SAL unexposed people selected in Step 3 and the FP-SAL exposed time periods selected in Step 4. Each FP-SAL unexposed (TORCH-matched) person selected in Step 3 was matched 1:1 with the FP-SAL exposed time period from Step 4 with the closest propensity score. An individual could only appear once as an exposed participant in the final propensity score matched cohort, meaning that this step selected FP-SAL exposed participants from the initial pool of FP-SAL exposed time periods. It was possible for the same person to be included in the FP-SAL unexposed and FP-SAL exposed cohorts, with different start of follow-up dates in each cohort.

Note that we did not apply matching to the TORCH trial in order to select our FP-SAL exposed group, because we wanted to develop propensity score methodology for obtaining balanced groups that could then be applied to the study of groups of patients who were not included in the trial at all.

Selection of participants: FP-SAL exposed versus SAL exposed

The participant selection approach was analogous to the FP-SAL exposed versus FP-SAL unexposed participant selection, except the comparator group was those exposed to SAL, and the study period was 2000–2017 (to obtain sufficient SAL-exposed numbers). The other differences in participant selection are detailed in appendix 1.

Exposures, outcomes and covariates

Exposures

Exposure status was determined using CPRD prescribing records. Being prescribed FP-SAL was the primary exposure of interest and the comparison exposure groups were 1) people not being prescribed FP-SAL and 2) people being prescribed SAL only. Refer to appendix 1 for further details.

Outcomes

Outcomes were COPD exacerbation, all-cause mortality, pneumonia and time to treatment discontinuation (refer to appendix 1: Outcomes – further details on outcome definitions).

Covariates

Covariates available for inclusion in the propensity score models have been detailed previously and are listed in appendix 1 [2].

Statistical analysis

Propensity score for addressing confounding

A pool of initial variables were selected based upon a priori knowledge/clinical expertise. Those variables not associated with outcome in crude analysis were then removed, before applying multivariable logistic regression (on drug exposure status) to generate propensity scores [2]. Variables were selected for inclusion in the final propensity score multivariable logistic regression model using likelihood ratio tests for goodness of fit. Starting from an initial fully adjusted model that included all initial variables found to be associated with outcome, goodness of fit was tested after removing variables sequentially from the logistic regression model (starting with the variable most weakly associated with exposure in the fully adjusted model). Variables with likelihood ratio test p-value >0.1 were removed from the model. Separate propensity scores were developed in this way for each outcome. Standardised differences were used to assess any residual imbalances after matching (with sd >0.1 indicating substantial/important imbalance) [3].

Methods of analysis

Comparisons were made for each outcome over 3 years between 1) people exposed to FP-SAL versus people unexposed to FP-SAL (matched on propensity score) and 2) people exposed to FP-SAL versus people exposed to SAL (matched on propensity score). All analyses were performed according to the intention-to-treat principle (appendix 1), estimating the same effect measures as TORCH. The number of exacerbations was modelled using a negative binomial model with the log of treated time as an offset variable. Time to mortality and treatment discontinuation were analysed using Cox proportional hazards regression, and pneumonia risk was analysed using Poisson regression.

Validation of results against TORCH

Detailed criteria for considering results to be comparable with TORCH were pre-specified and have been published previously (appendix 1) [2].

Missing data

Complete records analysis was applied given the low proportion of missing data (only socioeconomic status, alcohol or BMI had any missing data, and of these variables, none had >5% missing data).

Analysis of impact of TORCH matching and TORCH criteria (post hoc analysis)

A post hoc analysis was performed assessing the impact of 1) omitting TORCH matching and 2) omitting both TORCH matching and application of TORCH trial inclusion/exclusion criteria (refer to appendix 1 for details).

Ethics

Scientific approval was provided by the London School of Hygiene and Tropical Medicine research ethics commitee (ref 11997) and the independent scientific advisory committee of the Medicines and Healthcare Products Regulatory Agency (protocol no. 17_114R). CPRD data are already approved via a national research ethics committee for purely non-interventional research of this type. Approval for use of the TORCH trial data was obtained from the Wellcome Trust, the relevant sponsor (GSK) and an independent review panel.

Results

Participants

FP-SAL exposed versus FP-SAL unexposed

Between January 1, 2004 and January 1, 2017 there were 125 671 people in CPRD with a diagnosis of COPD, 73 889 (59%) of whom were from HES-linked CPRD practices (figure 1). Application of TORCH inclusion criteria reduced this to 18 715 people, contributing 35 746 unexposed to FP-SAL time periods and 26 390 exposed to FP-SAL time periods. After applying TORCH exclusion criteria, dropping records with missing covariate data and matching the unexposed patients to TORCH participants, there were 4196 unexposed patients available for propensity score matching to 10 463 FP-SAL exposed time periods. The final propensity score matched cohorts included 2652 patients in each exposure group for the exacerbations analysis, 2708 for mortality and 2779 for pneumonia.

FIGURE 1
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 1

Flow of number of individuals included in the exposed to fluticasone propionate (FP)-salmeterol (SAL) versus unexposed to FP-SAL cohort analysis. Current/previous use of COPD drugs relates to any of the drugs studied in TORCH, long-acting bronchodilators and oral corticosteroids; refer to supplementary table A1-1 for specific details. CPRD: Clinical Practice Research Datalink; HES: Hospital Episodes Statistics; BMI: body mass index; SES: socioeconomic status. #: see supplementary table A1-1 for detailed list of inclusion criteria; ¶: see supplementary table A1-1 for detailed list of exclusion criteria.

FP-SAL exposed versus SAL exposed

For the FP-SAL versus SAL analysis, there were 154 785 people with a diagnosis of COPD in CPRD between January 1, 2000 and January 1, 2017, 91 733 (59%) of whom were from HES-linked CPRD practices (figure 2). 1146 SAL exposed patients were available for propensity score matching to 11 235 FP-SAL-exposed periods. The final propensity score matched cohorts included 991 (exacerbations), 432 (mortality), 935 (pneumonia) and 996 (treatment discontinuation) patients per exposure group.

FIGURE 2
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 2

Flow of number of individuals included in the exposed to fluticasone propionate (FP)-salmeterol (SAL) versus exposed to SAL cohort analysis. Current/previous use of COPD drugs relates to any of the drugs studied in TORCH, long-acting bronchodilators and oral corticosteroids (OCS); refer to supplementary table A1-1 for specific details. CPRD: Clinical Practice Research Datalink; HES: Hospital Episodes Statistics; BMI: body mass index; SES: socioeconomic status. #: see supplementary table A1-1 for detailed list of inclusion criteria; ¶: see supplementary table A1-1 for detailed list of exclusion criteria.

Application of TORCH inclusion/exclusion criteria and matching to TORCH

Applying the TORCH inclusion/exclusion criteria and matching to TORCH resulted in cohorts that were much more similar to those recruited to the TORCH trial (e.g. FEV1 for the FP-SAL versus unexposed to FP-SAL analysis was 66.3% predicted in CPRD before applying any criteria or matching, compared to 47.2% pred after these steps, compared to a TORCH placebo group value of 44.2% pred; supplementary table A2-1). The largest residual difference to the TORCH placebo group was for prior cardiovascular disease for both comparisons (supplementary tables A2-1 and A2-2).

Propensity score matching of CPRD cohorts

Details of the variables included in the final propensity score models are provided in supplementary table A2-3.

FP-SAL exposed versus FP-SAL unexposed

Prior to propensity score matching, differences by exposure status were noted for sex, FEV1, BMI, prior exacerbations, coronary heart disease, peripheral vascular disease, cerebrovascular disease, prescriptions for aspirin, COPD medications, number of general practice (GP) consultations and number of distinct medications (table 1). After propensity score matching, only the difference with respect to coronary heart disease, peripheral vascular disease, LABA and consultations persisted (table 1). Plots of propensity score distributions indicated close propensity score matching for exacerbations and all other outcomes under study (supplementary figure A2-1).

View this table:
  • View inline
  • View popup
TABLE 1

Characteristics of the fluticasone propionate (FP)-salmeterol (SAL) versus unexposed to FP-SAL cohort before and after propensity score matching for the exacerbations analysis

FP-SAL exposed versus SAL exposed

For the FP-SAL versus SAL exacerbations analysis, after propensity score matching there were notable imbalances in prior prescriptions for a LABA or an ICS, with smaller imbalances for lung function, BMI, coronary heart disease, statin prescription, aspirin prescription, LAMA and ICS plus LABA (table 2). Plots of propensity score distribution indicated that overall groups were well matched on propensity score for each outcome (supplementary figure A2-1).

View this table:
  • View inline
  • View popup
TABLE 2

Characteristics of the fluticasone propionate (FP)-salmeterol (SAL) versus SAL cohort before and after propensity score matching for the exacerbations analysis

Main results

FP-SAL exposed versus FP-SAL unexposed

For the exacerbations analysis, the rate ratio in the propensity score matched groups was 1.30 (95% CI 1.19–1.42) (table 3). According to our pre-specified protocol this (harmful) association was not considered to be consistent with the (protective) TORCH placebo-controlled result for the same outcome (0.75, 0.69–0.81) [2]. Similarly, our result for the mortality outcome (hazard ratio (HR) 1.11, 95% CI 0.95–1.26) was in the opposite direction to the TORCH placebo-controlled result (0.83, 0.68–1.00). For the pneumonia analysis, we found weak evidence for a 14% increased risk associated with FP-SAL (risk ratio 1.14, 0.96–1.34) which was not consistent with the stronger harmful association found by the TORCH placebo-controlled analysis (1.59, 1.35–1.88).

View this table:
  • View inline
  • View popup
TABLE 3

Results for the analysis of exacerbations, mortality, pneumonia and time to treatment discontinuation for fluticasone propionate (FP)-salmeterol (SAL) versus no FP-SAL (compared to TORCH results)

FP-SAL exposed versus SAL exposed

For the exacerbations analysis, we obtained a propensity score matched rate ratio of 0.85 (95% CI 0.74–0.97). According to our pre-specified protocol this (protective) effect was considered to be consistent with the TORCH FP-SAL versus SAL result for the same outcome (0.88, 0.81–0.95) (table 4) [2]. Similarly, our result for the mortality outcome (HR 0.93, 95% CI 0.65–1.32) was consistent with the TORCH FP-SAL versus SAL result (0.93, 0.77–1.13). For the pneumonia analysis, we found evidence for a 39% increased risk associated with FP-SAL (risk ratio 1.39, 95% CI 1.04–1.87), which was also consistent with the harmful association found by the TORCH FP-SAL versus SAL analysis (1.47, 1.25–1.73). For the time to treatment discontinuation analysis, the effect was apparently much stronger outside of the trial setting (non-interventional HR 0.23, 95% CI 0.20–0.27 versus TORCH 0.89, 0.79–0.99).

View this table:
  • View inline
  • View popup
TABLE 4

Results for the analysis of exacerbations, mortality, pneumonia and time to treatment discontinuation for fluticasone propionate (FP)-salmeterol (SAL) versus SAL (compared to TORCH results)

Analysis of impact of TORCH matching and TORCH criteria (post hoc analysis)

Repeating the FP-SAL versus SAL analysis omitting the TORCH matching step led to an exacerbations rate ratio of 0.87 (95% CI 0.81–0.94) (table 5), very similar to both the main analysis and TORCH result. In contrast, applying neither the TORCH criteria nor matching led to a completely different effect estimate (1.64, 1.52–1.77).

View this table:
  • View inline
  • View popup
TABLE 5

Impact of choice of selection methods on ability to replicate trial results for the analysis of exacerbations, in people exposed to fluticasone propionate (FP)-salmeterol (SAL) versus people exposed to SAL

Discussion

We have demonstrated that methods applied to non-interventional data can generate results comparable to active comparator trials for COPD treatment effects. In contrast, we found that the same methods were unable to replicate placebo-controlled trial results.

Comparison with previous studies

Previous studies applying similar “trial-replication” approaches

Although a number of papers have compared the designs of observational studies with RCTs [4–9], and some studies have generated results similar to an earlier or subsequent trial [10–12], there are very few non-interventional studies that have set out to explicitly replicate a specific trial cohort and its results.

Hernán et al. [13] replicated the design and result of the Women's Health Initiative randomised trial on the effect of oestrogen/progestin therapy on coronary heart disease risk. Smeeth et al. [14] analysed the effect of statins on a range of health outcomes and replicated the Heart Protection Study randomised trial. Fralick et al. [15] applied trial criteria and utilised propensity score matching to replicate cardiovascular results from the ONTARGET trial (comparing telmisartan to ramipril).

Previous studies of COPD drug treatment effects

Results of five (LABA/ICS versus LABA) interventional studies (including TORCH) were summarised in a Cochrane review (rate ratio 0.76, 95% CI 0.68–0.84) [16]. Three out of these five studies estimated effect sizes considerably greater than TORCH; as we mirrored TORCH, our results aligned most closely to TORCH.

A number of studies have found strong survival benefits of ICS therapy after hospital discharge [17, 18]. After accounting for likely time-related biases impacting these studies, a null effect was obtained (rate ratio 0.94, 95% CI 0.81–1.09) [19]. The methodology we applied obtained a mortality effect estimate comparable to the analysis designed to account for time-related biases (0.93, 95% CI 0.65–1.32).

In line with TORCH, previous studies have found an increased risk of pneumonia associated with ICS-containing treatments for COPD [20, 21, 16]. Our result (risk ratio 1.39, 95% CI 1.04–1.87) was consistent with results of a meta-analysis of trials comparing LABA/ICS to LABA formulations (1.55, 1.20–2.01) [16] and very similar to a recent non-interventional study comparing LABA/ICS to LAMA formulations (HR 1.37, 95% CI 1.17–1.60) [22].

Our 3-year probability of treatment discontinuation for FP-SAL (28%) is comparable to non-adherence figures from previous non-interventional real-world data studies (49% [23] and 43% [24]). The probability of discontinuation of salmeterol that we observed (77%) was higher than these two previous non-interventional studies, leading to the discrepancy with TORCH. We hypothesised that during our study period a large proportion of the patients who would have been initially prescribed salmeterol would have been likely to switch to FP-SAL due to prescribing decisions in primary care; a post hoc analysis found that 43% of people prescribed salmeterol switched to FP-SAL during follow-up (compared to only 2% switching from FP-SAL to salmeterol).

Implications and further work

When studying COPD treatment effects, if 1) the analysis is of active comparators; 2) trial exclusion and inclusion criteria are applied; and 3) the propensity score models that we developed for each outcome are applied to balance exposure groups, then the results of studies carried out in routinely collected non-interventional data can be considered robust in the sense that they will be highly comparable to trial results. This now provides a methodological framework for being able to analyse COPD drug treatment effects in real-world data, focusing on groups that were either not included or underrepresented in trials [2].

Our inability to replicate placebo-controlled analyses suggests uncontrolled confounding by indication [25]. One possibility for how this confounding by indication may be manifesting relates to the aspect of our study design that allowed people to be included in both the exposed and unexposed cohorts; the result we obtained could be strongly influenced by people initially in the unexposed group who are relatively healthy, but then get sicker over time and require FP-SAL treatment and end up in the exposed group. However, in a post hoc analysis where we dropped the 730 (out of a total of 2652 per group) people who appeared in both cohorts, our effect estimate was nearly identical (rate ratio 1.33, 95% CI 1.20–1.47). However, we do consider that because COPD treatment is based on a step-up approach, it is highly likely that patients not exposed to FP-SAL in routine primary care are generally likely to be those with milder COPD.

One point of note relates to the large difference in incidence rate between the TORCH placebo group (1.13 exacerbations per person per year) and our FP-SAL unexposed group (0.53 exacerbations per person per year). In order to investigate underlying reasons for this discrepancy, we performed a post hoc analysis where we compared the characteristics of the 1753 people from TORCH who were not able to be matched to our unexposed to FP-SAL population in Step 3 with those who were successfully matched. We found that those not matched were younger (mean age 60.7 years versus 65.8 years), sicker (e.g. history of cardiovascular disease 93% versus 46%), had worse lung function (e.g. FEV1 34.9% pred versus 45.9% pred), and included a higher proportion recruited from Eastern European trial sites (27% versus 17%). People with these characteristics may have been highly suitable for recruitment to clinical trials, but are very difficult to find in UK primary care, and illustrate why it is likely to be challenging to obtain comparable absolute rates in emulated cohorts within a single country based on historical international trials.

Previous authors have recommended that when trying to emulate trial results, it is important to choose an active comparator trial [15]. However, there are examples where placebo-controlled analyses have been successfully replicated [13, 14]. One possibility is that replication of placebo-controlled results works better when the drug studied is 1) preventative and 2) used in a generally healthy cohort (for example, the cited studies were of statins and of post-menopausal hormone therapy both prescribed in some instances to people without a specific underlying chronic disease, in contrast to the patients with COPD who received therapy in our study). We consider that further avenues of research could be followed to understand if there remains a possibility of replicating placebo-controlled studies within a non-interventional setting for COPD therapies. These could include application of high-dimensional propensity scores or the use of instrumental variables. In addition, our work suggests that treatment discontinuation in the setting of non-interventional data may be driven by very different factors to those seen in trials and, at least in the setting of COPD, may not be a useful outcome to study. For example, it is difficult to establish from routinely collected data whether a patient has truly stopped taking their medicine, or is just taking the medicine differently than prescribed (e.g. is taking less than has been prescribed over a longer period).

Finally, in our post hoc analysis we found that the application of the trial-matching step did not confer any advantage over application of trial criteria alone in this setting. This suggests that treatment-covariate interactions are not as critical as we initially thought in this therapeutic area.

Limitations

Some of the TORCH inclusion criteria were not fully assessable using CPRD data, meaning that the inclusion/exclusion criteria are analogous with TORCH criteria, but we acknowledge they are not identical. We originally planned to apply frequency of COPD therapy prescriptions in the previous year as a matching character/criteria. In practice, this was not feasible. However, it appears that matching at this level of detail was not required to be able to replicate trial results for active comparator analysis. Finally, within TORCH, the dose of the fixed combination product FP-SAL was specified as 500 µg of FP and 50 µg of SAL (500/50), and the dose of SAL alone as 50 µg, whereas in our study we did not limit to a specific dose. The reason for this is that dosage information is incompletely captured in CPRD, but as these are the only approved doses of FP-SAL and of SAL for COPD in the UK, we consider the doses that people were prescribed in our study would have been generally similar to that administered in the TORCH trial.

Conclusions

By replicating the COPD TORCH trial selection procedures and inclusion/exclusion criteria in real-world data and developing propensity score models to account for any remaining differences between groups, we were able to obtain highly comparable relative effect estimates to the TORCH RCT active comparator analysis for exacerbations, mortality and pneumonia. Replication of placebo-controlled analyses was not possible, and further work to investigate whether likely residual confounding by indication can ever be accounted for in this therapeutic area is warranted. Application of the same selection procedures and propensity score models developed here to active comparator analyses of COPD drug treatment effects in groups underrepresented or excluded from trials provides a practical way for key evidence gaps to be filled.

Supplementary material

Supplementary Material

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Supplementary material: appendix 1 and 2 ERJ-01586-2020.SUPPLEMENT

Shareable PDF

Supplementary Material

This one-page PDF can be shared freely online.

Shareable PDF ERJ-01586-2020.Shareable

Footnotes

  • This article has supplementary material available from erj.ersjournals.com

  • Conflict of interest: K. Wing has nothing to disclose.

  • Conflict of interest: E. Williamson has nothing to disclose.

  • Conflict of interest: J.R. Carpenter reports grants from MRC (MC_UU_12023/21 and MC_UU_12023/29), personal fees for consultancy from Novartis and GSK, and has been involved in educational programmes for GSK, Bayer and Boehringer, outside the submitted work.

  • Conflict of interest: L. Wise reports personal fees for consultancy from the pharmaceutical industry, outside the submitted work.

  • Conflict of interest: S. Schneeweiss reports personal fees for consultancy from WHISCON LLC, personal fees for consultancy and owns equity in Aetion, grants from Bayer, Genentech and Boehringer, outside the submitted work.

  • Conflict of interest: L. Smeeth reports grants from Wellcome, MRC, NIHR, BHF, Diabetes UK, ESRC, EU and GSK, personal fees from GSK and AstraZeneca, and is a trustee of the British Heart Foundation, outside the submitted work.

  • Conflict of interest: J.K. Quint reports grants from MRC, Wellcome, BLF, GSK, Boehringer, AstraZeneca and Insmed, outside the submitted work.

  • Conflict of interest: I. Douglas reports grants, personal fees for consultancy and holds stocks in GSK, outside the submitted work.

  • Support statement: This work was supported by NIHR grant number 15/80/28. Funding information for this article has been deposited with the Crossref Funder Registry.

  • Received May 5, 2020.
  • Accepted September 26, 2020.
  • Copyright ©ERS 2021
https://www.ersjournals.com/user-licence

References

  1. ↵
    1. Calverley PMA,
    2. Anderson JA,
    3. Celli B, et al.
    Salmeterol and fluticasone propionate and survival in chronic obstructive pulmonary disease. N Engl J Med 2007; 356: 775–789. doi:10.1056/NEJMoa063070
    OpenUrlCrossRefPubMedWeb of Science
  2. ↵
    1. Wing K,
    2. Williamson E,
    3. Carpenter JR, et al.
    Real-world effects of medications for chronic obstructive pulmonary disease: protocol for a UK population-based non-interventional cohort study with validation against randomised trial results. BMJ Open 2018; 8: e019475. doi:10.1136/bmjopen-2017-019475
    OpenUrlAbstract/FREE Full Text
  3. ↵
    1. Williamson EJ,
    2. Forbes A
    . Introduction to propensity scores. Respirology 2014; 19: 625–635. doi:10.1111/resp.12312
    OpenUrlCrossRefPubMed
  4. ↵
    1. Concato J,
    2. Shah N,
    3. Horwitz RI
    . Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000; 342: 1887–1892. doi:10.1056/NEJM200006223422507
    OpenUrlCrossRefPubMedWeb of Science
    1. Golder S,
    2. Loke YK,
    3. Bland M
    . Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med 2011; 8: e1001026. doi:10.1371/journal.pmed.1001026
    OpenUrlCrossRefPubMed
    1. Deeks JJ,
    2. Dinnes J,
    3. D'Amico R, et al.
    Evaluating non-randomised intervention studies. Health Technol Assess 2003; 7: 1–173. doi:10.3310/hta7270
    OpenUrlCrossRefPubMed
    1. Lawlor DA,
    2. Smith GD,
    3. Kundu D, et al.
    Those confounded vitamins: what can we learn from the differences between observational versus randomised trial evidence? Lancet 2004; 363: 1724–1727. doi:10.1016/S0140-6736(04)16260-0
    OpenUrlCrossRefPubMedWeb of Science
    1. Vandenbroucke JP
    . The HRT controversy: observational studies and RCTs fall in line. Lancet 2009; 373: 1233–1235. doi:10.1016/S0140-6736(09)60708-X
    OpenUrlCrossRefPubMedWeb of Science
  5. ↵
    1. Danaei G,
    2. Rodríguez LAG,
    3. Cantero OF, et al.
    Observational data for comparative effectiveness research: an emulation of randomised trials of statins and primary prevention of coronary heart disease. Stat Methods Med Res 2013; 22: 70–96. doi:10.1177/0962280211403603
    OpenUrlCrossRefPubMed
  6. ↵
    1. Douglas IJ,
    2. Evans SJ,
    3. Pocock S, et al.
    The risk of fractures associated with thiazolidinediones: a self-controlled case-series study. PLoS Med 2009; 6: e1000154. doi:10.1371/journal.pmed.1000154
    OpenUrlCrossRefPubMed
    1. Langan SM,
    2. Smeeth L,
    3. Margolis DJ, et al.
    Herpes zoster vaccine effectiveness against incident herpes zoster and post-herpetic neuralgia in an older US population: a cohort study. PLoS Med 2013; 10: e1001420. doi:10.1371/journal.pmed.1001420
    OpenUrlCrossRefPubMed
  7. ↵
    1. Schneeweiss S,
    2. Seeger JD,
    3. Landon J, et al.
    Aprotinin during coronary-artery bypass grafting and risk of death. N Engl J Med 2008; 358: 771–783. doi:10.1056/NEJMoa0707571
    OpenUrlCrossRefPubMedWeb of Science
  8. ↵
    1. Hernán MA,
    2. Alonso A,
    3. Logan R, et al.
    Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology 2008; 19: 766–779. doi:10.1097/EDE.0b013e3181875e61
    OpenUrlCrossRefPubMedWeb of Science
  9. ↵
    1. Smeeth L,
    2. Douglas I,
    3. Hall AJ, et al.
    Effect of statins on a wide range of health outcomes: a cohort study validated by comparison with randomized trials. Br J Clin Pharmacol 2009; 67: 99–109. doi:10.1111/j.1365-2125.2008.03308.x
    OpenUrlCrossRefPubMedWeb of Science
  10. ↵
    1. Fralick M,
    2. Kesselheim AS,
    3. Avorn J, et al.
    Use of health care databases to support supplemental indications of approved medications. JAMA Intern Med 2018; 178: 55–63. doi:10.1001/jamainternmed.2017.3919
    OpenUrl
  11. ↵
    1. Nannini LJ,
    2. Lasserson TJ,
    3. Poole P
    . Combined corticosteroid and long-acting beta2-agonist in one inhaler versus long-acting beta2-agonists for chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2012; 9: CD006829.
    OpenUrlPubMed
  12. ↵
    1. Soriano JB,
    2. Vestbo J,
    3. Pride NB, et al.
    Survival in COPD patients after regular use of fluticasone propionate and salmeterol in general practice. Eur Respir J 2002; 20: 819–825. doi:10.1183/09031936.02.00301302
    OpenUrlAbstract/FREE Full Text
  13. ↵
    1. Mapel DW,
    2. Hurley JS,
    3. Roblin D, et al.
    Survival of COPD patients using inhaled corticosteroids and long-acting beta agonists. Respir Med 2006; 100: 595–609. doi:10.1016/j.rmed.2005.08.006
    OpenUrlCrossRefPubMedWeb of Science
  14. ↵
    1. Suissa S
    . Inhaled steroids and mortality in COPD: bias from unaccounted immortal time. Eur Respir J 2004; 23: 391–395. doi:10.1183/09031936.04.00062504
    OpenUrlAbstract/FREE Full Text
  15. ↵
    1. Suissa S,
    2. Patenaude V,
    3. Lapi F, et al.
    Inhaled corticosteroids in COPD and the risk of serious pneumonia. Thorax 2013; 68: 1029–1036. doi:10.1136/thoraxjnl-2012-202872
    OpenUrlAbstract/FREE Full Text
  16. ↵
    1. Ernst P,
    2. Gonzalez AV,
    3. Brassard P, et al.
    Inhaled corticosteroid use in chronic obstructive pulmonary disease and the risk of hospitalization for pneumonia. Am J Respir Crit Care Med 2007; 176: 162–166. doi:10.1164/rccm.200611-1630OC
    OpenUrlCrossRefPubMedWeb of Science
  17. ↵
    1. Suissa S,
    2. Dell'Aniello S,
    3. Ernst P
    . Comparative effectiveness of LABA-ICS versus LAMA as initial treatment in COPD targeted by blood eosinophils: a population-based cohort study. Lancet Respir Med 2018; 6: 855–862. doi:10.1016/S2213-2600(18)30368-0
    OpenUrl
  18. ↵
    1. Bogart M,
    2. Stanford RH,
    3. Laliberté F, et al.
    Medication adherence and persistence in chronic obstructive pulmonary disease patients receiving triple therapy in a USA commercially insured population. Int J Chron Obstruct Pulmon Dis 2019; 14: 343–352. doi:10.2147/COPD.S184653
    OpenUrl
  19. ↵
    1. Mueller S,
    2. Wilke T,
    3. Bechtel B, et al.
    Non-persistence and non-adherence to long-acting COPD medication therapy: a retrospective cohort study based on a large German claims dataset. Respir Med 2017; 122: 1–11. doi:10.1016/j.rmed.2016.11.008
    OpenUrlPubMed
  20. ↵
    1. Kyriacou DN,
    2. Lewis RJ,
    3. Rothman K, et al.
    Confounding by indication in clinical research. JAMA 2016; 316: 1818–1819. 10.1001/jama.2016.16435.
    OpenUrlCrossRefPubMed
PreviousNext
Back to top
View this article with LENS
Vol 57 Issue 3 Table of Contents
European Respiratory Journal: 57 (3)
  • Table of Contents
  • Index by author
Email

Thank you for your interest in spreading the word on European Respiratory Society .

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Real world effects of COPD medications: a cohort study with validation against results from randomised controlled trials
(Your Name) has sent you a message from European Respiratory Society
(Your Name) thought you would like to see the European Respiratory Society web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Print
Citation Tools
Real world effects of COPD medications: a cohort study with validation against results from randomised controlled trials
Kevin Wing, Elizabeth Williamson, James R. Carpenter, Lesley Wise, Sebastian Schneeweiss, Liam Smeeth, Jennifer K. Quint, Ian Douglas
European Respiratory Journal Mar 2021, 57 (3) 2001586; DOI: 10.1183/13993003.01586-2020

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero

Share
Real world effects of COPD medications: a cohort study with validation against results from randomised controlled trials
Kevin Wing, Elizabeth Williamson, James R. Carpenter, Lesley Wise, Sebastian Schneeweiss, Liam Smeeth, Jennifer K. Quint, Ian Douglas
European Respiratory Journal Mar 2021, 57 (3) 2001586; DOI: 10.1183/13993003.01586-2020
del.icio.us logo Digg logo Reddit logo Technorati logo Twitter logo CiteULike logo Connotea logo Facebook logo Google logo Mendeley logo
Full Text (PDF)

Jump To

  • Article
    • Abstract
    • Abstract
    • Introduction
    • Materials and methods
    • Results
    • Discussion
    • Supplementary material
    • Shareable PDF
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • PDF

Subjects

  • COPD and smoking
  • Tweet Widget
  • Facebook Like
  • Google Plus One

More in this TOC Section

Original Articles

  • Ambulatory management of secondary spontaneous pneumothorax
  • Systematic assessment of respiratory health in illness susceptible athletes
  • Identifying early PAH biomarkers in systemic sclerosis
Show more Original Articles

COPD

  • Flavonoid intakes inversely associate with COPD in smokers
  • Use of singing for lung health in COPD
  • Response to exercise training and withdrawal in COPD
Show more COPD

Related Articles

Navigate

  • Home
  • Current issue
  • Archive

About the ERJ

  • Journal information
  • Editorial board
  • Reviewers
  • Press
  • Permissions and reprints
  • Advertising

The European Respiratory Society

  • Society home
  • myERS
  • Privacy policy
  • Accessibility

ERS publications

  • European Respiratory Journal
  • ERJ Open Research
  • European Respiratory Review
  • Breathe
  • ERS books online
  • ERS Bookshop

Help

  • Feedback

For authors

  • Instructions for authors
  • Publication ethics and malpractice
  • Submit a manuscript

For readers

  • Alerts
  • Subjects
  • Podcasts
  • RSS

Subscriptions

  • Accessing the ERS publications

Contact us

European Respiratory Society
442 Glossop Road
Sheffield S10 2PX
United Kingdom
Tel: +44 114 2672860
Email: journals@ersnet.org

ISSN

Print ISSN:  0903-1936
Online ISSN: 1399-3003

Copyright © 2022 by the European Respiratory Society