Abstract
International treatment guidelines for idiopathic pulmonary fibrosis (IPF) give a conditional recommendation for anti-acid therapy. As some observational studies reported discrepant findings on the effectiveness of anti-acid therapy on mortality in IPF, we reviewed all studies to evaluate whether immortal time bias explains these discrepancies.
We searched the Embase and MEDLINE databases up to July 2017 for observational studies assessing the effectiveness of anti-acid therapy on mortality in IPF. Hazard ratios of mortality with anti-acid therapy were pooled across studies using random-effect models, stratified by the presence of immortal time bias.
We identified 10 observational studies. Four of the five studies reporting beneficial effects of anti-acid therapy use on mortality were affected by immortal time bias (pooled hazard ratio 0.46; 95% CI 0.30–0.69), while it was unclear whether the fifth was affected. The five studies that avoided immortal time bias reported no effect of anti-acid therapy on mortality (pooled hazard ratio 0.99; 95% CI 0.81–1.22).
The apparent beneficial effects of anti-acid therapy on mortality in patients with IPF result from observational studies affected by immortal time bias. Thus, the effectiveness of anti-acid therapy in IPF remains uncertain and needs to be reassessed with more accurate observational study methods and randomised trials.
Abstract
Immortal time bias explains apparent effectiveness of anti-acid drugs on mortality in idiopathic pulmonary fibrosis http://ow.ly/ajZl30jFTUf
Introduction
Idiopathic pulmonary fibrosis (IPF) is a rare and irreversible fibrotic lung disorder of unknown cause. It is characterised by a progressive decline of lung function and is associated with poor prognosis [1, 2]. Median survival after diagnosis is estimated to be 2–5 years [3, 4]. Between 2011 and 2015, two anti-fibrotic medications were approved for the treatment of patients with mild to moderate IPF in the United States, Canada and Europe: pirfenidone and nintedanib [5–7]. These two medications slow the decline in lung function, but have not been shown to reduce all-cause mortality in sufficiently powered studies [7]. Pirfenidone and nintedanib are now conditionally recommended by international IPF treatment guidelines [8].
Another conditionally recommended treatment for IPF is anti-acid therapy, which is normally indicated for gastro-oesophageal reflux disease (GORD), a common comorbidity in patients with IPF [9, 10]. However, the evidence supporting this recommendation, regardless of the presence of GORD, is generally weak with very low confidence in effect estimates [8, 11–14]. Thus, newer national guidelines (Germany, Switzerland and Sweden) recommend anti-acid therapy only as treatment for GORD in IPF [15–17]. After the latest update of the treatment guidelines in 2015, observational studies investigating the effectiveness of anti-acid therapy, including proton pump inhibitors (PPIs) continued to report conflicting results on all-cause mortality [18–24]. Results ranged from large reductions in mortality associated with use of anti-acid therapy to no association. The highly beneficial findings are of such remarkable magnitude that they are likely to be a result of biases, in particular time-related biases such as immortal time bias, that have been shown to affect observational studies of drug effects in various therapeutic areas [25–27].
Immortal time is a period during follow-up in which, by design, the study outcome cannot occur [28]. Immortal time is typically introduced when the individual's exposure/treatment status is determined after the start of follow-up. Individuals who are classified as exposed have to be alive and event-free until the exposure definition is met. Misclassification or exclusion of the immortal time period leads to immortal time bias. This type of bias is often introduced in cohort studies of drug effects and artificially reduces the rate of events occurring in the treated group, which leads to the conclusion that exposure to the treatment is protective, even when there is no treatment effect [27].
To date, immortal time bias has not been described in the field of IPF. In this methodological appraisal, we review observational studies evaluating the association between anti-acid therapy and mortality in IPF, particularly to identify those affected by immortal time bias. For the sake of brevity, our review focuses solely on mortality as the outcome of interest and not on other outcomes of interest in IPF, such as lung function decline and quality of life. In addition, we discuss other methodological issues that led to the discrepant findings among the observational drug effectiveness studies [29].
Methods
We identified publications and abstracts in Embase and MEDLINE (from the earliest available online year until July 2017) using a search strategy based on a combination of concepts addressing the study population, the exposure and the outcome of interest: IPF AND anti-acid therapy AND mortality. We used keywords and derivations thereof for “idiopathic pulmonary fibrosis”, “anti-acid therapy”, “GORD treatment” or “proton pump inhibitors”, “mortality” and “survival” (online supplementary material 1). Titles and abstracts were screened for eligibility, with full texts of eligible studies carefully reviewed. In addition, we examined the references of included articles and those in previous reviews. Studies had to provide information on mortality. We only included studies with a comparison group that reported hazard ratios (HRs) or other estimates, which allowed us to approximate the HR and its corresponding 95% confidence interval if no HR was reported. The Methods section of each included study was reviewed in depth for various sources of bias, including time-related biases, particularly immortal time bias, by assessing the available information on study design, exposure definition and statistical analysis [28]. Study-specific HRs were then pooled using random-effect models, stratified by studies with or without immortal time bias.
Results
Overall, we identified 10 relevant studies published from 2011 to 2017 (online supplementary material 2): seven cohort studies and three retrospective observational studies using pooled data from randomised controlled trials (RCTs) of other treatments for IPF [13, 14, 18–24, 30]. The pooled HR of all-cause mortality associated with anti-acid therapy use over all 10 studies was 0.63 (95% CI 0.45–0.90).
Five of the cohort studies associated use of anti-acid therapy with a reduced risk in mortality in IPF (table 1) [13, 18, 22, 24, 30]. Four of these cohort studies clearly introduced immortal time bias in their study design and analysis, as described in an example later. In contrast, the studies using methods to avoid immortal time bias, such as a time-dependent data analysis, did not find an association between anti-acid therapy use and mortality (table 2) [14, 19–21, 23]. Figure 1 displays the forest plot of the results for the four studies clearly demonstrating immortal time bias (pooled HR 0.46, 95% CI 0.30–0.69) in contrast with the five studies that avoided this bias (pooled HR 0.99, 95% CI 0.81–1.22), with the difference between the two HRs being statistically significant (p<0.01).
Immortal time bias in cohort studies investigating the effects of anti-acid therapy on all-cause mortality in idiopathic pulmonary fibrosis (IPF)
Observational studies investigating the effects of anti-acid therapy on all-cause mortality in idiopathic pulmonary fibrosis (IPF), avoiding immortal time bias
Forest plot of the association between the use of anti-acid therapy and all-cause mortality in studies with immortal time bias and with no time-related bias. Pooled estimates were computed using the random effects model. HR: hazard ratio.
Description and example of immortal time bias
An example of a study affected by immortal time bias in evaluating the effectiveness of PPIs in IPF was a retrospective cohort of 215 IPF patients who entered the cohort at the time of their initial clinic visit [18]. Exclusion criteria included loss to follow-up before 12 months and PPI therapy duration of <12 months for reasons other than death or lung transplantation. Patients were categorised as PPI users (PPI use ≥12 months, n=130) and non-users (no PPI prescription, n=85) and followed until lung transplantation or death, which occurred in 77 PPI users and 63 non-users. Cox proportional hazards models were used to estimate the crude and adjusted HRs. Use of PPIs was associated with a significant protective effect (adjusted HR 0.58, 95% CI 0.39–0.87) and an increase in survival time (median survival of 3.4 versus 2 years).
In this study, immortal time bias was introduced by the definition of exposure: IPF patients were classified as exposed if they used PPIs for ≥12 months during follow-up. The period from the start of follow-up until the 12-month exposure definition was fulfilled is immortal (figure 2). Indeed, while the 85 non-users were at risk of death immediately after cohort entry, the 130 PPI users had to survive ≥12 months after cohort entry. This immortal period led to a survival advantage among PPI users, resulting in the inaccurate conclusion that PPI users lived 1.4 years longer than non-users. Another limitation of the study was the exclusion of patients who used PPIs for <12 months, who should be included and considered as unexposed. These patients should contribute to the person-time and events necessary to estimate death rates among unexposed patients with IPF.
Illustration of immortal time bias using the proton pump inhibitor (PPI) exposure definition in the cohort study by Ghebremariam et al. [18]. Idiopathic pulmonary fibrosis (IPF) patients classified as exposed had to receive PPIs for ≥12 months during follow-up. The time between cohort entry (time 0) until the definition of exposure was met is immortal, because the patient had to survive this period to be classified as exposed. For example, user A initiated PPIs during follow-up. The span between time 0 until the initiation of PPIs (grey band) is immortal, but then the patient also had to survive the 12 months of PPI use (black band) before an event could occur, introducing two types of misclassified immortal time.
To illustrate how immortal time bias may affect effect estimates, we used crude data from the study conducted by Ghebremariam et al. [18]. As the data necessary to quantify the bias are not provided in the article, for the purpose of illustration, we approximated hazard rates based on the reported median survival, which generated a total follow-up of 377 person-years for PPI-use and 182 person-years for non-use. Furthermore, we assumed that all 130 PPI users had the minimum period of 1 year of immortal time due to the exposure definition. This would amount to 130 immortal person-years in which PPI users were not at risk of death and which should have been classified as unexposed person-time. Thus, 35% (130 out of 377 person-years) of total follow-up time among PPI users were immortal. Based on the median survival time and the number of deaths in each group the rates of death are 20.4 per 100 person-years for PPI use and 34.7 per 100 person-years for non-use, resulting in a crude HR of 0.59, which is very similar to the reported crude HR of 0.60. Accounting for the immortal person-time by adding this immortal person-time to person-time at risk in non-users produced a corrected crude HR of 1.55 (table 3), which is probably overestimated, as the number of deaths among unexposed patients who used PPIs for <12 months is unknown. This simple illustration shows how incorrectly classified person-time at risk due to immortal time can lead to beneficial effect estimates and that person-time rather than patients should be analysed. There are more sophisticated approaches to estimate adjusted HRs that appropriately account for time-varying exposures, such as extended Cox models [31].
An illustration of crude hazard ratios for death associated with proton pump inhibitor (PPI) use before and after correcting for immortal time bias
Other studies with immortal time bias
Other cohort studies also introduced immortal time bias by using an exposure definition that required a minimum duration of PPI use after cohort entry to define exposure. The Stanford study [30] included 132 patients with IPF. Patients were categorised as PPI users (PPI use ≥12 months, n=87) and non-users (no PPI prescription) and followed until lung transplantation or death. Mean survival in the PPI group was 3.4 years compared to 1.9 years in non-users, leading to the conclusion that PPIs improve survival in patients with IPF (adjusted HR 0.56, approximated 95% CI 0.34–92).
The South Korean study (n=786) introduced immortal time bias by using different cut-off durations to categorise exposure to PPIs. Cox proportional hazards models were used to evaluate whether PPIs were associated with IPF-related mortality. The authors concluded that using PPIs for >4 months may have a protective effect (adjusted HR 0.51, 95% CI 0.21–1.22) [22]. However, patients were by definition immortal during the 4 months after the first prescription. Corresponding Kaplan–Meier curves showed that there were no events in the exposed group in the first 4 months. If the cut-off duration was set at 2 or 3 months, there were no differences in mortality (2 months crude HR 0.87, 95% CI 0.54–1.41). This demonstrates how immortal time bias was introduced by exposure categories based on the prolonged duration of PPI use.
Finally, the Chinese cohort study [24] identified 69 patients with IPF and compared 34 anti-acid therapy users (≥6 months) to 35 non-users (<6 months or none). Median survival in anti-acid therapy users was 31 months compared to 23 months in non-users (adjusted HR 0.23, 95% CI 0.12–0.44). Again, immortal time bias was introduced by the exposure definition requiring a minimum duration of anti-acid therapy use. In addition to immortal time bias, the adjusted Cox model using stepwise regression led to an even more biased estimate, as this method selected variables that may not have been adequate for confounder adjustment [32].
Studies with no time-related bias
Five observational studies of the association between anti-acid therapy and mortality did not use exposure definitions or analyses that introduced immortal time bias (table 2). These included two cohort studies and three studies that analysed secondary data from RCTs. The Swedish cohort study [19] (n=462) used a time-dependent exposure definition allowing patients to move from an unexposed to an exposed status, thus avoiding immortal time bias. Drug exposure was assessed during each quarter based on dispensed prescriptions, and drug effects on mortality were estimated in patients with IPF who initiated long-term oxygen therapy. During the follow-up period (median survival 6.7 months), 329 (71%) IPF patients died. This study found no association between use of anti-acid treatment and death (HR 1.12, 95% CI 0.87–1.42).
The second cohort study from Germany [20] (n=272, 2004–2012) assessed use of PPIs only once at baseline (first clinic visit) using an intention-to-treat approach. 171 (63%) patients died during follow-up. Use of PPIs at baseline did not show any differences in median survival (48 months in users versus 42 months in non-users, approximated HR 0.88, 95% CI 0.42–1.83).
Lee et al. [14] retrospectively analysed data from 242 IPF patients randomised to placebo groups of three RCTs of other treatments for IPF. Using an intention-to-treat approach, patients were categorised into anti-acid therapy users (n=124) or non-users (n=118) based on reported use at baseline visit before randomisation. Data on self-reported anti-acid therapy use was recorded at each follow-up visit, but not included in the mortality analysis. All-cause mortality was assessed at 30 weeks. After confounder adjustment, time to all-cause mortality did not differ significantly between the two groups (approximated HR 0.61, 95% CI 0.33–1.14).
Another post hoc analysis was conducted by Kreuter et al. [21]. This study included 624 patients with IPF from the placebo arms of three RCTs evaluating pirfenidone. The authors assessed the effect of anti-acid therapy use on all-cause mortality at 52 weeks using an intention-to-treat approach. 291 patients received anti-acid therapy at baseline. Drug use was documented subsequently during the trials (25% of patients started anti-acid therapy after baseline), but was not implemented in the mortality analyses. After confounder adjustment, use of anti-acid therapy at baseline did not improve all-cause mortality (HR 0.80, 95% CI 0.30–1.70). The authors repeated the analysis with patients randomised to the pirfenidone treatment arms (n=623), and again did not find an association between all-cause mortality in anti-acid therapy use (HR 0.80, 95% CI 0.30–2.50) [23].
Unclassifiable study
In 2011, the first published study on the effectiveness of anti-acid therapy involved a retrospective cohort of 204 patients with IPF, with cohort entry taken as the date of the first clinic visit, defined as the date of diagnosis [13]. Patients were followed until lung transplantation or death, which occurred in 97 (48%) patients. Information on treatment was collected in a “prospective” manner, although the authors state that exposure was classified into anti-acid therapy users (n=96, 47%) and non-users at the time of diagnosis. The median survival time for those using anti-acid therapy was 65.5 months compared with 29.9 months for non-users. The resulting adjusted HR comparing anti-acid therapy use to non-use at IPF diagnosis was 0.47 (95% CI 0.24–0.93), suggesting a highly protective effect of anti-acid therapy. However, the article also reported that the median follow-up time between the two groups was similar (∼22 months). This discrepancy between the median follow-up time (22 months) and median survival time (36 months) of the cohort is unexpected and raises the question of whether immortal time was not introduced by classifying patients who ever used anti-acid therapy during follow-up as exposed (figure 3). Based on the available information and inconsistencies in the reported data, it is unclear whether the protective effect is due to immortal time bias, rendering this study inconclusive.
Example of immortal time bias in patients with idiopathic pulmonary fibrosis (IPF) exposed to anti-acid therapy who died from any cause. Anti-acid therapy users were ever-users, i.e. they used anti-acid therapy at diagnosis and throughout follow-up (patient B) or they were non-users at IPF diagnosis and initiated anti-acid therapy during follow-up (patient A). The time between cohort entry and the first anti-acid prescription is immortal (grey band), because the subject must survive to receive this first anti-acid prescription. Additionally, this time is misclassified as exposed to anti-acid therapy when it is in fact unexposed, leading to immortal time bias.
Discussion
Between 2011 and 2017, 10 observational studies evaluated the effectiveness of anti-acid therapies among patients with IPF on mortality. Our review did not assess other outcomes and excluded studies that evaluated outcomes such as lung function decline, but did not report mortality data [33]. Five studies, including the first published in 2011, reported highly beneficial effects of anti-acid therapies in IPF, with significant reductions in mortality, whereas the remaining studies did not find an association. Pooling the effect estimates of all included studies produced a HR of 0.63 (95% CI 0.45–0.90). However, when we stratified the analyses by the presence of immortal time bias, we found that the direction of the overall pooled HR was driven by the studies with immortal time bias. This highlights the need to identify and exclude studies affected by immortal time bias from such pooled analyses to avoid biased results.
We showed that four of the cohort studies reporting significant reductions in mortality associated with anti-acid therapy were affected by immortal time bias. This bias was introduced by a required minimum duration of anti-acid therapy use to define exposure status. The exposed group, by design, had to survive that period and was, thus, immortal, whereas the unexposed group was at risk of death immediately after cohort entry. The immortal-biased cohort studies led to exaggerated results with a highly significant 54% reduction in all-cause mortality (95% CI 31–70%), which probably motivated the initiation of a pilot RCT that currently investigates the effectiveness of the PPI omeprazole in patients with IPF in the UK (www.clinicaltrials.gov identifier NCT02085018) [34]. Immortal time bias can be avoided using appropriate study designs and analyses [27].
Five studies, including three retrospective analyses using data from RCTs, did not find any association between anti-acid therapy use and mortality. One of these studies used a time-dependent analysis to avoid immortal time bias, whereas the other studies used an intention-to-treat approach from cohort entry onwards. Although free from immortal time bias, these studies have several other limitations, including residual confounding due to the observational nature of the studies. First, the Swedish cohort study using an appropriate statistical analysis only enrolled IPF patients with advanced disease who required supplemental oxygen, and was thus not generalisable to the general IPF population. Second, patient populations enrolled in RCTs represent a homogenous patient population which is different from the heterogeneous “real-world” IPF population. Third, the RCTs were not designed to investigate mortality as the primary outcome. Thus, with an estimated median survival of 2–5 years after diagnosis, 1-year RCTs are probably too short to observe any differences in mortality associated with use of anti-acid therapy. Fourth, the intention-to-treat approach uses only one single-exposure measurement and assumes that patients adhere to treatment until the end of follow-up. If anti-acid information on treatment is recorded during follow-up but not analysed, this leads to exposure misclassification. To better understand treatment patterns and effects it is necessary to additionally assess anti-acid therapy use in a time-dependent manner during a sufficient period of follow-up [35, 36].
Conclusions
Immortal time bias is evident in the four cohort studies that suggested that anti-acid therapy in IPF is highly effective at reducing mortality. Since the studies unaffected by this bias did not find that anti-acid therapy is effective, it is imperative to reanalyse the data from the four studies affected by immortal time bias using proper methods that avoid this bias. In addition to ongoing randomised trials, new observational studies that use proper methods of design and analysis to avoid such time-related biases are now needed to assess the effectiveness of PPIs in IPF in the real-world setting of clinical practice. Until such further work is undertaken, the scientific evidence of the potential beneficial effects of anti-acid therapy on survival remains uncertain.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
EMBASE Search Strategy (July 3, 2017) ERJ-00376-2018_supp1
Flow chart of the study selection for the methodological review ERJ-00376-2018_supp2
Footnotes
This article has supplementary material available from erj.ersjournals.com
Author contributions: T. Tran served as the principal author. Both authors contributed substantially to the study design and interpretation, and the writing of the manuscript. Both authors approved the final version of the manuscript.
Conflict of interest: None declared.
Support statement: S. Suissa is the recipient of the James McGill Chair award. The institute was not involved in the study design, data analysis or interpretation, or the preparation of the manuscript.
This article has supplementary material available from erj.ersjournals.com
- Received December 14, 2017.
- Accepted April 22, 2018.
- Copyright ©ERS 2018
References
