The last decade has seen important progress in the clinical investigation of idiopathic pulmonary fibrosis (IPF). Basic insights into mechanisms of fibroproliferation have been translated into novel investigational agents, networks of clinical centres capable of enrolling hundreds of patients in research studies have been developed, and multiple high-quality treatment trials have been successfully completed and published 1–5. These are all major accomplishments that unarguably move the field forward. But to patients and providers faced with this devastating diagnosis, there remain distressingly few management options, and no definitive therapy has yet been identified.
It is into this environment of hope and frustration that the latest clinical trial of pirfenidone arrives 6. It has been 11 years since a small trial of pirfenidone therapy in patients with IPF suggested a possible benefit 7, and substantial excitement surrounds the publication of these data. Do the results of Taniguchi et al. 6 justify the use of pirfenidone in patients with IPF? Ultimately, this is for the academic and regulatory communities to decide. This editorial aims to inform the discussion by summarising the results and limitations of this important trial.
275 subjects were blindly randomised to high-dose pirfenidone, low-dose pirfenidone or placebo. There was a significant reduction in the rate of decline in vital capacity favouring pirfenidone (-0.09 L versus -0.16 L; p = 0.04). Progression-free survival (defined as change in vital capacity of >10% from baseline or death) also showed a significant difference between groups, favouring pirfenidone. These results are compelling and suggest that pirfenidone slows the progression of disease. However, important study design issues challenge the validity of this conclusion. Three areas are of particular concern: 1) the change in primary end-point during the course of the trial; 2) the handling of missing data; and 3) the absence of patient-reported outcomes.
CHANGE IN PRIMARY END-POINT
Primary end-points should reflect the hypothesis being tested and provide sufficient evidence to fully characterise clinically the effect of an intervention 8. Treatment trials aimed at slowing the progression of IPF have generally chosen change in pulmonary function parameters over a set period of time as the primary end-point, as this has been shown to be a surrogate for survival time 9, 10. Time-to-event end-points have also been used to directly capture a wider range of clinically relevant outcomes (e.g. mortality, categorical changes in pulmonary function and acute respiratory worsening). There are merits to both approaches, and often both are included in the overall study design.
The original primary end-point for this trial was change in lowest oxygen saturation during 6 min steady-state exercise test. During the course of this multi-year trial, the academic community's views on appropriate primary end-points in IPF evolved, and this led the investigators to consider making a change to the primary end-point. Indeed, there is ample justification given in the discussion section for this change. However, additional communication from the authors states that the decision to change end-points involved members of the data safety and monitoring board who recommended this change after a discussion of blinded interim comparative data (i.e. they had knowledge of whether there were significant differences between study groups with respect to the primary and secondary end-points). If a change to the primary end-point of an ongoing clinical trial is made with knowledge of interim comparative data (even if it is blinded), the credibility and integrity of the trial is compromised 11. It is simply impossible for readers to assess the impact of this knowledge on the decision.
MISSING DATA
Missing data is a reality in all clinical trials, particularly those with primary end-points that require longitudinal measurement. The current trial reflects this, with incomplete data obtained for a full one-third of subjects. Unfortunately, no statistical method for handling missing data is without potential for error. Last observation carried forward analysis was used in this instance, and is a commonly employed approach. Importantly, this technique may underestimate the true variability of missing data and inflate the type 1 error rate (i.e. the rate of finding a statistically significant difference when a difference does not truly exist) 12. In studies such as this one, with a small treatment effect and marginal p-value, significance may hinge on the method of statistical adjustment used. To address this issue, analysis using other statistical approaches, such as multiple imputation (in which the observed data are used to construct a predictive distribution from which the missing values are then randomly selected) or “complete case” analysis, can assess the robustness of such findings. Indeed, looking at one such measure, the crude data on change in vital capacity over 52 weeks, it appears much of the difference between groups seen using the last observation carried forward adjustment is no longer present.
ABSENCE OF PATIENT-REPORTED OUTCOMES
Dyspnoea and quality of life are increasingly recognised as important outcome measures in clinical trials as they measure distinct, directly relevant aspects of morbidity. Several metrics have been studied in IPF, and longitudinal changes appear to correlate with important objective measures of disease progression 9, 13. Quality of life is particularly relevant in trials of novel therapeutics with significant side-effects; it is possible that pirfenidone could slow the rate of decline in vital capacity but worsen quality of life through side-effects such as photosensitivity and anorexia. The absence of data on patient-reported outcomes limits the reader's ability to fully weigh up the benefits and risks of therapy.
Taniguchi et al. 6 summarise their results appropriately in the text: “Treatment with pirfenidone may decrease the rate of decline in vital capacity and may increase the progression-free survival time over 52 weeks. Additional studies are needed to confirm these findings”. Two additional trials of pirfenidone have in fact been completed, and the preliminary results have been presented to the scientific community 14. Peer-review of the complete results of these additional trials will no doubt add important information to the pirfenidone discussion, and may allow more sophisticated statistical approaches, such as meta-analysis, to provide a reliable estimate of pirfenidone's treatment effect.
Statement of interest
A statement of interest for H. Collard can be found at www.erj.ersjournals.com/misc/statement.dtl
- © ERS Journals Ltd