Idiopathic pulmonary fibrosis (IPF) is a chronic progressive lung disease with a median survival of approximately 3 years from diagnosis. Accurate prediction of mortality is important in IPF, as it helps determine the urgency of lung transplantation, guides other management and end-of-life decisions, and facilitates enrichment of clinical trial populations. Although most patients with IPF die from progressive respiratory failure [1], predicting outcomes is challenging due to the heterogeneity of disease progression.
In this issue of the European Respiratory Journal, du Bois et al. [2] use the INSPIRE (International Study of Survival Outcomes in Idiopathic Pulmonary Fibrosis with Interferon-γ-1b) dataset to describe a new prediction model that estimates mortality in IPF. The INSPIRE study was a multicentre double-blind randomised trial comparing interferon-γ-1b to placebo in 826 patients with mild to moderate IPF [3]. This trial did not meet its primary end-point but has generated high-quality data that continue to provide valuable insight into the progression of IPF. du Bois et al. [4] have previously used the INSPIRE data to show that age, baseline forced vital capacity (FVC), 24-week change in FVC, and recent respiratory hospitalisation independently predict 1-year mortality in patients with IPF [4]. These authors have now extended their findings by showing that mortality prediction is improved by adding both baseline and 24-week change in 6-min walking distance (6MWD) to this multivariate model [2]. This new study is a noteworthy addition to the literature, providing important data on the prediction of mortality in IPF and suggesting potential options for clinical trial end-points.
Defining mortality risk in IPF
The model developed by du Bois et al. [2] accurately estimates 1-year mortality in patients with IPF with a C-statistic of 0.80. The C-statistic, or C-index, is a measure of how well a model predicts the risk of an outcome (e.g. 1-year mortality) in any two individuals randomly selected from the cohort of interest. A C-statistic of 1.0 indicates that the model appropriately identifies the individual at higher risk of the outcome in every paired comparison. A C-statistic above 0.70 is generally considered acceptable, and a C-statistic of 0.50 indicates that the prediction model performs no better than chance. The C-statistic of 0.80 in this study compares favourably with models that are in routine clinical use in other fields and other validated models in IPF [5]. It is therefore tempting to use this new model in the clinical setting; however, there are some questions that require further study.
The inclusion of both 6MWD and FVC in this model raises the question as to whether FVC and 6MWD are both markers of IPF severity or whether 6MWD instead represents functional limitation that is not directly related to IPF. FVC does not fully account for the severity of radiological fibrosis [6], and the 6MWD could therefore reflect a component of IPF severity that is not captured by the FVC. An additional possibility is that 6MWD measures the prognostic importance of concurrent emphysema or pulmonary hypertension. Although the impact of emphysema is unclear [7], abnormal pulmonary haemodynamics predict increased mortality in IPF and the presence of pulmonary hypertension could account for the independent relationship of 6MWD with mortality [8]. Finally, 6MWD is influenced by extrapulmonary disease and a reduced 6MWD could indicate the presence of other comorbidities or frailty that increase mortality independent of IPF severity. The absence of 6MWD or similar functional variables in other IPF risk prediction models may account for differences in their performance.
There are some limitations that impact the use of this new risk prediction model in the clinical setting. The most important limitation is the lack of validation. Pre-specification of the study objective and hypothesis protects against the possibility of overfitting to some degree but proper external validation remains essential. The model also requires longitudinal data that are not always available in clinical practice, and generalisability beyond the INSPIRE population is unknown (e.g. mild to moderate IPF with limited comorbidities). Although it is a relatively simple test, performance and quality control of the 6MWD are difficult to standardise compared to FVC. This further compromises the generalisability of data that were obtained in clinical trials that have strict study procedures. Finally, this model provides estimates of 1-year mortality; however, longer-term estimates are desirable in IPF, particularly if the model is intended to guide referral or listing for lung transplantation. Despite these issues, the model is highly accurate and thus deserves further study for validation, to demonstrate its accuracy in a broader IPF population, and to determine its ability to predict longer-term outcomes.
Defining end-points in IPF
Mortality is the most clinically relevant and important end-point in IPF. However, using death as a primary outcome dramatically increases the cost of clinical trials by requiring a larger sample size and longer study duration [9]. Alternative end-points are therefore highly sought-after. In their study, du Bois et al. [2] propose that a composite end-point of change in FVC or 6MWD may be an appropriate outcome for IPF clinical trials. FVC and change in FVC consistently and independently predict mortality in IPF [4, 5, 10–14], and du Bois et al. [2] have now shown similar findings for 6MWD. These data support the use of these variables as surrogates of mortality. However, there is recent debate regarding the need for additional validation [15, 16].
A valid surrogate end-point should be on a causative pathway between the biological actions of a specific therapy and improvement in an outcome that is important to patients. For example, FVC would be a valid surrogate of death if a therapy was shown to improve mortality through its effects on lung fibrosis and ventilatory restriction. Demonstrating this causal relationship is clearly problematic in diseases such as IPF that have no established therapies that alter mortality. Although the need for this high standard of proof is contested [15], both FVC and 6MWD require this final validation step before universal acceptance as valid surrogate end-points [16]. Unlike FVC, however, the 6MWD has the advantage of also being important to patients because it is closely related to physical function and independence. This feature would allow investigators and regulatory bodies to consider the 6MWD a patient-important outcome measure itself, circumventing some of the criticism directed at surrogate end-points.
Whether considered a surrogate endpoint or a patient-important outcome, the 6MWD also has limitations that affect its use in testing potential IPF pharmacotherapies. The key weakness of the 6MWD in this context is that it is multidimensional, being partly influenced by ventilatory mechanics, gas exchange, cardiovascular performance and musculoskeletal function. Pharmacotherapies of IPF are unlikely to target all dimensions of the 6MWD and the lack of benefit in the unaffected dimensions increases noise and diminishes power to detect between-group differences. Additional treatments or lifestyle changes (e.g. exercise training, weight loss) could improve these other dimensions of the 6MWD during the course of a clinical trial and further obscure potential benefits of the study medication. Despite some advantages of the 6MWD, FVC is likely to remain the most common primary end-point in IPF trials because of these and other concerns.
Remaining questions
Multiple groups have recently developed risk prediction models in IPF [4, 5, 17, 18]. However, there are remaining questions that have limited their application to routine clinical practice. For example, these models are considered acceptable based on their C-statistics, which range from 0.7 to 0.8, but is this accurate enough to reliably estimate prognosis and influence the decision to start expensive or potentially toxic IPF therapies? Alternatively, does this same level of accuracy justify use of these models to alter critical decisions such as listing for lung transplant? From a research perspective, can these models be used to enrich or stratify clinical trial populations by identifying those at higher risk of an outcome of interest? And more importantly, are there now enough data to confirm beyond reasonable doubt that FVC and 6MWD are valid surrogate end-points for clinical trials?
Based on the available evidence from this and previous studies, it is likely that it is reasonable to use these models to provide patients with general estimates of mortality, facilitate enrichment of clinical trial populations, and stratify clinical trial participants by category of risk. For all of these purposes, however, it is essential that patients, physicians and researchers understand the limitations of these models and the uncertainty in prognostic estimates when applied at an individual level. Further incorporation into clinical and research settings will require additional research to determine the role of these models in guiding management decisions. Studies are also required to determine which of the many prediction models provides the optimal balance between model performance and practicality.
The evaluation of potential IPF therapies has been impeded by the challenges in predicting mortality and the lack of clear guidance on appropriate end-points. Many recent publications have begun to address these issues and cumulatively these studies are a major step forward in identifying and testing effective treatments for IPF. The study by du Bois et al. [2] is a significant addition to this literature and provides valuable new information in these controversial but important areas of IPF research.
Footnotes
Conflict of interest: Disclosures can be found alongside the online version of this article at www.erj.ersjournals.com
- Received February 2, 2014.
- Accepted February 10, 2014.
- ©ERS 2014
References
