The INPULSIS trials of idiopathic pulmonary fibrosis treatment: explaining further discrepancies on exacerbations

To the Editor:

In a recent paper on the INPULSIS trials, two duplicate 1-year randomised controlled trials evaluating nintedanib in the treatment of idiopathic pulmonary fibrosis (IPF) [1], we provided the methodological explanation for the apparent inconsistent results arising from the two different definitions of exacerbations used in those studies [2]. These trials reported vastly different findings in their pooled analysis [1]. Indeed, the risk of an investigator-reported acute exacerbation was lower by 36% with nintedanib compared with placebo but not statistically significant (p=0.08), while, after adjudication according to a complex definition involving multiple criteria, the risk was lower by 68% and statistically significant (p=0.001). We explained that such differences in risk reductions and in statistical significance are simply the result of including outcome events that are not actual exacerbations, leading to the phenomenon of treatment effect dilution and false nonsignificant effects [2]. The correct risk reduction of 68% based on the accurate adjudicated events is diluted to a less impressive and now nonsignificant risk reduction of 36% when “false” events were added in the same proportion to both groups.

Since then, we have been asked on several occasions whether this phenomenon can perhaps also explain another apparent anomaly between these two trials, and a challenge in interpretation, namely the “mixed results in terms of … exacerbations” [3]. Indeed, the effects of treatment on investigator-reported acute exacerbations between the INPULSIS-1 and INPULSIS-2 trials were also heterogeneous [1]. In the INPULSIS-1 trial, the risk of an investigator-reported acute exacerbation with nintedanib was no different to placebo and was not statistically significant (hazard ratio 1.15, 95% CI 0.54–2.42; p=0.67), whereas in INPULSIS-2, the risk was lower with nintedanib and statistically significant (hazard ratio 0.38, 95% CI 0.19–0.77; p=0.005). In discussing these vastly different results, the authors commented that “exacerbations are relatively rare events in patients with idiopathic pulmonary fibrosis who are in clinical trials and are difficult to assess and categorize, which may explain some of the heterogeneity in our findings” [1]. In this correspondence, we describe how these heterogeneous results can also be largely explained by the phenomenon of treatment effect dilution due to inaccurate definitions of exacerbations.

There were 69 investigator-reported acute exacerbations in the two INPULSIS trials, 33 in INPULSIS-1 and 36 in INPULSIS-2. In the adjudication process, two of these exacerbations had insufficient information and the remaining 67 were classified by the adjudication committee, who determined that 31 were not actual exacerbations. Table 1 displays the analyses of these data before and after adjudication, separately for the two trials. It shows that for investigator-reported acute exacerbations, the rate ratio comparing nintedanib with placebo in INPULSIS-1 is 1.02 (95% CI 0.52–2.00), changing to 0.58 (95% CI 0.21–1.57) when using the adjudicated exacerbations. In INPULSIS-2, the corresponding rate ratio is 0.36 (95% CI 0.19–0.69), changing to 0.21 (95% CI 0.08–0.56) after adjudication. Note that our estimates for the investigator-reported acute exacerbations differ slightly from those reported in the publication since we did not have the raw data and thus had to use rates and rate ratios rather than hazard ratios.

Thus, using the investigator-reported acute exacerbations appears to induce more heterogeneity between the INPULSIS-1 and INPULSIS-2 trials (rate ratios of 1.02 and 0.36, respectively) than after using the adjudicated ones (rate ratios of 0.58 and 0.21, respectively). While there still are some differences between the rate ratios of the two trials after adjudication, these differences are now more compatible with random variability.

Acute exacerbations of IPF, associated with high morbidity and mortality, represent an important outcome to consider in trials of drug effectiveness [4, 5]. However, their determination is complex and presents major challenges to adjudication as they require extensive clinical data from medical records that may often be incomplete [6]. Nevertheless, our analysis highlights the importance of using an accurate measure of these exacerbations, clear of false events, to ensure accurate estimates of the effectiveness of treatment. As such, the importance of an accurate assessment of exacerbations by careful adjudication should not be underestimated.