Number needed to treat: enigmatic results for exacerbations in COPD

From the author:

S. Lettis and O. Keene raise questions about my paper on some misleading uses of the number needed to treat (NNT) for study outcomes such as chronic obstructive pulmonary disease (COPD) exacerbations. Certainly, when dealing with recurrent events such as exacerbations, it is statistically more informative to analyse all events with tools such as incidence rates, rate ratios and rate differences. However, some critical assumptions about the rates are essential to obtain valid estimates of these measures and, consequently, a valid estimate of the NNT.

Indeed, the underlying rates, estimated by the total number of exacerbations divided by the total time of follow-up, provide a mean number of exacerbations per year, such as two exacerbations per year over, for example, the 3-year follow-up of a trial. However, the statistics behind the estimation of this rate, be it the Poisson or negative binomial distribution, require that the rate be constant over the 3-year follow-up, which entails that the rate of two per year applies equally in the first, second and third year of follow-up. If not, rate differences may differ across the follow-up time and the resulting event-based NNT, computed by inverting the overall rate differences, will simply be incorrect.

My paper used the example of the 3-year Towards a Revolution in COPD Health (TORCH) trial to show why the reported NNT of 4 to prevent one exacerbation in 1 year is misleading [1]. A reader of the TORCH paper would logically understand this statement to mean that treatment of four patients for 1 year (any 1 year: the first, the last, etc.) leads to the prevention of one exacerbation. However, this is not the case. Indeed, it was quite arbitrary that the TORCH trial was a 3-year rather than, say, a 1-year trial. My paper shows that had TORCH been conducted as a 1-year trial, it would have reported a NNT of 0.5 to prevent one exacerbation in 1 year, not the NNT of 4. Moreover, the paper shows that the NNT to prevent one exacerbation with 6 months of treatment is 0.5 patients during the first 6 months in TORCH, while it is 20 patients during the last 6 months.

S. Lettis and O. Keene agree that the exacerbation rate clearly changes over time in TORCH and in other such COPD trials. They argue that, because of differential discontinuation, the patient characteristics are no longer the same after the first year of the TORCH trial and, therefore, the NNT for years 2 and 3 are incorrect, with only the NNT for the first year being correct. This is certainly a valid point. However, this argument could have been extended to equally contend that, for the very same reasons, the overall NNT based on pooling the 3 years together must therefore also be incorrect as the patients are no longer comparable over time.

Such deliberations illustrate well the problems with the “event-based” NNT, which was precisely the point of my paper, namely to raise awareness that using this event-based approach to compute an NNT of 4 to prevent one exacerbation in 1 year can be misleading if the underlying assumptions are not satisfied. For the same reasons, the NNTs of 0.5 and 20 patients during the first and last 6 months in TORCH, respectively, are also incorrect. However, a robust solution is simply to use the time to the first exacerbation to compute the NNT for 1 year to prevent an exacerbation, a clearer and more valid approach. The NNT can be easily estimated from the Kaplan–Meier curve of the cumulative incidence of the first exacerbation and computed for any treatment duration [2, 3].

S. Lettis and O. Keene also bring up the issue of using the NNT in a benefit–risk context, such as exacerbations versus pneumonias in COPD. Besides the arguments against the lax use of the event-based NNT described above, even the proper use of the NNT in this context requires additional precaution. For example, one must be careful when using the NNT in comparing milder exacerbations mostly treated as outpatients with more severe pneumonias that generally require hospitalisation in this age group [4]. A more rigorous use of this approach would be to compare like with like, namely to compare COPD exacerbations with pneumonias that are both treated as outpatients, that both require hospitalisation or that are both fatal.

Finally, I thank the authors for noting the mislabelling of columns in the tables. In all, while the NNT can be a useful measure of impact when computed properly, it can also result in misleading values when underlying statistical assumptions are not satisfied, such as the reported NNT of 4 to prevent one exacerbation in 1 year in the 3-year TORCH trial. Use with caution.