Statistical compared to clinical significance and the risk of misattribution

To the Editor:

We read with interest the observational analysis by Vozoris et al. [1] of the putative association between the prescription of new antidepressants (selective serotonin reuptake inhibitors (SSRIs) or serotonin-noradrenaline reuptake inhibitors (SNRIs)), and hospitalisation and mortality in people with a diagnosis of chronic obstructive pulmonary disease (COPD). In this nonrandomised analysis of administrative health data, people who got a new prescription of these antidepressants faced a worse prognosis including a 26% increase in pneumonia-related mortality and a 20% higher risk of death from all causes [1]. This analysis used similar methodology as previous papers on benzodiazepines and opioids and shares the same important shortcomings [2, 3].

The associations between antidepressant prescription and increased hospitalisation and mortality likely reflect the underlying illness(es) and other unmeasured factors rather than an actual drug effect; the authors acknowledge the general inability of observational studies to establish causative relationships. This analysis, therefore, does not support the authors' conclusions that these findings “should prompt prescribers to consider the potential for increased respiratory-related morbidity and mortality in SSRI/SNRI prescribing decision making” [4].

People who have a clinical encounter which results in a prescription for antidepressants are likely to differ from those who do not in important ways. The severity of underlying illness, in this case both of COPD or anxiety/depression, and the indication for the drug treatment are recognised as key confounders [5], neither of which can be controlled for in this dataset. In addition, data were unavailable on other important factors such as smoking exposure, lung function, alcohol consumption, socioeconomics, symptoms, dietary status and behaviours including fitness and activity. We acknowledge that the authors did what can be done to control for the limited factors available, but excluding people with a reimbursed palliative care consultation in a health system where only 5% of all decedents from COPD have a palliative care consultation [6] is clearly insufficient. The risk of death or hospitalisation is not equal for all people included in this cohort study. The findings of the study, therefore, reflect an association between expected deaths and symptom interventions offered to people with symptomatic life-limiting disease. In light of this, the moderately higher rates of hospitalisation and death in people with treatment is no surprise.

That the findings are related to confounding is supported by the fact that the increased risks disappeared when analysed only in people with diagnosed psychiatric illness. The reason for this is more likely better adjustment for underlying illness (confounding) than a biological rationale. There was also no dose–response relationship. Of note, the large number of participants in the study increases precision but does not affect confounding, risking yielding a more precise estimate of a spurious association [7].

We are not suggesting that observational studies of potential drug effects should be avoided. However, crucial factors must be considered in the design and interpretation of these studies such as the pre-likelihood of a true effect (in line with a Baysian approach), the size of the effects, and the ability of the study to control for key confounders such as severity of the illness(es) related to the treatment and outcomes (both benefits and harms). The risk of confounding in observational studies is high when the treatment is strongly related to underlying illnesses which, in turn, have strong relationships with the studied outcomes. Using a similar design as in the present study, one may find substantial negative associations between treatment of any serious and distressing symptom and clinical outcomes, e.g. for bronchodilators for airflow limitation, for starting pulmonary rehabilitation, for use of mobility aids and even for smoking cessation. Confounding is likely for all palliative interventions given to relieve symptoms in the setting of advancing disease.

We agree with the authors' statement that “absolute adverse event rates of our positive outcomes are relatively small, and therefore may not be clinically significant or they may not have remained positive had additional covariates been controlled for.” The clinician (as well as other people) should remain sceptical, interpret the findings in the light of pre-likelihood of true effect and the risk of residual confounding, and await further evaluation of net-benefit using randomised data where possible before changing clinical practice.