Misleading communication of risk

In 1996 a review of mammography screening reported in its abstract a 24% reduction of breast cancer mortality1; a review in 2002 claimed a 21% reduction.2 Accordingly, health pamphlets, websites, and invitations broadcast a 20% (or 25%) benefit.3 Did the public know that this impressive number corresponds to a reduction from about five to four in every 1000 women, that is, 0.1%? The answer is, no. In a representative quota sample in nine European countries, 92% of about 5000 women overestimated the benefit 10-fold, 100-fold, and more, or they did not know.4 For example, 27% of women in the United Kingdom believed that out of every 1000 women who were screened, 200 fewer would die of breast cancer. But it is not only patients who are misled. When asked what the “25% mortality reduction from breast cancer” means, 31% of 150 gynaecologists answered that for every 1000 women who were screened, 25 or 250 fewer would die⇓.3

• Download figure
• Open in new tab
• Download powerpoint

David Page Photography/Alamy

In 1995, the UK Committee on Safety of Medicines issued a warning that third generation oral contraceptive pills increased the risk of potentially life threatening thrombosis twofold. The news provoked great anxiety, and many women stopped taking the pill, which led to unwanted pregnancies and abortions––some 13 000 additional abortions in the next year in England and Wales—and an extra £46m (€55m; $71m) in costs for the NHS.5 Yet how big was the twofold risk? The studies revealed that for every 7000 women who took the earlier, second generation pills, one had a thrombosis, and this number increased to two in women who took third generation pills. The problem of misleading reporting has not gone away. In 2009, the BMJ published two articles on oral contraceptives and thrombosis; one made the absolute numbers transparent in the abstract,6 whereas the other reported that “oral contraceptives increased the risk of venous thrombosis fivefold.”7

These two examples illustrate a general point. Absolute risks (reductions and increases), such as from one to two in 7000, are transparent, while relative risks such as “twofold” provide incomplete and misleading risk information.3 8 Relative risks do not inform about the baseline risk—for example, whether twofold means from one to two or from 50 to 100 in 7000—and without this information, people overestimate benefits or harms.3 9 In the case of the pill scare, the losers were women, particularly adolescent girls, taxpayers, and the drug industry. Reporting relative risks without baseline risk is practised not only by journalists because big numbers make better headlines or by health organisations because they increase screening participation rates. The source seems to be medical journals, from which figures spread to press releases, health pamphlets, and the media.

An analysis of the articles published in the Annals of Internal Medicine, BMJ, JAMA, Journal of the National Cancer Institute, Lancet, and the New England Journal of Medicine, 2003-4, showed that 68% (150/222) failed to report the underlying absolute risks in the abstract. Among those, about half did report the absolute risks elsewhere in the article, but the other half did not.10 Similarly, an analysis of 119 systematic reviews in BMJ, JAMA, and Lancet from 2004 to 2006 showed that every second article discussed only relative risks or odds ratios.11

Conveying relative risks without baseline risk is the first “sin” against transparent reporting. The second is mismatched framing—reporting benefits, such as relative risk reductions, in big numbers and harms, such as absolute risk increases, in small numbers.3 If we use the example of a treatment that reduces the probability of getting disease A from 10 to five in 1000, whereas it increases the risk of disease B from five to 10 in 1000, authors who use mismatched framing would report the benefit as a 50% risk reduction and the harm as an increase of five in 1000; that is, 0.5%. Medical journals permit mismatched framing. One in three articles in the BMJ, JAMA, and Lancet from 2004 to 2006 used mismatched framing when both benefits and harms were reported.11

Have editors since stopped non-transparent reporting? To check the current situation, we examined the abstracts of all free accessible research articles published in the BMJ in 2009 that reported drug interventions. Of the 37 articles identified, 16 failed to report the underlying absolute numbers for the reported relative risk measures in the abstract. Among these, 14 reported the absolute risks elsewhere in the article, but two did not report them anywhere. Moreover, absolute risks or the number needed to treat (NNT) were more often reported for harms (10/16) than for benefits (14/27).

How can those who are responsible for accurate communication of risk do better? And who should be monitoring them to ensure that they do? Steps can be taken to improve the transparency of risk communication.12 Firstly, editors should enforce transparent reporting in journal abstracts: no mismatched framing, no relative risks without baseline risks, and always give absolute numbers such as absolute risks and NNT.

Secondly, institutions that subscribe to medical journals could give journal publishers two years to implement the first measure and, if publishers do not comply, cancel their subscriptions.

Thirdly, writers of guidelines, such as the CONSORT statement, should stipulate transparent reporting of benefit and harms in abstracts.