Detection, verification, and quantification of adverse drug reactions
BMJ 2004; 329 doi: https://doi.org/10.1136/bmj.329.7456.44 (Published 01 July 2004) Cite this as: BMJ 2004;329:44
- Bruno HCh Stricker, professor of pharmacoepidemiology1 (b.stricker{at}erasmusmc.nl),
- Bruce M Psaty, professor of medicine, epidemiology, and health services2
- 1 Pharmacoepidemiology Unit, Department of Epidemiology and Biostatistics, Erasmus Medical Center, PO Box 1738, 3000 DR Rotterdam
- 2 Cardiovascular Health Research Unit, Departments of Medicine and Epidemiology, University of Washington, Seattle, USA
- Correspondence to: B HCh Stricker
- Accepted 30 May 2004
Introduction
Although some will question the use of the term experiment on formal grounds, most experts will likely agree that the widespread marketing of a new drug is in fact a large experiment on a population. This is especially the case when it concerns a novel molecular entity with potentially a new set of clinical experiences. As the marketing of new drugs includes the discovery of adverse effects, the public's health would be best protected by a complementary set of techniques for the detection, verification, and quantification of safety issues. Yet the current approach to this is scattered and disappointing. We discuss why healthcare professionals are not aware of all safety problems of a drug at its introduction and why pharmacoepidemiology should complement the indispensable observational method of case reporting.
Sources and selection criteria
Our review is based on a search of PubMed using the terms adverse effect, adverse reaction, ADR, adverse event, adverse reaction monitoring, pharmacovigilance, cohort study, and case-control study. This yielded several thousand papers from which we excluded individual case reports and case series. From a list of cohort and case-control studies, we picked some recent examples of pharmacoepidemiological studies of adverse events with databases. We used the references from 50 of the most recent reviews, to gather the most important papers on this subject.
Credit: SUE SHARPLES
Limitations of clinical trials
Before drugs are marketed, they are extensively tested in animals and in clinical trials in humans. These tests tell much about the drug's efficacy but for several reasons relatively little about safety (box).
Good economical reasons preclude an endless quest for research before drugs are registered. Such research would make the development of drugs expensive, a price that ultimately is paid by the consumer.1 More effort therefore should be put into researching the safety of drugs after marketing, especially in the evidence-free zone when drugs are first launched.2
Protecting society against the adverse effects of drugs requires early detection, valid verification, and quantification, which include assessing the frequency and severity of adverse events, dose relations, the time course, and susceptibility factors.3
In daily practice many signals of a potential adverse event are not followed by a systematic process of verification and quantification. One reason for this may be the transitional period between the signal of a problem and its confirmation. At this point every sensible professional response is open to discussion. If the problem is highlighted by a single case report, attention may fade; even if similar signals are produced by other case reports, the likelihood of a causal relation is often just debated unless more evidence emerges. Subsequent well designed epidemiological studies may provide additional verification. They also fulfil the need for quantification of the adverse effect. Whether such studies are performed usually depends on the initiatives of scientific groups with an interest in the topic. Another reason why safety issues may not be pursued systematically is related to who has the responsibility. Formally, liability rests with the pharmaceutical company, which acts as marketing authorisation holder, but as it does not have an economic interest in detecting safety issues it may lack incentive to investigate problems. Additionally, many problems related to a drug are caused by class effects, such as extrapyramidal effects due to neuroleptics. Then it is not always possible to blame a particular product or company. Drug authorities should take responsibility to safeguard public health in such a situation.
Adverse drug reactions
Detection
Adverse effects cannot be detected without astute professional observers. Case reports are among the most important tools for observational research.4 All people exposed to a new drug comprise the potential catchment population for adverse effects. In a country such as the United Kingdom, with some 60 million inhabitants, a 1% cumulative exposure to a drug yearly would equate to 600 000 people using the drug at any time during that year. A rare adverse effect with an incidence of 1 in 10 000 might be detected in such a population, particularly when the adverse effect has a low background incidence, making it easily recognised. In the case of phocomelia due to thalidomide, for instance, recognition should have been easy but because of the unfamiliarity with drug safety problems at that time it took several years to identify a causal relation.5 When an adverse effect is non-specific and has an appreciable background incidence, detection is more difficult. Similarly, it may be difficult to detect an adverse effect that is indistinguishable from the disease being treated—for example, arrhythmia as an adverse effect of antiarrhythmics. Such adverse effects may therefore remain unnoticed. One of the most common reasons for withdrawal of drugs from the market is hepatitis, which has a high background incidence making it potentially difficult to determine a causal relation.6
Limitations of most clinical trials in highlighting a drug's safety
Homogeneous populations
Most trials assess relatively healthy patients with only one disease and mostly exclude specific groups such as pregnant women, children, and elderly people
Sample size
Small sample size (up to 1000 patients) reduces the chance of finding rare adverse effects
Limited duration
Trials of short duration preclude the discovery of long term consequences such as cancer
Inability to predict the real world
Drug interactions can be substantial in a population as patients may take drugs concomitantly, a situation that can almost never be predicted from clinical trials
Voluntary reporting
Despite these problems in detecting genuine adverse effects, voluntary reporting of adverse reactions by clinical observers is inexpensive and effective. Basically, there are two systems. The first is a virtual one and relates to all correspondence and short reports in the medical literature. The second consists of national and international adverse drug reaction monitoring centres.
The medical literature
The medical literature is probably by far the most effective system for initial detection because case reports are detailed, assessed for quality by reviewers, mostly independent from commercial incentives, and open to interested parties. The collaborative output of the 20 largest medical journals in highlighting a new problem by a single case report easily outclasses every other system. Fourteen of 18 important adverse drug reactions were shown to be detected and verified by voluntary reporting, mostly through the literature.7 Of 47 anecdotal reports published in 1963 in four major general medical journals, 35 were clearly correct, which seems to be satisfactory.8 Not all published case reports represent genuine adverse reactions as there is always the risk of false positive signals.8 Despite this, the medical literature is a highly efficient warning system for new adverse reactions, and often recognises rare events and people at high risk.9 10
Monitoring centres
The second system comprises national adverse drug reaction monitoring centres and the WHO Collaborating Centre for International Drug Monitoring. These are not open to third parties such as consumers, patients, and healthcare professionals. Most of these systems work with a yellow card scheme, which can be productive if active and well qualified staff considers the detection of adverse reactions as its primary objective. Such monitoring centres are, however, often understaffed, work in isolated non-academic environments, have all kinds of responsibilities for regulation, and are overwhelmed by reports from the pharmaceutical industry. Consequently there is little time for analysing the reports on adverse reactions sent in by doctors. The legal requirements for reporting adverse events to the Food and Drug Administration and European Medicines Evaluation Agency result in vast numbers of reports, all of which have to be processed. Many of these have a low likelihood of a causal relation. When it is likely that the literature produces really new signals in which more than 50% are causally related, most accounts of the systems that accumulate reports from industry consist of either known adverse reactions or unrelated adverse events. Considering the enormous investment in human resources in industry and government to run this system, efficiency is low. As in the end the cost will inevitably be paid by health insurance systems and patients, the price of drugs will further increase in the future.
Hypothesis generating signals
Over the past 30 years attempts have been made to enhance the recognition of adverse effects by “data dredging” or “data mining.” The contribution of such techniques to detecting adverse reactions has been modest. When substantial numbers of reports are available, however, comparing the proportion of reports of an adverse effect with similar drugs may provide strong hypothesis generating signals, such as for rhabdomyolysis associated with cerivastatin.11 Such comparison may even facilitate some sort of hypothesis testing relative risk assessment.12 13 The validity of such techniques, however, is probably much lower than that of formal designs of epidemiological studies.
The originally embraced technique of prescription event monitoring has proved more difficult than expected.14 Use of research databases such as the General Practice Research Database for the detection of adverse effects by generating an hypothesis has had only limited success. It seems better to restrict the use of such databases to hypothesis-testing epidemiological studies.
Verification and quantification
As soon as an adverse reaction is signalled, a hypothesis is raised that requires formal testing. Relevant questions are how often the adverse reaction occurs and how many times the risk of the adverse effect is increased by the drug.
To this end the observational designs of the cohort study and the case-control study have proved useful.15 Cohort studies are particularly useful for uncommonly used drugs but are less useful if the disease being assessed is rare. Case-control studies are efficient for studying rare diseases but require that the drug is commonly used. An example of a successful case-control study was the determination of an association between vaginal carcinoma of the female offspring of women who had been treated with diethylstilbestrol during pregnancy.16 The authors were able to show the association with only eight cases and 40 controls.
Summary points
More effort should be put into researching the safety of drugs after marketing
In daily practice many signals of a potential adverse event are not followed by a systematic process of verification and quantification
The medical literature is probably by far the most effective system for initial detection of adverse reactions to drugs
The current emphasis on the costly procedures of mandatory reporting should shift towards epidemiological studies for testing a hypothesis
Well designed cohort or case-control studies can adequately deal with bias and confounding, the two potential criticisms of these study designs. The issue of confounding by indication is seldom a problem with rare adverse effects because such unpredictable effects are usually not associated with the indication for treatment.17 Although in such studies confounding by contraindication may play a part, it leads to a conservative estimate rather than to an overestimation of the true risk.18
The cohort and case-control designs can be used to test hypotheses in de novo field studies. Several databases facilitate the performance of such studies with prospectively gathered information on exposure to a drug and disease (see bmj.com). With these data resources, several successful pharmacoepidemiological studies have been performed (table). Unfortunately and despite the enormous growth of pharmacoepidemiology and its capabilities, most drugs are withdrawn on the bases of case reports and case series alone.6 Therefore it is time these databases are used more consistently for hypothesis testing in research concerning drug safety.
Databases used by pharmacoepidemiological studies to test hypotheses. Values are relative risks (95% confidence intervals) unless stated otherwise
In conclusion, society has the right to be safeguarded against the adverse effects of new drugs. The current emphasis on the costly procedures of mandatory reporting should perhaps shift towards epidemiological studies for testing a hypothesis. When particular drug classes and clusters of disease are involved, regulatory authorities and drug inspectorates should take the lead to fulfil their primary task of guaranteeing safe health care.
Footnotes
-
Systems for drug safety research and references w1-w10 are on bmj.com -
We thank J P Vandenbroucke for his critical advice in preparing this manuscript.
-
Contributors BHcS wrote the first draft of the article. BMP critically reviewed the manuscript. BHcS will act as guarantor for the article.
-
Competing interests None declared.