Pity the poor clinical researcher who is charged with identifying and quantifying chronic obstructive pulmonary disease (COPD) exacerbation events in clinical trials. An exacerbation of COPD is a clinical event that: 1) has no standard, consensus definition [1]; 2) often goes unreported and undetected if patients choose to stay at home instead of presenting to a healthcare provider [2]; 3) often occurs suddenly with little or no warning [3]; and 4) is subject to diagnostic uncertainty and is easily confused with pulmonary embolism, congestive heart failure or pneumonia [4–6].
Given these inherent difficulties, much-needed efforts have been made towards developing better measurement tools for quantifying COPD exacerbation events [7]. One such new measurement tool is the Exacerbations of Chronic Pulmonary Disease Tool (EXACT), a 14-item, patient-reported daily symptom diary that attempts to accurately capture the frequency, severity, and duration of exacerbations, and which is meant to be used in clinical trials and cohort studies of COPD patients [8]. A tool such as EXACT is desperately needed by COPD clinical researchers [9], assuming that it works.
In this issue of the European Respiratory Journal, Mackay et al. [10] have attempted to test EXACT to see if it fulfils its expected functions. The investigators prospectively administered EXACT as well as their previously validated London COPD Cohort diary cards to 58 patients in their COPD cohort on a daily basis [10]. Clearly, the hope going into the study was that EXACT events and London diary card events would be concordant, and that EXACT would provide additional important information on the severity and duration of the exacerbation event that is currently not available from the London diary card.
The results of the study are certainly mixed. The good news is that mean EXACT scores did increase, as expected, during exacerbation events relative to the stable state, and that the time taken for EXACT scores to return to baseline was significantly correlated to both diary-card symptom recovery time and lung function recovery. This information suggests that EXACT can be used to measure the duration of COPD exacerbation events.
The bad news is that this study suggests that the EXACT seems to be relatively insensitive in detecting exacerbation events. Only 34 (27%) out of 128 of London diary card exacerbations exceeded the EXACT threshold for an exacerbation event (defined as a 12-point increase in EXACT score above baseline for two consecutive days or a 9-point increase for three days). Even more worryingly, of the 85 London COPD Cohort diary card-defined exacerbations that were treated with oral antibiotics and/or corticosteroids by the study team during the 2-year study period, only 34% were picked up using EXACT. The study results thus suggest that EXACT may be failing to detect up to two-thirds of clinically important outpatient COPD exacerbations. This deficiency could severely compromise the ability of EXACT to function as a useful measurement tool for COPD clinical trials.
EXACT was developed under the EXACT-PRO (Exacerbations of Chronic Pulmonary Disease Tool – Patient Reported Outcome) initiative, a multiyear, multisponsor project involving experts in pulmonary medicine, instrument development and drug development [8]. A lot of time and money was spent on development of EXACT. Before we throw out the baby with the bathwater based on the results of the study by Mackay et al. [10], it is worth noting potential limitations of this study which may have affected results.
The most important limitation of the study is that it may not be generalisable to typical patients with COPD. This was a study of the London COPD Cohort, and these patients had been specifically trained to fill out London diary cards and react to London diary card exacerbation events, rather than to EXACT events. These patients may have presented for medical attention earlier in the course of their exacerbations than typical patients with COPD and they may have received earlier treatment [11], thus potentially dampening their EXACT symptom scores and making them less likely to cross the EXACT threshold for symptom-defined events. A more fair assessment of EXACT would involve repeating this study in an unselected cohort of COPD patients who are instructed to seek medical attention and treatment according to usual care and practice. This would provide a more “real-world” assessment of whether EXACT is as sensitive as the London diary card for detection of acute exacerbations of COPD (AECOPD). An ongoing study that is currently evaluating beclomethasone/formoterol for treatment of COPD is comparing physician-diagnosed COPD exacerbation events to exacerbation events detected using EXACT. Results of this study, once available, may give us important information on EXACT’s sensitivity for detecting exacerbations in a more generalisable COPD population [12].
Another potential limitation of the study by Mackay et al. [10] is the absence of a reliable gold standard on which to base measurements of validity [13]. In the absence of a reliable biomarker for diagnosis of AECOPD, and in the absence of a gold standard definition for AECOPD, it is difficult to be sure that EXACT was really as insensitive as it seems. It is probably incorrect to assume that the London diary card is the gold standard and that all of the 128 events detected using the London diary cards were true COPD exacerbation events. It may be that the EXACT definition of an acute COPD exacerbation is more specific than that of the London diary card and, theoretically, EXACT may be ignoring “false-positive” events that were incorrectly identified by the London diary card as acute exacerbations. However, in fairness to the design of the study by Mckay et al. [10], the 85 diary card events that lead to a physician encounter and physician-directed treatment with antibiotics and/or steroids are certainly less likely to have been false-positive events: these are the events that are traditionally classified as moderate COPD exacerbations and EXACT needs to detect these events in order to be a useful measurement tool for COPD clinical trials.
Finally, it may be that the EXACT thresholds still need tweaking. Perhaps a 12-point increase in EXACT score above baseline for two consecutive days is too stringent? A sensitivity analysis discussed in the online supplementary of the article by Mckay et al. [10] demonstrates that an EXACT threshold of a 7.5-point increase for two consecutive days detected 43 (51%) out of 85 healthcare utilisation exacerbations. Perhaps EXACT is a good tool but the published thresholds defining an exacerbation need to be lowered to allow for more optimal receiver operating characteristics.
At present, it is difficult to conclude that EXACT is effective at measuring COPD exacerbation events, especially when the study by Mckay et al. [10] suggests that EXACT is missing two-thirds of the clinically important events in question. One cannot measure something if one does not know that it is there.
Footnotes
Conflict of interest: None declared.
- Received September 9, 2013.
- Accepted September 26, 2013.
- ©ERS 2014