Abstract
Our objective was to summarise systematically all research evidence related to how patients value outcomes in chronic obstructive pulmonary disease (COPD).
We conducted a systematic review (systematic review registration number CRD42015015206) by searching PubMed, Embase, PsycInfo and CINAHL, and included reports that assessed the relative importance of outcomes from COPD patients' perspective. Two authors independently determined the eligibility of studies, abstracted the eligible studies and assessed risk of bias. We narratively summarised eligible studies, meta-analysed utilities for individual outcomes and assessed the certainty of evidence using the Grading of Recommendations, Assessment, Development and Evaluations approach.
We included 217 quantitative studies. Investigators most commonly used utility measurements of outcomes (n=136), discrete choice exercises (n=13), probability trade-off (n=4) and forced choice techniques (n=46). Patients rated adverse events as important but on average, less so than symptom relief. Exacerbation and hospitalisation due to exacerbation are the outcomes that COPD patients rate as most important. This systematic review provides a comprehensive registry of related studies.
Abstract
Systematic review of the importance placed by patients on COPD outcomes informs the trade-off between benefits and harms http://ow.ly/l5De30kgD9g
Introduction
Considering patient values and preferences regarding the benefits and harms of a health intervention is essential for clinical evidence-based decision-making [1–4]. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group has recently operationalised patient values and preferences as “the relative importance patients place on outcomes” [3, 5]. Information about the relative importance of outcomes is critical to weigh the health benefits and harms of interventions and test strategies [5], including those recommended in clinical practice guidelines. Indeed, numerous studies have addressed how patients value chronic obstructive pulmonary disease (COPD) outcomes but to appropriately inform practice and guidelines, this evidence should be summarised in systematic reviews that allow retrieving and summarising the best evidence from individual studies on health outcomes [2, 6–9]. Considering the disease burden of COPD [10], such a review would inform decision-making for a large patient community globally. We therefore conducted this systematic review to summarise all research evidence that addressed the question, what is the relative importance patients place on COPD-related outcomes [3, 5]?
Methods
Protocol and registration
We conducted this systematic review of the literature in accordance with the Preferred Reporting in Systematic Reviews and Meta-Analyses guidelines [11] and registered the review protocol on PROSPERO (registration number CRD42015015206).
Information sources
We searched Medline (through PubMed), Embase, PsycInfo and CINAHL from inception date to October 15, 2017, using an extensive search strategy developed for retrieving this type of evidence (supplementary material) [12], including reference lists of identified studies.
Study selection
Two authors independently determined the eligibility of studies by reviewing titles and abstracts and, for potentially eligible studies, through review of full-text articles with a standardised and piloted screening form. Reviewers resolved disagreement by discussion or through third-party adjudication. Eligible studies reported patient values and preferences of COPD patients, with no limits on the type of study design, language or treatments. Studies with the following characteristics were eligible for reporting the relative importance of outcomes [4].
1) Patient utility and health state value studies: studies that examined how patients value alternative health states and experiences with treatment. The eligible measurement techniques were: standard gamble, time trade-off (TTO), visual analogue scale (VAS), or mapping results based on either generic (EuroQol-5D (EQ-5D) or SF-36) [13] or specific measurement (i.e. Chronic Respiratory Questionnaire) of health-related quality of life. We expected one major category of eligible studies to be “utility” studies. Utilities represent the strength of an individual's preferences for different outcomes. They are expressed on a scale from 0 indicating dead to 1 indicating perfect health (for some variations of the scale, the upper bound may be 100). The higher the utility is (the closer the estimate is to perfect health), the more value patients will place on the outcome.
2) Direct choice studies: studies that examined patients' choice when they were presented with a description of hypothetical states or during decision making for their own actual health states (i.e. forced choice when presented with a decision aid, probabilistic trade-off techniques, discrete choice, willingness to pay, randomised controlled trials (RCTs) for preferences, etc.).
3) Other quantitative studies on outcome importance: studies that quantitatively examined the patients' views, attitudes or preferences on outcome importance through self-developed questionnaires or instruments that were not utility measurement techniques.
We included only quantitative studies reporting COPD as a comorbidity if they reported COPD relative importance of outcomes information separately. We excluded non-original studies such as clinical practice guidelines, reviews, commentaries, letters or viewpoints. We also excluded case reports, case series and health economic evaluation studies without original utility elicitation. Qualitative studies that explored patients' views, attitudes or preferences related to different treatment options were excluded from this review but included and reported in a subsequent review.
Data collection and certainty of evidence
Two authors independently recorded data: principal author, publication year, participant demographics (sample size, age, sex, etc.), survey techniques or methodologies used, relative importance of outcome results and risk of bias assessments.
Since there is no accepted risk of bias or study quality assessment tool for value and preference studies, we used an approach that we developed, validated and reported in a separate project [14]. The key items to assess the risk of bias include sample selection, response rate (or attrition rate if participants were followed up), choice and administration of the instrument, outcome (or health state) presentation, participants' understanding of the methodology, and data analysis (if applicable). We then used the GRADE approach to rate the certainty of the overall body of evidence for outcome importance [14, 15]. The GRADE approach classifies certainty of evidence as high, moderate, low or very low based on domains of risk of bias, inconsistency, indirectness, imprecision, publication bias and upgrading domains.
Data analysis
A priori, we set the disease severity following the Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria, indicating the severity of airflow obstruction, as a potential subgroup factor to consider [16]. We used the severity of airflow obstruction categories of mild (forced expiratory volume in 1 s (FEV1) ≥80% predicted), moderate (FEV1 ≥50% to <80% predicted), severe (FEV1 ≥30% to <50% predicted) and very severe (FEV1 <30% predicted) reported by authors to determine subgroups. Information on the relative importance of outcomes exists in a variety of formats, including the utility of outcomes or disease stages, proportion of choice, rankings or scores on a scale. For the sake of simplicity, we report all estimates using the descriptive term “utility” to indicate the health status values elicited from standard gamble, TTO, VAS and results from indirect utility measurements [17]. We conducted meta-analyses to synthesise the utility results for same outcomes using a random-effects inverse variance method in Stata 11.0 (StatCorp, College Station, TX, USA) [18]. For consistency, we presented the results on a 0–1 scale even if they had been elicited on a 0–100 scale. For nonutility results regarding patient values and preferences, we narratively summarised the results.
Results
Study selection and study characteristics
Of 54 598 records, after excluding duplicates, 41 781 titles and abstracts remained; 3154 articles proved potentially eligible and underwent full-text screening. Of these, 217 quantitative studies reporting patient values and preferences on COPD outcomes proved eligible (figure 1 and references of all included studies in the supplementary material).
Flow diagram for systematic review on chronic obstructive pulmonary disease patients’ values and preferences.
Of the 217 eligible studies, 136 reported utility or health state values for COPD outcomes, of which: 69 utilised the feeling thermometer or VAS, including the EQ-5D VAS; eight the standard gamble; and six the TTO. For indirect measurements, 82 studies reported EQ-5D utilities, 14 SF-6D utilities, seven Health Utility Index (HUI), seven 15D and three the Quality of Well-Being utilities. Of 65 direct choice studies, 46 used forced choice techniques, 13 discrete choice exercise/conjoint analysis or willingness to pay, four probability trade-off, and three ranking methods (supplementary table 1). Regarding study design, 127 were cross-sectional studies, 21 cohort studies, 11 repeated surveys, 51 RCTs and seven quasirandomised trials.
Summary of findings
The outcomes studied typically included exacerbation or hospitalisation due to exacerbation, adverse events, symptom relief, and different severities of COPD. Table 1 summarised this type of evidence (also see supplementary table 1). Despite the large number of eligible studies, few reported the relative importance of outcome information on the same outcomes. Meta-analyses were restricted to studies focusing on exacerbation, and different COPD severities measured with VAS and EQ-5D utility. We found no compelling evidence of publication bias.
Supplementary table 2 summarises the risk of bias assessment. Studies suffered from serious risk of bias related to limitations in the validity and reliability of the measurement tools (68 studies directly asking participants to choose among a set of options) and use of a convenience sampling strategy or a volunteer sample (14 studies); or response rates <50% (32 studies). For other risk of bias considerations, we classified most studies as low risk of bias (supplementary table 2).
Utility of exacerbation or hospitalisation due to exacerbation
Importance of exacerbation
The measurements used to elicit the importance of exacerbation or hospitalisation due to exacerbation include VAS (including the EQ-5D VAS) (10 studies [19–27, 47]). TTO (one study [27]) and the EQ-5D utility (seven studies [21–24, 39, 40, 43]). We observed variations in the description of “exacerbation”. Three studies utilised clinical diagnoses, without a specific definition of an exacerbation [24, 43, 47]. All remaining studies defined exacerbation as worsened symptoms [19–27, 39, 40, 43, 47, 48]. Of these, three studies explicitly reported a category for exacerbation, with one using the definition of the British Thoracic Society [21] and the two others the American Thoracic Society/European Respiratory Society definition [23, 40]. The other reports varied in defining “exacerbation,” with three reports focusing on exacerbations needing hospitalisation [19, 20, 26] and another one specifying the length of symptoms [22]. The estimates varied from 0.259 to 0.580 on the VAS, and 0.430 to 0.740 with the EQ-5D utility. We conducted meta-analysis using the inverse variance method to pool the estimates based on VAS and EQ-5D, yielding utility of exacerbation of 0.462 (95% CI 0.453–0.471, I2=98.2%, p<0.001 for the test of heterogeneity) on the VAS and 0.519 (95% CI 0.502–0.537, I2=95.5%, p<0.001 for the test of heterogeneity) with the EQ-5D utility. Of the eight studies included in the meta-analysis, six recruited patient populations with a mean age between 66 and 69 years [19–21, 24–26]; of those, four were from the UK [21, 24–26], two from other European countries [19, 20], one from the USA [22], and another was a multicentre study conducted in countries including the USA, UK and other countries [23]. The study populations were similar across the studies regarding age and setting. We could not explain the large degree of inconsistency and, thus, rated down the certainty of evidence as moderate (table 1). For studies that used the EQ-5D utility measurement, we further rated down for indirectness given the indirect measurement tool used (i.e. the patients participating did not themselves place a value on exacerbations, but merely reported the consequences on EQ-5D items). One study used a more granular approach to addressing the importance of exacerbations: Rutten van Molken et al. [27] reported the values of different severities of exacerbations. The authors described serious and nonserious exacerbations according to the severity of increase in respiratory and nonrespiratory symptoms, impact on daily activities, and response to treatment. To summarise, for a nonserious exacerbation, patients will experience mild-to-moderate worsening of breathlessness and cough, and the symptoms interfere with daily activities; while patients with a serious exacerbation will experience severe-to-very severe worsening of breathlessness and cough, and the symptoms will completely disrupt daily activities. Based on VAS and TTO measurements, respectively, the disutility (defined as a reduction in utility) for one nonserious exacerbation was 0.037 (VAS) and 0.010 (TTO); for two nonserious exacerbations, 0.068 and 0.021; for one serious exacerbation, 0.090 and 0.042; for one serious exacerbation and one nonserious, 0.130 and 0.088 (table 2). The certainty of this evidence is high. Other studies suggested patients have lower utility as the exacerbations became more frequent or more severe [40, 43].
Importance of dyspnoea
Few studies explored the importance that patients place on dyspnoea. Three studies reported utilities related to dyspnoea. Kim et al. [38] reported the utilities measured by VAS by levels of breathlessness: 0.751, 0.656 and 0.526 for level 1 (short of breath during strenuous activities), level 2 (stopping to catch breath after a few minutes walking) and level 3 (breathless when dressing or washing) breathlessness, respectively. The estimates were based on a small sample, so we downgraded the certainty for the estimates by one level for level 2 and two levels for level 3 breathlessness due to concerns about imprecision. Two other reports corroborated that the more severe the dyspnoea symptom, the lower utility patients place on their health, though the specific levels of breathlessness were described differently (table 3) [43, 49]. Other structured surveys, without reporting utility values, also suggested dyspnoea as burdensome and a very important consideration in COPD-related decision-making [31, 50–58].
Importance on breathlessness, shortness of breath, or dyspnoea
Adverse events
Table 4 summarises the results related to the importance of adverse events. One of the two included discrete-choice studies compared the “possibility of adverse effects” with “the extent to which treatment seems to relieve symptoms”, “the extent to which the doctor gives sufficient time to listen to the patient”, “costs of treatment”, “the extent to which the patient sees the same doctor each time”, and “the extent to which the doctor treats the patient as an entire person” [29]. The extent of symptom relief was deemed to be more important than adverse effects but the possibility of adverse effects was more important than other outcomes. Another discrete choice study suggested symptom relief to be the most important outcome, while the possibility of adverse events was considered more important than the timing and use of (rescue) medicine [30]. The latter study was an online voluntary study in 515 participants, which we rated as having serious risk of bias due to selection bias and limited validity of the instrument. None of the studies explicitly described the outcome of “adverse events.” The overall certainty of evidence about the importance of adverse events, based on these two discrete choice studies, is moderate due to serious risk of bias.
Importance of adverse events
Symptom relief
In general, patients considered symptom relief important. In one survey, 46.6% of patients considered relief of symptoms (i.e., chest pain due to coughing, shortness of breath, nausea, etc.) as extremely important (ranking second after “not to be kept alive on life support when there is little hope for a meaningful recovery”) [61]. For the extent of symptom relief, two discrete-choice studies suggested the extent of symptom relief as more important than adverse effects, the doctor giving sufficient time to listen to the patient, costs of treatment, seeing the same doctor each time, being treated as an entire person, onset time of medication, ease of medication use and use of rescue medication [29, 30]. A large proportion of the study participants were recruited through an online survey, and the eligibility of the participants and the accuracy of their answers were in question. For these reasons, we classified this study at high risk of bias and downgraded the certainty of evidence as moderate (table 1). Three other forced-choice studies corroborated this result [31–33]. For example, in a survey addressing expectation of treatment, 82.3% of the respondents chose greater symptomatic relief as the most important outcome [33]. Because the instruments in these surveys lacked evidence of validity, we rated down the certainty of evidence for risk of bias (moderate certainty evidence).
Utility of COPD
Most studies addressing the utility of the experience of COPD itself were based on EQ-5D, HUI and 15D. Table 5 summarises the utilities based on various instruments across the airflow obstruction levels. Based on the EQ-5D only, we observed a gradient of disutility across GOLD stages: pooled estimates for EQ-5D measurements of mild COPD 0.873 (95% CI 0.863–0.883, I2=91.3%, p<0.001 for heterogeneity) [28, 35–37, 42, 43, 45], moderate 0.821 (95% CI 0.815–0.826, I2=97.8%, p<0.001 for heterogeneity) [28, 35–37, 41–45]; severe 0.741 (95% CI 0.734–0.749, I2=94.5%, p<0.001 for heterogeneity) [28, 35–37, 39–42, 44] and very severe 0.681 (95% CI 0.667–0.694, I2=80.2%, p<0.001 for heterogeneity) [28, 35–37, 39, 40–42, 44], respectively (figure 2). We rated down the certainty of evidence for these utilities due to unexplained inconsistency and for indirectness of the measurement tool (EQ-5D) (low-certainty evidence); we also observed a similar trend with VAS results (table 1).
Utility of different chronic obstructive pulmonary disease (COPD) severities
Forest plots for EuroQol-5D (EQ-5D) utility of chronic obstructive pulmonary disease patients with forced expiratory volume in 1 s a) ≥80%, b) <80–≥50%, c) <50–≥30% and d) <30% predicted.
Other results
We also identified studies reporting importance on other outcomes (supplementary table 3); for example, intubation and speed of symptom relief.
Discussion
We have conducted the most comprehensive systematic review to date of how COPD patients value outcomes. The identified studies were highly variable in their designs, measurement instruments used and outcomes addressed. Patients rated exacerbations of COPD or hospitalisation due to exacerbations as very important. Studies, primarily using the EQ-5D, consistently reported that the utility associated with living with COPD decreases as the disease progresses. Patients considered symptom relief important and more important than adverse events from treatment.
Several aspects distinguish our work from previous published literature reviews [63–67]. Our work yielded more studies because of the broad definition focusing on the importance of outcomes, and including all types of relevant studies and measurement tools. For example, our work is more comprehensive than the work Moayeri et al. [63] who evaluated EQ-5D utilities of COPD stages, though the results of our pooled EQ-5D utilities proved similar. Two other reviews included only multiattribute utility results [63, 64]. Brooker et al. [67] identified 10 studies on patient preferences for mechanical ventilation in COPD, most of them cross-sectional surveys with forced choice questions. A second aspect in which our work differs is the critical assessment, both at the individual-study level for risk of bias, and at the body of evidence level with the GRADE approach and the associated summary of findings table [68].
Our study has some limitations. First, because of the paucity of evidence based on standard gamble and TTO, we were only able to conduct meta-analysis across severity levels of EQ-5D utility and VAS measurements. For the same reason, we were unable to quantitatively explore the study population characteristics as potential sources of inconsistency through approaches such as metaregression. Second, we identified a relatively small number of discrete choice and probability trade-off studies. These studies could provide information on the threshold for a change in decision [69] and have the merit of allowing customisation of the methodology according to the study objectives. The few probability trade-off and discrete choice exercise studies reported only a limited range of attributes and levels of attributes [70–72]. Lastly, given the lack of empirical knowledge in what manner and to what extent publication bias may affect our systematic review results, our assessment of publication bias is limited.
Given the breadth of findings, this systematic review has implications for healthcare providers, researchers including systematic review authors and guideline developers. This systematic review summarises current evidence to inform guideline developers about how important the benefits and harms of COPD treatment strategies are from the patients' perspective. The results will inform clinicians who make decisions with COPD patients. This systematic review provides empirical evidence to support using the relative importance of outcomes to inform values and preferences, and the methods can be used by systematic review authors who are interested in other disease topics. The utilities summarised serve as the parameter inputs for cost analyses. When guideline developers determine the balance between benefits and harms, they can take into consideration both the probability and the importance of benefits (e.g. symptom relief) and harms (e.g. adverse events) from this review. Additionally, the results of this review also help researchers identify research gaps for designing new studies.
Research gaps exist when there is no evidence, or the certainty of evidence is low or very low. For example, although there is evidence about the importance of adverse events, guideline developers need to know the exact types and probabilities of adverse events considered by patients. Researchers can use standard gamble, discrete choice and probability trade-off techniques to address the levels of adverse events, with the severities or probabilities directly relevant to the research questions [29]. Additionally, for better understanding and application of the findings, researchers also need to further explore the socioeconomic, cultural and disease-specific characteristics that influence patient values on the COPD outcomes.
There are still unanswered challenges related to the optimal strategy to elicit the outcome importance evidence. For considering the risk of bias, one concern is the merits of measurement tools involving a valuation of hypothetical scenarios in relation to measurements of an actual outcome that participants experience. If the participants value a health state specified by the investigators, barring only different interpretations or limited understanding, they value the same outcome; but if, for example, participants are asked to evaluate the outcome “shortness of breath” they are experiencing, or having experienced in the past, the degree of shortness may vary a lot across participants. Further studies are also necessary to validate the search strategy for these types of studies. Our strategy, which is sensitive but not specific, led to a large number of hits [12], replication of which would place a substantial burden for systematic review authors and guideline panels (as it did for us).
Conclusion
Our systematic review showed that patients value the outcome of exacerbation or hospitalisation due to exacerbation as very important. We observed large variability in the utility associated with COPD severity across studies. We identified a gradient of disutility as the disease progresses, from both the direct utility instrument VAS and the indirect utility instrument EQ-5D. Quantitative approaches, including direct and indirect utility measurement of outcomes, discrete choice exercise, probability trade-off and forced choice, represent the predominant measurement instruments investigators have used to address the importance patients place on outcomes.
Although further studies are necessary to explore the unsolved methodological questions, through this systematic review process, we demonstrated the usefulness of systematic reviews as a potential strategy for summarising evidence in this field and informing decision makers, both in the context of health technology assessments and guidelines.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Search strategy ERJ-00222-2018_Search_Strategy
References of all included studies ERJ-00222-2018_References_of_Included_Studies
TABLE S1 Study characteristics ERJ-00222-2018_tableS1
TABLE S2 Summary of risk of bias ERJ-00222-2018_tableS2
TABLE S3 Quantitative results ERJ-00222-2018_tableS3
Acknowledgements
We are grateful to Amiram Gafni (McMaster University) for the comments on the manuscript, and Sean Doran (University of Missouri, Kansas City, MO, USA) for title and abstract screening.
Footnotes
This article has supplementary material available from erj.ersjournals.com
Author Contributions: Y. Zhang and H.J. Schünemann designed the study; Y. Zhang, R.L. Morgan, P. Alonso-Coello, A. Selva, H. Ara Begum, G.P. Morgano, W. Wiercioch, M. Ventresca, M.M. Bała, R.R. Jaeschke, M.T. Wasylewski, K. Styczeń, H. Pardo-Hernandez, A. Agarwal, J-L. Kerth, L. Blanco-Silvente, M. Wang, Y. Zhang, S. Narsingam and Y. Fei screened the literature and abstracted the data; Y. Zhang, R.L. Morgan, P. Alonso-Coello, G. Guyatt and H.J. Schünemann drafted the manuscript; all authors read and approved the final manuscript; and H.J. Schünemann conceived of and funded the study.
Conflict of interest: H.J. Schünemann reports that he has no financial conflict of interest. He is Co-chair of the GRADE working group.
Support statement: This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors but was supported by the MacGRADE center at McMaster University. This project does not have any sponsorship from industrial or governmental sources in its design, collection of data, analysis or interpretation of data, writing of the report, or decision to submit for publication. Funding information for this article has been deposited with the Crossref Funder Registry.
- Received January 21, 2018.
- Accepted May 21, 2018.
- The content of this work is copyright of the authors or their employers. Design and branding are copyright ©ERS 2018.
References
