Cost-effectiveness comparisons using “real world” randomized trials: The case of new antidepressant drugs

doi:10.1016/0895-4356(94)00141-C

Journal of Clinical Epidemiology

Volume 48, Issue 3, March 1995, Pages 363-373

https://doi.org/10.1016/0895-4356(94)00141-C Get rights and content

Abstract

While randomized clinical trials remain the “gold standard” for comparisons of treatment efficacy, conventional randomized trials may not reflect the relative effectiveness of treatments under “real world” conditions. Observational data more closely reflect the conditions of actual practice, but are often limited in clinical detail and subject to bias in selection of treatments. The authors use available data on the cost-effectiveness of new antidepressant drugs to illustrate the limitations of these methods. Modifications to the traditional randomized trial may allow more accurate assessment of effectiveness in everyday practice. The authors describe the design of such a “real world” randomized trial comparing newer antidepressants with older alternatives.

References (43)

D Schwartz et al.
Explanatory and pragmatic attitudes in clinical trials
J Chronic Dis
(1967)
B Flay
Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs
Preventive Med
(1986)
H Schulberg et al.
Strategies for evaluating treatments for major depression in primary care patients
Gen Hosp Psychiat
(1991)
G Simon et al.
Patterns of antidepressant use in community practice
Gen Hosp Psychiat
(1993)
H Schulberg et al.
Treating depression in primary care practice: an application of decision analysis
Gen Hosp Psychiat
(1989)
G Simon et al.
Telephone assessment of depression severity
J Psychiatr Res
(1993)
D Sackett et al.
Controversy in counting and attributing events in clinical trials
NEJM
(1979)
A Feinstein
An additional basic science for clinical medicine: II. The limitations of randomized trials
Ann Intern Med
(1983)
R Fletcher et al.
G Diamond et al.
Alternative perspectives on the biased foundations of medical technology assessment
Ann Intern Med
(1993)

A Leaf

Cost effectiveness as a criterion for Medicare coverage

NEJM

(1989)

J Wennberg

Outcomes research, cost containment, and the fear of health care rationing

NEJM

(1990)

M Chassin et al.

Variations in the use of medical and surgical services by the medicare population

NEJM

(1986)

J Wennberg

Which rate is right?

NEJM

(1986)

JJ Clinton

Outcomes research—a way to improve medical practice. From the Agency of Health Care Policy and Research

JAMA

(1991)

J Kassirer et al.

Decision analysis: a progress report

Ann Intern Med

(1987)

L Melton et al.

Referral bias in diabetes research

Diabetes Care

(1984)

L Sireling et al.

Depression in general practice: clinical features and comparison with outpatients

Br J Psychiat

(1985)

J Ormel et al.

Recognition, management, and course of anxiety and depression in general practice

Arch Gen Psychiat

(1991)

JC Schulberg et al.

Six-month outcomes for medical patients with major depressive disorders

J Gen Int Med

(1987)

C Blacker et al.

Depressive disorders in primary care

Br J Psychiat

(1987)

Cited by (142)

Series: Pragmatic trials and real world evidence: Paper 8. Data collection and management
2017, Journal of Clinical Epidemiology
Citation Excerpt :
This information is usually entered in a structured, coded format, but it may not be very detailed. Consequently, claims databases often lack information on relevant clinical variables and patient characteristics, and often the information will not be timely [12–14]. There is a trend though among database owners of claims data, especially in the United States, to increase the number of clinical variables to make the databases usable for scientific research purposes (eg, Optum) [15].
Pragmatic trials can improve our understanding of how treatments will perform in routine practice. In a series of eight papers, the GetReal Consortium has evaluated the challenges in designing and conducting pragmatic trials and their specific methodological, operational, regulatory, and ethical implications. The present final paper of the series discusses the operational and methodological challenges of data collection in pragmatic trials. A more pragmatic data collection needs to balance the delivery of highly accurate and complete data with minimizing the level of interference that data entry and verification induce with clinical practice. Furthermore, it should allow for the involvement of a representative sample of practices, physicians, and patients who prescribe/receive treatment in routine care. This paper discusses challenges that are related to the different methods of data collection and presents potential solutions where possible. No one-size-fits-all recommendation can be given for the collection of data in pragmatic trials, although in general the application of existing routinely used data-collection systems and processes seems to best suit the pragmatic approach. However, data access and privacy, the time points of data collection, the level of detail in the data, and the lack of a clear understanding of the data-collection process were identified as main challenges for the usage of routinely collected data in pragmatic trials. A first step should be to determine to what extent existing health care databases provide the necessary study data and can accommodate data collection and management. When more elaborate or detailed data collection or more structured follow-up is required, data collection in a pragmatic trial will have to be tailor-made, often using a hybrid approach using a dedicated electronic case report form (eCRF). In this case, the eCRF should be kept as simple as possible to reduce the burden for practitioners and minimize influence on routine clinical practice.
The "efficacy-effectiveness gap": Historical background and current conceptualization
2016, Value in Health
Citation Excerpt :
The concept of pragmatism holds that this lack of generalizability has led to an EEG and that any direct dissemination of evidence arising from clinical trials into clinical practice might be inadequate [52,53]. In line with this concept, the generation of real-life evidence on the impact of drugs is becoming increasingly recommended, namely, for pharmaceutical companies [10,51,54], in which such evidence is seen as complementary [55–57]. More recently, a third paradigm has been developed in which any quantitative difference in drugs effect estimates, as measured in an experimental setting or in routine practice, may be understood as the result of interaction of multiple real-life characteristics on the purely biological effect of the drug.
The concept of the “efficacy-effectiveness gap” (EEG) has started to challenge confidence in decisions made for drugs when based on randomized controlled trials alone. Launched by the Innovative Medicines Initiative, the GetReal project aims to improve understanding of how to reconcile evidence to support efficacy and effectiveness and at proposing operational solutions.
The objectives of the present narrative review were 1) to understand the historical background in which the concept of the EEG has emerged and 2) to describe the conceptualization of EEG.
A focused literature review was conducted across the gray literature and articles published in English reporting insights on the EEG concept. The identification of different “paradigms” was performed by simple inductive analysis of the documents’ content.
The literature on the EEG falls into three major paradigms, in which EEG is related to 1) real-life characteristics of the health care system; 2) the method used to measure the drug’s effect; and 3) a complex interaction between the drug’s biological effect and contextual factors.
The third paradigm provides an opportunity to look beyond any dichotomy between “standardized” versus “real-life” characteristics of the health care system and study designs. Namely, future research will determine whether the identification of these contextual factors can help to best design randomized controlled trials that provide better estimates of drugs’ effectiveness.
A scale for distinguishing efficacy from effectiveness was adapted and applied to stroke rehabilitation studies
2010, Journal of Clinical Epidemiology
To apply a tool that purports to differentiate between efficacy and effectiveness studies to stroke rehabilitation trials and to evaluate its applicability and reliability.
Three raters developed item operational definitions before independently applying the seven-item scale to 151 randomized controlled trials (RCT), published during or after 1997, that evaluated either a pharmacologic (P, n = 78) or a nonpharmacologic (NP, n = 73) intervention. Inter-rater reliability was assessed for both individual items and total scores, separately for P and NP trials.
Item inter-rater reliability (multiple-rater kappa) ranged from 0.00 (95% CI [confidence interval]: −0.13, 0.13) to 0.85 (95% CI: 0.73, 0.98) and from 0.21 (95% CI: 0.08, 0.34) to 0.79 (95% CI: 0.66, 0.92) for P and NP RCTs, respectively. For the total score (dichotomized), kappa values were 0.43 (95% CI: 0.31, 0.56) and 0.51 (95% CI: 0.37, 0.64) for P and NP trials, respectively.
The tool provides a solid foundation on which to base further discussion of the differential criteria of efficacy–effectiveness trial design. Scale items should be properly operationalized depending on the research question of interest and evaluated for reliability before the scale is used for definitively judging a given study's design or the external validity of its results.
Health economics of osteoporosis
2008, Best Practice and Research: Clinical Endocrinology and Metabolism
Citation Excerpt :
Whereas randomized controlled trials (RCTs) remain the gold standard for comparing alternative treatments, the high internal validity required to demonstrate efficacy comes at the expense of external validity. The results of such trials may therefore generalize poorly to clinical practice.19,20 For example, adherence as observed in clinical trials is likely to be higher than in clinical practice, which in the context of the health economic analysis would yield lower benefits of therapy and potentially overestimated cost-effectiveness when using clinical trial data on adherence and efficacy.
Rather than reviewing the many studies of cost effectiveness in osteoporosis, this paper reviews the principles of cost-effectiveness analysis and the gaps in our knowledge that are required to improve such analyses in osteoporosis. These include more information on the cost of fractures and their consequences on health states, particularly on an international basis. New developments include the incorporation of adherence into models, the use of the FRAX™ tool to assess cost-effectiveness in individuals with any combination of risk factors for fracture, and the setting of intervention thresholds based on cost-effectiveness.
When Can We Rely on Real-World Evidence to Evaluate New Medical Treatments?
2022, Clinical Pharmacology and Therapeutics
Principles and procedures for data and safety monitoring in pragmatic clinical trials
2019, Trials

View all citing articles on Scopus

View full text

Original articleCost-effectiveness comparisons using “real world” randomized trials: The case of new antidepressant drugs

Abstract

J Chronic Dis

Preventive Med

Gen Hosp Psychiat

Gen Hosp Psychiat

Gen Hosp Psychiat

J Psychiatr Res

Controversy in counting and attributing events in clinical trials

NEJM

An additional basic science for clinical medicine: II. The limitations of randomized trials

Ann Intern Med

Alternative perspectives on the biased foundations of medical technology assessment

Ann Intern Med

Cost effectiveness as a criterion for Medicare coverage

NEJM

Outcomes research, cost containment, and the fear of health care rationing

NEJM

Variations in the use of medical and surgical services by the medicare population

NEJM

Which rate is right?

NEJM

Outcomes research—a way to improve medical practice. From the Agency of Health Care Policy and Research

JAMA

Decision analysis: a progress report

Ann Intern Med

Referral bias in diabetes research

Diabetes Care

Depression in general practice: clinical features and comparison with outpatients

Br J Psychiat

Recognition, management, and course of anxiety and depression in general practice

Arch Gen Psychiat

Six-month outcomes for medical patients with major depressive disorders

J Gen Int Med

Depressive disorders in primary care

Br J Psychiat

Original article
Cost-effectiveness comparisons using “real world” randomized trials: The case of new antidepressant drugs