Performance of tests of significance based on stratification by a multivariate confounder score or by a propensity score

doi:10.1016/0895-4356(89)90036-X

Journal of Clinical Epidemiology

Volume 42, Issue 4, 1989, Pages 317-324

https://doi.org/10.1016/0895-4356(89)90036-X Get rights and content

Abstract

Stratification by a multivariate confounder score or by a propensity score has been proposed for the multi-confounder situations that are commonly encountered in epidemiologic evaluations of the effects of a treatment or a triage decision. However, the use of these scores in clinical research has been limited, perhaps in part because of the concern that the stated level of statistical significance may be exaggerated when there is a high degree of correlation between the exposure and the set of confounders. We present a specific example and computer simulations to suggest that exaggeration of statistical significance occurs only under unusual circumstances when the correlation between the exposure and the confounders is extreme. Our simulations also suggest that an analysis based on stratification by a propensity score is less affected by high correlation between the exposure and the confounders than is an analysis based on a multivariate confounder score.

References (11)

O.S. Miettinen
Efficacy of therapeutic practices; will epidemiology provide the answers?
A.R. Feinstein
An additional basic science for clinical medicine: II The limitations of randomized trials
Ann Intern Med
(1983)
E.F. Cook et al.
Asymmetric stratification: An outline for an efficient method for controlling confounding in cohort studies
Am J Epidemiol
(1988)
O.S. Miettinen
Stratification by a multivariate confounder score
Am J Epidemiol
(1976)
P.R. Rosenbaum et al.
The central role of the propensity score in observational studies for causal effects
Biometrika
(1983)

There are more references available in the full text version of this article.

Cited by (104)

Choice of time period to identify confounders for propensity score matching, affected the estimate: a retrospective cohort study of drug effectiveness in asthmatic children
2018, Journal of Clinical Epidemiology
Citation Excerpt :
It is used to adjust for imbalances between treatment groups. The utility of PS in such situations has been extensively demonstrated [8–14]. The factors that exacerbate asthma and result in treatment step-up are likely to occur shortly before treatment step-up, but the relevant period over which confounding occurs is not clear.
To control for confounding by indication in comparative (drug) effectiveness studies, propensity score (PS) methods may be used. Since childhood diseases or outcomes often present as acute events, we compared the effect of using different look-back periods in electronic health-care data, to construct PSs. This was applied in our research on the effect of a combination of inhaled corticosteroids/long-acting beta-2 agonists (ICS + LABA), either as fixed combination or used as loose combination (2 separate inhaler devices) in the prevention of severe asthma exacerbations.
We created a cohort of children (5–17 years) diagnosed with asthma from the Dutch Integrated Primary Care information database. Within this cohort, we identified new users of ICS + LABA, either as fixed combination or loose combination (2 separate inhaler devices). The outcome of interest was severe asthma exacerbations. PSs for type of treatment were created using comorbidity and drug use history in different time windows: 1 week, 1 month, 3 months, 1 year, and full history prior to the start of treatment. PSs were used for matching subjects in both exposure groups. Time to first asthma exacerbation was analyzed with Cox proportional hazard regression. The results were compared with published clinical trials.
Of 39,682 asthmatic children, 3,500 (8.8%) were new users of either ICS + LABA fixed (3,324 [95.0%]) or loose (176 [5.0%]). The crude hazard ratio (HR) for a severe asthma exacerbation, comparing ICS + LABA fixed to loose was 0.37 (95% confidence interval [CI]: 0.20–0.66). PS-matched HRs (1 week, 1 month, 3 month, 1 year, and full history) were 0.48 (95% CI: 0.22–1.04); 0.60 (95% CI: 0.26–1.38), 0.69 (95% CI: 0.31–1.57), 0.56 (CI: 0.25–1.24), and 0.58 (CI: 0.24–1.36), respectively.
PS matching can be used to control for confounding in pediatric comparative (drug) effectiveness studies, the impact of different look-back periods to implement the PS is important. Controlling for confounders occurring in the 3 months preceding drug exposure may yield results comparable to clinical trial results.
Analytical approaches for post-authorization safety studies
2018, Post-Authorization Safety Studies of Medicinal Products: The PASS Book
Coronary artery disease is associated with an increased mortality rate following video-assisted thoracoscopic lobectomy
2017, Journal of Thoracic and Cardiovascular Surgery
To compare the incidence of major adverse cardiac events (MACE) and mortality following video-assisted thoracoscopic surgery (VATS) lobectomy in patients with and without coronary artery disease (CAD).
Multicentre retrospective analysis of 1699 patients undergoing VATS lobectomy (January 2012-March 2015). CAD definition: previous acute myocardial infarct (AMI), angina, percutaneous coronary intervention (PCI) or coronary artery bypass graft (CABG). MACE definition: postoperative acute myocardial ischemia, cardiac arrest or any cardiac death. Propensity score analysis was performed to match patients with and without CAD. Outcomes of the 2 matched groups were compared.
The incidence of MACE and mortality for the entire population was 0.4% (7 patients) and 1.7% (29 patients); 218 patients (13%) had a history of CAD: 106 previous AMI, 55 angina, 32 CABG, and 81 PCI. The propensity score yielded 2 well-balanced groups of 218 pairs with and without CAD. MACE (CAD 2 [0.9%] vs no-CAD 1 [0.5%]; P = 1), cardiovascular and pulmonary complications (CAD 61 [28%] vs no-CAD 51 [23%]; P = .3) and postoperative stay (CAD 7.3 days vs no-CAD 6.2 days; P = .3) were not different between the groups. The incidence of atrial fibrillation (CAD 31 [14%] vs no-CAD 18 [8.2%]; P = .07), 30-day mortality (CAD: 11 [5%] vs no-CAD 2 [0.9%]; P = .02) and death among complicated patients (CAD 18% vs no-CAD 3.9%; P = .009) were higher in the CAD group.
The incidence of MACE following VATS lobectomy in patients with CAD is low and similar to patients without CAD. However, their risk of postoperative mortality is fivefold higher compared with non-CAD patients, warranting refined preoperative functional evaluation and more intense postoperative monitoring.
Diffusion of Innovations model helps interpret the comparative uptake of two methodological innovations: co-authorship network analysis and recommendations for the integration of novel methods in practice
2017, Journal of Clinical Epidemiology
Citation Excerpt :
DRS was first proposed in 1976 [13], yet a simulation paper published in 1979 introduced early uncertainty in the method by concluding that it overestimates confounding and thus induces bias [22]. A subsequent simulation published in 1989 concluded that overestimation of confounders was rare [23], and more recent contributions corroborate DRS ability to control for confounding and highlight its potential advantages [19–21,24,25]. This case study thus provides an opportunity to consider the diffusion of an innovation introduced during the infancy of the field of pharmacoepidemiology and over a 40-year time span in the context of initial uncertainty.
The objective of this study was to characterize the diffusion of methodological innovation.
Comparative case study analysis of the diffusion of two methods that summarize confounder information into a single score: disease risk score (DRS) and high-dimensional propensity score (hdPS). We completed systematic searches to identify DRS and hdPS papers in the field of pharmacoepidemiology through to the end of 2013, plotted the number of papers and unique authors over time, and created sociograms and animations to visualize co-authorship networks. First and last author affiliations were used to ascribe institutional contributions to each paper and network.
We identified 43 DRS papers by 153 authors since 1981, reflecting slow uptake during initial periods of uncertainty and broader diffusion since 2001 linked to early adopters from Vanderbilt. We identified 44 hdPS papers by 147 authors since 2009, reflecting rapid and integrated diffusion, likely facilitated by opinion leaders, early presentation at conferences, easily accessible statistical code, and improvement in funding. Most contributions (87% DRS, 96% hdPS) were from North America.
When proposing new methods, authors are encouraged to consider innovation attributes and early evaluation to improve knowledge translation of their innovations for integration into practice, and we provide recommendations for consideration.
Data integration in causal inference
2023, Wiley Interdisciplinary Reviews: Computational Statistics
Advance in distributed evidence-based causal inference methods
2022, Chinese Journal of Disease Control and Prevention

View all citing articles on Scopus

View full text

Original articlePerformance of tests of significance based on stratification by a multivariate confounder score or by a propensity score

Abstract

Efficacy of therapeutic practices; will epidemiology provide the answers?

An additional basic science for clinical medicine: II The limitations of randomized trials

Ann Intern Med

Asymmetric stratification: An outline for an efficient method for controlling confounding in cohort studies

Am J Epidemiol

Stratification by a multivariate confounder score

Am J Epidemiol

The central role of the propensity score in observational studies for causal effects

Biometrika

Original article
Performance of tests of significance based on stratification by a multivariate confounder score or by a propensity score