1
|
Liau MYQ, Toh EQ, Muhamed S, Selvakumar SV, Shelat VG. Can propensity score matching replace randomized controlled trials? World J Methodol 2024; 14:90590. [PMID: 38577204 PMCID: PMC10989411 DOI: 10.5662/wjm.v14.i1.90590] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 01/05/2024] [Accepted: 02/23/2024] [Indexed: 03/07/2024] Open
Abstract
Randomized controlled trials (RCTs) have long been recognized as the gold standard for establishing causal relationships in clinical research. Despite that, various limitations of RCTs prevent its widespread implementation, ranging from the ethicality of withholding potentially-lifesaving treatment from a group to relatively poor external validity due to stringent inclusion criteria, amongst others. However, with the introduction of propensity score matching (PSM) as a retrospective statistical tool, new frontiers in establishing causation in clinical research were opened up. PSM predicts treatment effects using observational data from existing sources such as registries or electronic health records, to create a matched sample of participants who received or did not receive the intervention based on their propensity scores, which takes into account characteristics such as age, gender and comorbidities. Given its retrospective nature and its use of observational data from existing sources, PSM circumvents the aforementioned ethical issues faced by RCTs. Majority of RCTs exclude elderly, pregnant women and young children; thus, evidence of therapy efficacy is rarely proven by robust clinical research for this population. On the other hand, by matching study patient characteristics to that of the population of interest, including the elderly, pregnant women and young children, PSM allows for generalization of results to the wider population and hence greatly increases the external validity. Instead of replacing RCTs with PSM, the synergistic integration of PSM into RCTs stands to provide better research outcomes with both methods complementing each other. For example, in an RCT investigating the impact of mannitol on outcomes among participants of the Intensive Blood Pressure Reduction in Acute Cerebral Hemorrhage Trial, the baseline characteristics of comorbidities and current medications between treatment and control arms were significantly different despite the randomization protocol. Therefore, PSM was incorporated in its analysis to create samples from the treatment and control arms that were matched in terms of these baseline characteristics, thus providing a fairer comparison for the impact of mannitol. This literature review reports the applications, advantages, and considerations of using PSM with RCTs, illustrating its utility in refining randomization, improving external validity, and accounting for non-compliance to protocol. Future research should consider integrating the use of PSM in RCTs to better generalize outcomes to target populations for clinical practice and thereby benefit a wider range of patients, while maintaining the robustness of randomization offered by RCTs.
Collapse
Affiliation(s)
- Matthias Yi Quan Liau
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 308232, Singapore
| | - En Qi Toh
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 308232, Singapore
| | - Shamir Muhamed
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 308232, Singapore
| | - Surya Varma Selvakumar
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 308232, Singapore
| | - Vishalkumar Girishchandra Shelat
- Department of General Surgery, Tan Tock Seng Hospital, Singapore 308433, Singapore
- Surgical Science Training Centre, Tan Tock Seng Hospital, Singapore 308433, Singapore
| |
Collapse
|
2
|
Butala NM, Secemsky E, Kazi DS, Song Y, Strom JB, Faridi KF, Brennan JM, Elmariah S, Shen C, Yeh RW. Applicability of Transcatheter Aortic Valve Replacement Trials to Real-World Clinical Practice: Findings From EXTEND-CoreValve. JACC Cardiovasc Interv 2021; 14:2112-2123. [PMID: 34620389 DOI: 10.1016/j.jcin.2021.08.006] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Revised: 07/28/2021] [Accepted: 08/03/2021] [Indexed: 01/15/2023]
Abstract
OBJECTIVES The aim of this study was to examine the applicability of pivotal transcatheter aortic valve replacement (TAVR) trials to the real-world population of Medicare patients undergoing TAVR. BACKGROUND It is unclear whether randomized controlled trial results of novel cardiovascular devices apply to patients encountered in clinical practice. METHODS Characteristics of patients enrolled in the U.S. CoreValve pivotal trials were compared with those of the population of Medicare beneficiaries who underwent TAVR in U.S. clinical practice between November 2, 2011, and December 31, 2017. Inverse probability weighting was used to reweight the trial cohort on the basis of Medicare patient characteristics, and a "real-world" treatment effect was estimated. RESULTS A total of 2,026 patients underwent TAVR in the U.S. CoreValve pivotal trials, and 135,112 patients underwent TAVR in the Medicare cohort. Trial patients were mostly similar to real-world patients at baseline, though trial patients were more likely to have hypertension (50% vs 39%) and coagulopathy (25% vs 17%), whereas real-world patients were more likely to have congestive heart failure (75% vs 68%) and frailty. The estimated real-world treatment effect of TAVR was an 11.4% absolute reduction in death or stroke (95% CI: 7.50%-14.92%) and an 8.7% absolute reduction in death (95% CI: 5.20%-12.32%) at 1 year with TAVR compared with conventional therapy (surgical aortic valve replacement for intermediate- and high-risk patients and medical therapy for extreme-risk patients). CONCLUSIONS The trial and real-world populations were mostly similar, with some notable differences. Nevertheless, the extrapolated real-world treatment effect was at least as high as the observed trial treatment effect, suggesting that the absolute benefit of TAVR in clinical trials is similar to the benefit of TAVR in the U.S. real-world setting.
Collapse
Affiliation(s)
- Neel M Butala
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA; Cardiology Division, Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Eric Secemsky
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Dhruv S Kazi
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Yang Song
- Baim Institute for Clinical Research, Boston, Massachusetts, USA
| | - Jordan B Strom
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Kamil F Faridi
- Section of Cardiology, Department of Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - J Matthew Brennan
- Duke Clinical Research Institute, Duke University School of Medicine, Durham, North Carolina, USA
| | - Sammy Elmariah
- Cardiology Division, Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Changyu Shen
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Robert W Yeh
- Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Division of Cardiovascular Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA.
| |
Collapse
|
3
|
Engelhardt KE, Bilimoria KY, Johnson JK, Hewitt DB, Ellis RJ, Hu YY, Chung JW, Kreutzer L, Love R, Blay E, Odell DD. A National Mixed-Methods Evaluation of Preparedness for General Surgery Residency and the Association With Resident Burnout. JAMA Surg 2021; 155:851-859. [PMID: 32804992 DOI: 10.1001/jamasurg.2020.2420] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
Importance Differences in medical school experiences may affect how prepared residents feel themselves to be as they enter general surgery residency and may contribute to resident burnout. Objectives To assess preparedness for surgical residency, to identify factors associated with preparedness, to examine the association between preparedness and burnout, and to explore resident and faculty perspectives on resident preparedness. Design, Setting, and Participants This cross-sectional study used convergent mixed-methods analysis of data from a survey of US general surgery residents delivered at the time of the 2017 American Board of Surgery In-Training Examination (January 26 to 31, 2017) in conjunction with qualitative interviews of residents and program directors conducted as part of the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) trial. A total of 262 Accreditation Council for Graduate Medical Education-approved US general surgery residency programs participated. Survey data were collected from 3693 postgraduate year (PGY) 1 and PGY2 surgical residents (response rate, 99%) and 98 interviews were conducted with residents and faculty from September 1 to December 15, 2018. Data were analyzed from June 1, 2017, to February 15, 2018. Main Outcomes and Measures Hierarchical regression models were developed to examine factors associated with preparedness and to assess the association between preparedness and resident burnout. Qualitative interviews were conducted to identify themes associated with preparation for residency. Results Of the 3693 PGY1 and PGY2 residents who participated (2258 male [61.1%]), 1775 (48.1%) reported feeling unprepared for residency. Approximately half of surgery residents took overnight call infrequently (≤2 per month) during their core medical student clerkship (1904 [51.6%]) or their subinternship (1600 [43.3%]); 524 (14.2%) took no call during their core clerkship. In multivariable analysis, residents were more likely to report feeling unprepared for residency if they were female (odds ratio [OR], 1.34; 95% CI, 1.15-1.57) or did not take call as a medical student (OR for 0 vs >4 calls, 2.72; 95% CI, 2.10-3.52). Residents who did not complete a subinternship were less likely to report feeling prepared for residency (OR, 0.68; 95% CI, 0.48-0.96). Feeling adequately prepared for residency was associated with a nearly 2-fold lower risk of experiencing burnout symptoms (OR, 0.57; 95% CI, 0.48-0.68). In interviews, the dominant themes associated with preparedness included the following: (1) various regulations limit the medical school experience, (2) overnight call facilitates preparation and selection of a specialty compatible with their preferences, and (3) adequate perceptions of residency improve expectations, resulting in improved preparedness, lower burnout rates, and lower risk of attrition. Conclusions and Relevance In this cross-sectional study, the perception of feeling unprepared was associated with inadequate exposure to resident responsibilities while in medical school. These findings suggest that effective preparation of medical students for residency may result in lower rates of subsequent burnout.
Collapse
Affiliation(s)
- Kathryn E Engelhardt
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois.,Department of Surgery, Medical University of South Carolina, Charleston
| | - Karl Y Bilimoria
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Julie K Johnson
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - D Brock Hewitt
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois.,Department of Surgery, Thomas Jefferson University, Philadelphia, Pennsylvania
| | - Ryan J Ellis
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Yue Yung Hu
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Jeanette W Chung
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Lindsey Kreutzer
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Remi Love
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Eddie Blay
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois.,Department of Surgery, Temple University, Philadelphia, Pennsylvania
| | - David D Odell
- Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| |
Collapse
|