1
|
Ginzburg SB, Sein AS, Amiel JM, Auerbach L, Cassese T, Konopasek L, Ludwig AB, Meholli M, Ovitsh R, Brenner J. An Examination of Grade Appeals Via a Root Cause Analysis. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2025; 100:666-672. [PMID: 39961086 DOI: 10.1097/acm.0000000000006000] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2025]
Abstract
ABSTRACT Undergraduate medical educators seek to optimize student learning, improve grading transparency and fairness, and provide useful information to residency programs. Recently, the United States Medical Licensing Examination's shift to pass/fail scoring for step 1 disrupted curricular and assessment operations, and schools' tiered grading practices have been scrutinized. In noting that significant institutional time and energy were being expended in addressing the current levels of student grade appeals, 6 public and private medical schools in the Northeastern United States engaged in an examination of grade appeals via a root cause analysis (RCA). From November 2021 to April 2022, the authors reviewed specific instances of grading challenges that the team of educators encountered previously. From May to June 2022, the authors met for a facilitated discussion of the question, "Why are students challenging grading processes and systems or outcomes?" From July to October 2022, the authors identified root causes by analyzing results from the fishbone diagram (process, equipment, materials, people, and environment) and using the "five whys" technique. Several potential explanations for grade appeals and challenging grading systems across institutions were identified, including variability in the quality or experience of evaluators, lack of clarity about the goals and expectations of clerkships and a lack of transparency about the grading process, having a tiered grading system, technical issues with equipment, and clinical productivity demands of faculty. In proposing solutions to root causes identified in the RCA, factors were mapped to Liaison Committee on Medical Education (LCME) standards to facilitate quality and process improvements in grading. Aiming to support the learning environment and a fair and equivalent assessment process, the authors present a novel RCA and LCME method that can contribute to improving grading systems and has the potential to enhance learning and success.
Collapse
|
2
|
Burnside RD, Boothe M, Ledbetter DH, Stalker H, Starostik P, Trapane P. Reconciling competencies in undergraduate medical genetics education: APHMG versus PCME competencies. Genet Med 2025; 27:101448. [PMID: 40289928 DOI: 10.1016/j.gim.2025.101448] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2024] [Revised: 04/14/2025] [Accepted: 04/15/2025] [Indexed: 04/30/2025] Open
Abstract
PURPOSE We wanted to understand whether there were gaps within and/or between the Association of Professors of Human and Medical Genetics (APHMG) and the Association of Pathology Chairs (APC) published competencies for undergraduate medical education pertaining to topics in medical and/or laboratory genetics. METHODS This study compared and contrasted the APHMG and APC competencies related to genetics to identify gaps between and within each to inform the closure of those gaps in undergraduate medical education curriculum development for medical and laboratory genetics at the University of Florida. RESULTS Gaps were identified within and between both documents, many relating to neoplasia for nonheritable cancers and various topics related to laboratory genetics, such as interpretation of results, principles of laboratory diagnostics, and explaining results to others. CONCLUSION APHMG and APC should consider the gaps identified in this study in future updates to their respective competencies. Additionally, medical school curriculum committees may also wish to consider addressing these gaps in the development of medical genetics curricula.
Collapse
Affiliation(s)
- Rachel D Burnside
- Department of Pathology, Immunology, and Laboratory Medicine, University of Florida, Gainesville, FL.
| | - Megan Boothe
- Hayward Telegenetic Center, Division of Pediatric Genetics and Metabolism, Department of Pediatrics, University of Florida, Gainesville, FL
| | - David H Ledbetter
- Office of Research Affairs and Departments of Pediatrics and Psychiatry, University of Florida College of Medicine- Jacksonville, Jacksonville, FL
| | - Heather Stalker
- Hayward Telegenetic Center, Division of Pediatric Genetics and Metabolism, Department of Pediatrics, University of Florida, Gainesville, FL
| | - Petr Starostik
- Department of Pathology, Immunology, and Laboratory Medicine, University of Florida, Gainesville, FL
| | - Pamela Trapane
- Division of Pediatric Genetics, Department of Pediatrics, University of Florida at Jacksonville, Jacksonville, FL
| |
Collapse
|
3
|
Warm E, Hirsh DA, Kinnear B, Besche HC. The Shadow Economy of Effort: Unintended Consequences of Pass/Fail Grading on Medical Students' Clinical Education and Patient Care Skills. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2025; 100:419-424. [PMID: 39773910 DOI: 10.1097/acm.0000000000005973] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/11/2025]
Abstract
ABSTRACT The shift to pass/fail grading in undergraduate medical education was designed to reduce medical students' stress. However, this change has given rise to a "shadow economy of effort," as students move away from traditional didactic and clinical learning to engage in increasing numbers of research, volunteer, and work experiences to enhance their residency applications. These extracurricular efforts to secure a residency position are subphenomena of the hidden curriculum. Medical schools do not officially require all the activities students need to be most competitive for residency selection; therefore, students, as rational actors, participate in the activities they think will most help them succeed.Here, the authors frame residency application and selection as a complex adaptive system (CAS), which self-organizes without centralized control or hierarchical intent. Individuals in a CAS operate in environments marked by volatility, randomness, and uncertainty-all of which are abundant in the residency selection process. Outcomes in such systems, like the development of a shadow economy, are novel, emergent, and cannot always be anticipated. To address these challenges, the authors suggest the need for deep understanding of the system's elements, interrelationships, and dynamics, including feedback loops and emergent properties. Optimizing the results of a CAS requires incentivizing outcomes over activities, ensuring open information flow, and engaging in continuous monitoring and evaluation.The current pass/fail era and resultant shadow economy of effort risk creating a triple harm by devaluing clinical excellence, burning out medical students, and potentially producing superficial or, worse, inauthentic academic and community work. Medical educators must optimize residency application and selection for cooperative outcomes and design incentives to ensure the outputs of medical education align student, institutional, patient, and societal goals. Without a set of predictive "answers," the authors suggest a process of determining actions to advance this ultimate aim and reduce harm.
Collapse
|
4
|
Kelleher M, Schumacher DJ, Zhou C, Kwakye D, Santen SA, Warm E, Kinnear B. Public Board Score Reporting Undermines Holistic Review for Residency Selection. J Gen Intern Med 2025; 40:17-21. [PMID: 39496852 PMCID: PMC11780034 DOI: 10.1007/s11606-024-09133-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/11/2024] [Accepted: 10/09/2024] [Indexed: 11/06/2024]
Abstract
Holistic review has become the gold standard for residency selection. As a result, many programs are de-emphasizing standardized exam scores and other normative metrics. However, if standardized exam scores predict passing of an initial certifying exam, this may lead to an increase in board failure rates within specific residency training programs who do not emphasize test scores on entry. Currently, the board pass rates of residency programs from many of the American Board of Medical Subspecialities (ABMS) are publicly reported as a rolling average. In theory, this should create accountability but may also create pressure and distort the way residency program selects applicants. The risk to programs of having a lower board pass rate publicly reported incentivizes programs to focus increasingly on standardized test scores, threatening holistic review. All programs do not recruit students entering residency with an identical chance of passing boards. Therefore, we believe the ABMS member boards should stop publicly reporting raw certifying exam rates above a certain threshold for normative comparison. We strongly encourage the use of learning analytics to create a residency "expected board pass rate" that would be a better metric for program evaluation and accreditation.
Collapse
Affiliation(s)
- Matthew Kelleher
- Internal Medicine and Pediatrics Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Children's Hospital Medical Center, Cincinnati, USA.
| | - Daniel J Schumacher
- University of Cincinnati College of Medicine/Cincinnati Children's Medical Center, Cincinnati, OH, USA
| | - Christine Zhou
- University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Derek Kwakye
- University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Sally A Santen
- University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Eric Warm
- University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Benjamin Kinnear
- Internal Medicine and Pediatrics Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Children's Hospital Medical Center, Cincinnati, USA
| |
Collapse
|
5
|
Lomis K. Leveraging a Systems Perspective to Empower Health Science Educators in the Age of Augmented Intelligence. MEDICAL SCIENCE EDUCATOR 2024; 34:5-7. [PMID: 40084120 PMCID: PMC11896958 DOI: 10.1007/s40670-024-02236-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 10/29/2024] [Indexed: 03/16/2025]
Affiliation(s)
- Kimberly Lomis
- Medical Education Innovations, American Medical Association, Chicago, IL USA
| |
Collapse
|
6
|
Zeng Y, Yang J, Zhang JW. Post competency training in standardized training of resident physicians and integrated postgraduates. World J Clin Cases 2024; 12:6250-6254. [PMID: 39417052 PMCID: PMC11372526 DOI: 10.12998/wjcc.v12.i29.6250] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Revised: 06/22/2024] [Accepted: 06/28/2024] [Indexed: 08/28/2024] Open
Abstract
We focus on the importance and necessity of post-competency-based training in medical education alongside its current challenges and opportunities. Integrating post competency training into standardized educational frameworks is increasingly recognized as a critical component of preparing residents and postgraduates for real-world clinical practice. Post competency training represents a commitment to excellence in medical education, striving to produce competent, skilled practitioners to meet the challenges of modern healthcare.
Collapse
Affiliation(s)
- Yan Zeng
- Department of Psychology, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China
| | - Jian Yang
- Department of Gastroenterology, The First Affiliated Hospital of Chongqing Medical University, Chongqing 400016, China
| | - Jun-Wen Zhang
- Department of Gastroenterology, The First Affiliated Hospital of Chongqing Medical University, Chongqing 400016, China
| |
Collapse
|
7
|
Busari JO, Diffey L, Hauer KE, Lomis KD, Amiel JM, Barone MA, Schultz K, Chen HC, Damodaran A, Turner DA, Jones B, Oandasan I, Chan MK. Advancing anti-oppression and social justice in healthcare through competency-based medical education (CBME). MEDICAL TEACHER 2024; 46:1167-1174. [PMID: 38215046 DOI: 10.1080/0142159x.2023.2298763] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Accepted: 12/20/2023] [Indexed: 01/14/2024]
Abstract
Competency-based medical education (CBME) focuses on preparing physicians to improve the health of patients and populations. In the context of ongoing health disparities worldwide, medical educators must implement CBME in ways that advance social justice and anti-oppression. In this article, authors describe how CBME can be implemented to promote equity pedagogy, an approach to education in which curricular design, teaching, assessment strategies, and learning environments support learners from diverse groups to be successful. The five core components of CBME programs - outcomes competency framework, progressive sequencing of competencies, learning experiences tailored to learners' needs, teaching focused on competencies, and programmatic assessment - enable individualization of learning experiences and teaching and encourage learners to partner with their teachers in driving their learning. These educational approaches appreciate each learner's background, experiences, and strengths. Using an exemplar case study, the authors illustrate how CBME can afford opportunities to enhance anti-oppression and social justice in medical education and promote each learner's success in meeting the expected outcomes of training. The authors provide recommendations for individuals and institutions implementing CBME to enact equity pedagogy.
Collapse
Affiliation(s)
- Jamiu O Busari
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
- Department of Pediatrics, Dr. Horacio Oduber Hospital, Oranjestad, Aruba
| | - Linda Diffey
- Community Health Sciences, Max Rady College of Medicine, University of Manitoba, Winnipeg, Canada
| | - Karen E Hauer
- University of California, San Francisco School of Medicine, San Francisco, CA, USA
| | | | - Jonathan M Amiel
- Office of Innovation in Health Professions Education and Department of Psychiatry, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA
| | - Michael A Barone
- NBME, Philadelphia, PA, USA
- Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Karen Schultz
- PGME Queen's University, Kingston, Canada
- Department of Family Medicine, Queen's University, Kingston, Canada
| | - H Carrie Chen
- Georgetown University School of Medicine, Washington, DC, USA
| | - Arvin Damodaran
- School of Clinical Medicine, Faculty of Medicine and Health, UNSW Sydney, Sydney, Australia
| | - David A Turner
- Department of Pediatrics, Division of Pediatric Critical Care, Duke Health System, Durham, NC, USA
- Competency-Based Medical Education, American Board of Pediatrics, Chapel Hill, NC, USA
| | - Benjamin Jones
- Health Systems Collaborative, Nuffield Department of Medicine, Oxford, UK
| | - Ivy Oandasan
- Toronto General Hospital Research Institute (TGHRI), Toronto, Canada
| | - Ming-Ka Chan
- Department of Pediatrics & Child Health, Office of Leadership Education, Rady Faculty of Health Sciences and Equity, Diversity, Inclusivity and Social Justice Lead, University of Manitoba and The Children's Hospital of Winnipeg, Winnipeg, Canada
| |
Collapse
|
8
|
Kelleher M, Kinnear B, Weber DE, Knopp MI, Schumacher D, Warm E. Point/counterpoint: Should we stop writing and reading letters of recommendation for residency selection? J Hosp Med 2024; 19:858-862. [PMID: 38923809 DOI: 10.1002/jhm.13440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Revised: 06/11/2024] [Accepted: 06/11/2024] [Indexed: 06/28/2024]
Affiliation(s)
- Matthew Kelleher
- Department of Pediatrics, Division of Hospital Medicine, Internal Medicine and Pediatrics Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA
| | - Benjamin Kinnear
- Department of Pediatrics, Division of Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Childrens Medical Center, Cincinnati, Ohio, USA
| | - Danielle E Weber
- Department of Pediatrics, Division of Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Childrens Medical Center, Cincinnati, Ohio, USA
| | - Michelle I Knopp
- Department of Pediatrics, Division of Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Childrens Medical Center, Cincinnati, Ohio, USA
| | - Daniel Schumacher
- Department of Pediatrics, Division of Emergency Medicine, University of Cincinnati College of Medicine/Cincinnati Childrens Medical Center, Cincinnati, Ohio, USA
| | - Eric Warm
- Department of Internal Medicine and Division of General Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| |
Collapse
|
9
|
Norcini J. On Purpose: The Case for Alignment in Assessment. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:1240-1242. [PMID: 37556812 DOI: 10.1097/acm.0000000000005430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/11/2023]
Abstract
In this issue, Ryan and colleagues underscore the need for criterion-based assessments in the context of competency-based curricula in undergraduate medical education (UME). They also point out that the same scores are often interpreted from a norm-referenced perspective to support the admissions process for residency training. This problem is not unique to UME because in graduate medical education (GME), the same assessments are often used for both decision making and providing feedback. Unfortunately, an assessment with 2 purposes is neither optimal nor efficient for either purpose and may be accompanied by significant side effects. One approach to addressing these challenges is to develop a system of assessment that addresses both purposes but where each component is focused on a single purpose. This leads to alignment and transparency from purpose to test content and from test content to score interpretation and/or feedback. It ensures that the test material is optimized for the task, that individual assessments are constructed to enhance the validity of their scores, and that undesirable side effects are limited.
Collapse
Affiliation(s)
- John Norcini
- J. Norcini is research professor, Department of Psychiatry, SUNY Upstate Medical University, Syracuse, New York; ORCID: https://orcid.org/0000-0002-8464-4115
| |
Collapse
|