1
|
Mascheroni J, Stockburger M, Patwala A, Retzlaff H, Garweg C, Verbelen T, Gallagher AG. Rigor of cardiac device implant training affects performance outcome. Int J Cardiol 2025; 430:133217. [PMID: 40157614 DOI: 10.1016/j.ijcard.2025.133217] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/29/2024] [Revised: 03/07/2025] [Accepted: 03/26/2025] [Indexed: 04/01/2025]
Abstract
BACKGROUND Cardiac Implantable Electronic Devices (CIED) training for novice implanters typically occurs in vivo, varies across institutions, and results in inconsistent skill levels. We previously demonstrated the superior effects of Proficiency-Based Progression (PBP) training on implanters' performance in a randomized controlled trial (RCT). Beforehand, we conducted a 'pilot' study to evaluate the robustness of PBP training, hypothesizing that a substantial majority of trainees would achieve the target proficiency after training. METHODS In this international, prospective, single-arm study, novice implanters completed a metrics-based simulation training curriculum, requiring proficiency benchmark demonstration at each stage to advance. Trainees ultimately performed a cardiac resynchronization therapy implant on virtual reality simulation which was video-recorded and then scored using validated Metrics (Steps, Critical Errors, Errors non-critical) by independent assessors. The primary outcome was the number of trainees meeting the benchmark. Findings informed corrective actions for the subsequent RCT training, whose effectiveness was verified by comparing PBP-trained participants from the pilot study (PBP-PILOT) and the RCT (PBP-RCT) on Steps completed, All Errors, and benchmark achievement. RESULTS Only 17 % achieved the proficiency benchmark. Investigations revealed insufficient faculty adherence to the PBP methodology resulting in premature trainee advancement. Corrective actions improved the subsequent RCT training: compared to the PBP-PILOT group, PBP-RCT trainees completed 15 % more Steps (p = .014), made 69 % fewer All Errors (p = .015), and 93 % met the benchmark (p < .001). CONCLUSIONS Pilot testing revealed deficiencies in PBP training execution, highlighting the importance of rigor and faculty compliance. Verifying novel curricula before large-scale implementation is crucial to ensuring robust training outcomes.
Collapse
Affiliation(s)
- Jorio Mascheroni
- Department of Cardiovascular Sciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium; Department of Cardiac Rhythm Management Training & Education, Medtronic International Trading Sàrl, Route du Molliau 31, 1131 Tolochenaz, Switzerland.
| | - Martin Stockburger
- Department of Cardiology and Internal Medicine, Havelland Kliniken, Ketziner Straße 21, 14641 Nauen, Germany; Institute of Medical Sociology and Rehabilitation Science, Charité - Universitaetsmedizin Berlin, Charitéplatz 1, 10117 Berlin, Germany
| | - Ashish Patwala
- Department of Cardiology, University Hospital of North Midlands, Newcastle Road, Stoke-on-Trent ST4 6QG, United Kingdom
| | - Hartwig Retzlaff
- Training Concept Consulting, Alpenstrasse 14a, D-82194 Groebenzell, Germany
| | - Christophe Garweg
- Department of Cardiovascular Sciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium; Department of Cardiology, UZ Leuven, Herestraat 49, 3000 Leuven, Belgium
| | - Tom Verbelen
- Department of Cardiovascular Sciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium; Department of Cardiac Surgery, UZ Leuven, Herestraat 49, 3000 Leuven, Belgium
| | - Anthony G Gallagher
- Orsi Academy, Proefhoevestraat 12, 9090 Melle, Belgium; Faculty of Medicine, KU Leuven, Herestraat 49, 3000 Leuven, Belgium; School of Medicine, Faculty of Life and Health Sciences, Ulster University, Magee Campus, Northland Rd, Londonderry BT48 7JL, United Kingdom
| |
Collapse
|
2
|
Yanazume S, Kobayashi Y, Furuzono N, Fukuda M, Togami S, Kobayashi H. Validation of objective performance metrics via an intelligent medical network in gynecological oncology robotic surgery. Jpn J Clin Oncol 2025:hyaf031. [PMID: 39969976 DOI: 10.1093/jjco/hyaf031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2024] [Accepted: 02/05/2025] [Indexed: 02/20/2025] Open
Abstract
BACKGROUND Automated performance metrics (APMs) are potentially useful to accurately assess and improve surgeon skills and patient outcomes, while their clinical use is currently limited. We report on the use of the Medicaroid Intelligent Network System (MINS™), a network support system platform used together with the "hinotori" surgical robot system (hinotori™) for the collection of data logs from surgeries, and discuss its potential to improve surgical outcomes. METHODS This study prospectively evaluated the efficacy of MINS™ for collecting data logs in gynecologic oncologic robotic surgery between December 2022 and February 2024 in nine patients. MINS™ regularly communicated with the hinotori™ via a secure network to collect and send system logs to a cloud server, quantifying various performance data for the evaluation of surgical outcomes. RESULTS Clinical data on hinotori™ movement were successfully extracted. The number of operation arm (OA) changes was significantly higher in Patient No. 7, who underwent pelvic lymph node dissection. OA3 monopolar was used more frequently than OA1 bipolar for coagulation (mean: 6.7% vs 2.5%, P ˂0.001). The error count and percentage of inactive time associated with OA collisions decreased dramatically after Patient No. 6, following the version upgrade in July 2024. CONCLUSION MINS™ utilizes technology to connect the hinotori™ to various systems via the Internet, allowing objective evaluations of surgical procedures from data logs. MINS™ is a clinically applicable APM system that objectively analyzes a surgeon's individuality and has the potential to improve surgical techniques and promote standardization.
Collapse
Affiliation(s)
- Shintaro Yanazume
- Faculty of Medicine, Department of Obstetrics & Gynecology, Kagoshima University, 8-35-1 Sakuragaoka, Kagoshima 890-8520, Japan
| | - Yusuke Kobayashi
- Faculty of Medicine, Department of Obstetrics & Gynecology, Kagoshima University, 8-35-1 Sakuragaoka, Kagoshima 890-8520, Japan
| | - Nozomi Furuzono
- Faculty of Medicine, Department of Obstetrics & Gynecology, Kagoshima University, 8-35-1 Sakuragaoka, Kagoshima 890-8520, Japan
| | - Mika Fukuda
- Faculty of Medicine, Department of Obstetrics & Gynecology, Kagoshima University, 8-35-1 Sakuragaoka, Kagoshima 890-8520, Japan
| | - Shinichi Togami
- Faculty of Medicine, Department of Obstetrics & Gynecology, Kagoshima University, 8-35-1 Sakuragaoka, Kagoshima 890-8520, Japan
| | - Hiroaki Kobayashi
- Faculty of Medicine, Department of Obstetrics & Gynecology, Kagoshima University, 8-35-1 Sakuragaoka, Kagoshima 890-8520, Japan
| |
Collapse
|
3
|
Dhillon J, Tanguilig G, Kraeutler MJ. Virtual and Augmented Reality Simulators Show Intraoperative, Surgical Training, and Athletic Training Applications: A Scoping Review. Arthroscopy 2025; 41:505-515. [PMID: 38387769 DOI: 10.1016/j.arthro.2024.02.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Accepted: 02/08/2024] [Indexed: 02/24/2024]
Abstract
PURPOSE To review published literature to identify and evaluate the effect of virtual reality (complete immersion) and augmented reality (overlay of digital information onto the physical world) simulators on intraoperative use for orthopaedic surgeons, orthopaedic surgical education, and athletic training. METHODS A systematic review was performed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to identify studies, published since 2014, that evaluated the role of augmented/virtual reality on intraoperative use for orthopaedic surgeons, orthopaedic surgical education, and athletic training. RESULTS Virtual reality (VR) simulators provide 3-dimensional graphical simulation of the physical world, and augmented reality (AR) simulators overlay digital information onto the physical world. Simulators can include interactive features (i.e., replication of intraoperative bleeding), haptic feedback, and unrestricted task repetition, and they can record, compare, and analyze performance while being easily accessible and eliminating the need for the presence of a mentor or coach. Four studies reported on VR for intraoperative use, 47 studies on surgical education, and 10 studies on athletic training. Two studies revealed the advantages of using VR simulation during intraoperative procedures, specifically showcasing its benefits for elbow arthroscopy, while 2 studies demonstrated similar positive outcomes for hip arthroscopy. Seventeen studies demonstrated that a VR simulator could be a beneficial tool to assist in surgical education for the knee, while 12 studies found that VR simulation is a valuable tool for aiding in surgical education of shoulder arthroscopy. Ten studies demonstrated that VR simulation improves skills in the operating room. Three studies revealed that individuals with more experience exhibit superior performance on these simulators compared to those with less experience. In the realm of athletic training, 10 studies showcased the potential of VR simulation to play a significant role in athletic performance and injury rehabilitation. CONCLUSIONS VR simulation shows benefits in the operating room, is a valuable tool for surgical education resulting in improved skills, and can be used to enhance athletic performance and injury rehabilitation. CLINICAL RELEVANCE Understanding that VR simulators can improve surgical outcomes, surgical skill training, and athletic training and rehabilitation could facilitate development and adoption of this advanced technology.
Collapse
Affiliation(s)
- Jaydeep Dhillon
- Rocky Vista University College of Osteopathic Medicine, Greenwood Village, Colorado, U.S.A
| | - Grace Tanguilig
- Tulane University School of Medicine, New Orleans, Louisiana, U.S.A
| | - Matthew J Kraeutler
- Department of Orthopedics, University of Colorado Anschutz Medical Campus, Aurora, Colorado, U.S.A..
| |
Collapse
|
4
|
Mascheroni J, Stockburger M, Patwala A, Mont L, Rao A, Retzlaff H, Garweg C, Gallagher AG, Verbelen T. Surgical skill simulation training to proficiency reduces procedural errors among novice cardiac device implanters: a randomized study. Europace 2024; 26:euae229. [PMID: 39257213 PMCID: PMC11412769 DOI: 10.1093/europace/euae229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2024] [Revised: 08/15/2024] [Accepted: 09/01/2024] [Indexed: 09/12/2024] Open
Abstract
AIMS In cardiac device implantation, having both surgical skills and ability to manipulate catheter/lead/wire is crucial. Few cardiologists, however, receive formal surgical training prior to implanting. Skills are mostly acquired directly on-the-job and surgical technique varies across institutions; suboptimal approaches may increase complications. We investigated how novel proficiency-based progression (PBP) simulation training impacts the surgical quality of implantations, compared to traditional simulation (SIM) training. METHODS AND RESULTS In this international prospective study, novice implanters were randomized (blinded) 1:1 to participate in a simulation-based procedure training curriculum, with proficiency demonstration requirements for advancing (PBP approach) or without (SIM). Ultimately, trainees performed the surgical tasks of an implant on a porcine tissue that was video-recorded and then scored by two independent assessors (blinded to group), using previously validated performance metrics. Primary outcomes were the number of procedural Steps Completed, Critical Errors, Errors (non-critical), and All Errors Combined. Thirty novice implanters from 10 countries participated. Baseline experiences were similar between groups. Compared to SIM-trained, the PBP-trained group completed on average 11% more procedural Steps (P < 0.001) and made 61.2% fewer Critical Errors (P < 0.001), 57.1% fewer Errors (P = 0.140), and 60.7% fewer All Errors Combined (P = 0.001); 11/15 (73%) PBP trainees demonstrated the predefined target performance level vs. 3/15 SIM trainees (20%) in the video-recorded performance. CONCLUSION Proficiency-based progression training produces superior objectively assessed novice operators' surgical performance in device implantation compared with traditional (simulation) training. Systematic PBP incorporation into formal academic surgical skills training is recommended before in vivo device practice. Future studies will quantify PBP training's effect on surgery-related device complications.
Collapse
Affiliation(s)
- Jorio Mascheroni
- Department of Cardiovascular Sciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
- Department of Cardiac Rhythm Management Training & Education, Medtronic International Trading Sàrl, Route du Molliau 31, 1131 Tolochenaz, Switzerland
| | - Martin Stockburger
- Department of Cardiology and Internal Medicine, Havelland Kliniken, Ketziner Straße 21, 14641 Nauen, Germany
- Institute of Medical Sociology and Rehabilitation Science, Charité—Universitaetsmedizin Berlin, Charitéplatz 1, 10117 Berlin, Germany
| | - Ashish Patwala
- Department of Cardiology, University Hospital of North Midlands, Newcastle Road, ST4 6QG Stoke-on-Trent, UK
| | - Lluís Mont
- Department of Cardiology, Hospital Clinic, Universitat de Barcelona, C/Villarroel 170, 08036 Barcelona, Catalonia, Spain
- Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), c/Rosselló 149, 08036 Barcelona, Catalonia, Spain
- Centro de Investigación Biomédica en Red Cardiovascular (CIBERCV), Av. Monforte de Lemos 3-5, 28029 Madrid, Spain
| | - Archana Rao
- Department of Cardiology, Liverpool Heart and Chest Hospital, Thomas Drive, L14 3PE Liverpool, UK
| | - Hartwig Retzlaff
- Training Concept Consulting, Alpenstrasse 14a, D-82194 Groebenzell, Germany
| | - Christophe Garweg
- Department of Cardiovascular Sciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
- Department of Cardiology, UZ Leuven, Herestraat 49, 3000 Leuven, Belgium
| | - Anthony G Gallagher
- Orsi Academy, Proefhoevestraat 12, 9090 Melle, Belgium
- Faculty of Medicine, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
- School of Medicine, Faculty of Life and Health Sciences, Ulster University, Magee Campus, Northland Rd, BT48 7JL Londonderry, UK
| | - Tom Verbelen
- Department of Cardiovascular Sciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
- Department of Cardiac Surgery, UZ Leuven, Herestraat 49, 3000 Leuven, Belgium
| |
Collapse
|
5
|
Baker JD, Mason G, Bowers Z, Wilson D, Plucknette B, Sabbag C. Implementation of a Hand Training Curriculum in Junior Resident Education: Experience at a Military Orthopedic Residency Program. JOURNAL OF HAND SURGERY GLOBAL ONLINE 2024; 6:545-550. [PMID: 39166195 PMCID: PMC11331213 DOI: 10.1016/j.jhsg.2024.04.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2024] [Accepted: 04/16/2024] [Indexed: 08/22/2024] Open
Abstract
Purpose The American Society for Surgery of the Hand developed the Surgery Training and Educational Platform (STEP) in order to assess essential skills in hand surgery. The American Society for Surgery of the Hand designed modules spanning both osseous and soft tissue skills aimed to be cost effective for the purpose of orthopedic surgical education. The STEP curriculum was adapted and implemented at a single military orthopedic residency program. Methods The following six modules were implemented: (1) depth of plunge, (2) scaphoid pinning, (3) phalangeal fracture pinning, (4) microsurgery, (5) full-thickness skin graft harvest, and (6) wrist arthroscopy. Both first- (PGY1) and second-year (PGY2) residents participated. Scores were calculated according to the original STEP curriculum criteria and were compared with historic data from the previous year. All residents responded to an evaluation questionnaire following the performance of the tasks. Results The PGY2 cohort outperformed PGY1 cohorts across all modules except for the depth of plunge and scaphoid fixation modules. In the phalangeal pinning module, PGY2s did significantly better when compared with pooled PGY1 performance and their own PGY1 performance (P < .05). In the microsurgery module, PGY2s scored better than pooled PGY1s. In the full-thickness skin grafting module, PGY2s outperformed PGY1s (P < .05). On the post-task evaluation, residents unanimously responded that this was a valuable exercise, but the time required to complete all the modules was significant, similar to that of the previous year survey. Conclusions The STEP simulation is a cost effective and reliable program to engage residents in hand surgery-related skills. However, adaptations should be encouraged according to institutional resources to provide the most inclusive training platform possible per institutional constraints. The STEP simulation is interpreted by residents as a valuable exercise but requires a significant time commitment that could be a barrier to implementation and regular use. Type of study/level of evidence Therapeutic IV.
Collapse
Affiliation(s)
- James D. Baker
- Department of Orthopaedic Surgery, Brooke Army Medical Center, San Antonio, TX
| | - Gabriel Mason
- Department of Orthopaedic Surgery, Brooke Army Medical Center, San Antonio, TX
| | - Zachary Bowers
- Department of Orthopaedic Surgery, Brooke Army Medical Center, San Antonio, TX
| | - David Wilson
- Department of Orthopaedic Surgery, Brooke Army Medical Center, San Antonio, TX
| | - Benjamin Plucknette
- Department of Orthopaedic Surgery, Brooke Army Medical Center, San Antonio, TX
| | - Casey Sabbag
- Department of Orthopaedic Surgery, Brooke Army Medical Center, San Antonio, TX
| |
Collapse
|
6
|
Testa EJ, Green A. Neurocognitive Concepts of Arthroscopic Surgical Training. JBJS Rev 2024; 12:01874474-202407000-00008. [PMID: 39028834 DOI: 10.2106/jbjs.rvw.24.00058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/21/2024]
Abstract
» Arthroscopy is used to treat a broad variety of orthopaedic conditions.» The technical aspects of arthroscopic surgery are distinct from traditional open surgery and require different approaches to education and training.» There are neurocognitive concepts related to learning, memory, and performance that are recognized and understood in many fields and relevant but not commonly and specifically considered in orthopaedic surgery.» The purpose of this review was to introduce and discuss neurocognitive principles and concepts of visuospatial motor skill acquisition and proficiency to provide a background to support the development of arthroscopy educational curricula and training.
Collapse
Affiliation(s)
- Edward J Testa
- Department of Orthopaedic Surgery, Alpert Medical School, Brown University, Providence, Rhode Island
| | - Andrew Green
- Division of Shoulder and Elbow Surgery, Department of Orthopaedic Surgery, Alpert Medical School, Brown University, East Providence, Rhode Island
| |
Collapse
|
7
|
Dos Santos Almeida Farinha RJ, Piro A, Mottaran A, Paciotti M, Puliatti S, Breda A, Porter J, Van Cleynenbreugel B, Vander Sloten J, Mottrie A, Gallagher AG. Development and validation of metrics for a new RAPN training model. J Robot Surg 2024; 18:153. [PMID: 38563887 DOI: 10.1007/s11701-024-01911-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2024] [Accepted: 03/13/2024] [Indexed: 04/04/2024]
Abstract
Robot-assisted partial nephrectomy (RAPN) is a complex and index procedure that urologists need to learn how to perform safely. No validated performance metrics specifically developed for a RAPN training model (TM) exist. A Core Metrics Group specifically adapted human RAPN metrics to be used in a newly developed RAPN TM, explicitly defining phases, steps, errors, and critical errors. A modified Delphi meeting concurred on the face and content validation of the new metrics. One hundred percent consensus was achieved by the Delphi panel on 8 Phases, 32 Steps, 136 Errors and 64 Critical Errors. Two trained assessors evaluated recorded video performances of novice and expert RAPN surgeons executing an emulated RAPN in the newly developed TM. There were no differences in procedure Steps completed by the two groups. Experienced RAPN surgeons made 34% fewer Total Errors than the Novice group. Performance score for both groups was divided at the median score using Total Error scores, into HiError and LoError subgroups. The LowErrs Expert RAPN surgeons group made 118% fewer Total Errors than the Novice HiErrs group. Furthermore, the LowErrs Expert RAPN surgeons made 77% fewer Total Errors than the HiErrs Expert RAPN surgeons. These results established construct and discriminative validity of the metrics. The authors described a novel RAPN TM and its associated performance metrics with evidence supporting their face, content, construct, and discriminative validation. This report and evidence support the implementation of a simulation-based proficiency-based progression (PBP) training program for RAPN.
Collapse
Affiliation(s)
| | - Adele Piro
- Division of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Angelo Mottaran
- Division of Urology, IRCCS Azienda Ospedaliero - Universitaria di Bologna, Bologna, Italy
| | - Marco Paciotti
- Department of Urology, IRCCS Humanitas Research Hospital, Rozzano, Milan, Italy
| | - Stefano Puliatti
- Division of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Alberto Breda
- Department of Urology, Universitat Autonoma de Barcelona, Fundació Puigvert, Barcelona, Spain
| | - James Porter
- Swedish Urology Group, Swedish Medical Center, Seattle, WA, USA
| | - Ben Van Cleynenbreugel
- Department of Urology, University Hospitals Leuven, Leuven, Belgium
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
| | - Jos Vander Sloten
- Department of Mechanical Engineering, Section of Biomechanics, KU Leuven, Leuven, Belgium
| | - Alexandre Mottrie
- Orsi Academy, Proefhoevestraat 12, 9090, Ghent, Belgium
- Department of Urology, Onze-Lieve-Vrouw Ziekenhuis, Aalst, Belgium
| | - Anthony G Gallagher
- Orsi Academy, Proefhoevestraat 12, 9090, Ghent, Belgium
- Faculty of Medicine, KU Leuven, Leuven, Belgium
- Faculty of Life and Health Sciences, Ulster University, Derry, Northern Ireland, UK
| |
Collapse
|
8
|
Testa EJ, Fadale PD. Arthroscopic Training: Historical Insights and Future Directions. J Am Acad Orthop Surg 2023; 31:1180-1188. [PMID: 37703548 DOI: 10.5435/jaaos-d-23-00254] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 08/03/2023] [Indexed: 09/15/2023] Open
Abstract
Arthroscopy is an orthopaedic technique that provides surgical solutions in a minimally invasive fashion. Since its introduction, arthroscopy has become a preferred surgical approach for treating various orthopaedic pathologies, such as meniscal tears, anterior cruciate ligament ruptures, rotator cuff tears, and wrist, elbow, ankle, and hip conditions. Despite its ubiquity, surgical training in arthroscopy poses several challenges for educators and trainees. Arthroscopy involves neuromotor skills which differ from those of open surgery, such as the principles of triangulation, bimanual dexterity, and the ability to navigate a three-dimensional space on a two-dimensional screen. There remains no universally implemented curriculum for arthroscopic education within orthopaedic residency or fellowship training programs, permitting the potential for highly variable training experiences from institution to institution. Therefore, the current review seeks to highlight the history of arthroscopic education, strategies and current teaching modalities in modern arthroscopic education, and avenues for future educational pathways.
Collapse
Affiliation(s)
- Edward J Testa
- From the Department of Orthopaedic Surgery, Brown University, Warren Alpert Medical School, Providence, RI
| | | |
Collapse
|
9
|
Ind T. Providing a standardised educational programme in robot-assisted gynaecological surgery. Best Pract Res Clin Obstet Gynaecol 2023; 91:102399. [PMID: 37651956 DOI: 10.1016/j.bpobgyn.2023.102399] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 07/09/2023] [Accepted: 07/21/2023] [Indexed: 09/02/2023]
Abstract
Standardisation of an educational programme in robotic gynaecological surgery requires careful reflection to ensure that the correct surgeons are selected, that they are trained to the best of their ability, and that they have continued education into their careers. The generally agreed pathways included a proficiency-based progression model for procedures with validated assessment tools used for both formative and summative assessment. For new surgeons, a basic and advanced curriculum is required, involving tools on how to use the instruments as well as educational lectures and simulation. For advanced learning, there is a need for proctorship. To maintain their skills, a surgeon should demonstrate a reflective practice and continued good outcomes while adhering to a process of credentialing. Trainers should be validated on their ability to teach based on recognised training-the-trainers courses.
Collapse
Affiliation(s)
- Thomas Ind
- Department of Gynaecological Oncology, Royal Marsden Hospital, London, SW3 6JA, UK.
| |
Collapse
|
10
|
Bakhaidar M, Alsayegh A, Yilmaz R, Fazlollahi AM, Ledwos N, Mirchi N, Winkler-Schwartz A, Luo L, Del Maestro RF. Performance in a Simulated Virtual Reality Anterior Cervical Discectomy and Fusion Task: Disc Residual, Rate of Removal, and Efficiency Analyses. Oper Neurosurg (Hagerstown) 2023; 25:e196-e205. [PMID: 37441799 DOI: 10.1227/ons.0000000000000813] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2023] [Accepted: 05/05/2023] [Indexed: 07/15/2023] Open
Abstract
BACKGROUND AND OBJECTIVES Anterior cervical discectomy and fusion (ACDF) is among the most common spine procedures. The Sim-Ortho virtual reality simulator platform contains a validated ACDF simulated task for performance assessment. This study aims to develop a methodology to extract three-dimensional data and reconstruct and quantitate specific simulated disc tissues to generate novel metrics to analyze performance metrics of skilled and less skilled participants. METHODS We used open-source platforms to develop a methodology to extract three-dimensional information from ACDF simulation data. Metrics generated included, efficiency index, disc volumes removed from defined regions, and rate of tissue removal from superficial, central, and deep disc regions. A pilot study was performed to assess the utility of this methodology to assess expertise during the ACDF simulated procedure. RESULTS The system outlined, extracts data allowing the development of a methodology which accurately reconstructs and quantitates 3-dimensional disc volumes. In the pilot study, data sets from 27 participants, divided into postresident, resident, and medical student groups, allowed assessment of multiple novel metrics, including efficiency index (surgical time spent in actively removing disc), where the postresident group spent 61.8% of their time compared with 53% and 30.2% for the resident and medical student groups, respectively ( P = .01). During the annulotomy component, the postresident group removed 47.4% more disc than the resident groups and 102% more than the medical student groups ( P = .03). CONCLUSION The methodology developed in this study generates novel surgical procedural metrics from 3-dimensional data generated by virtual reality simulators and can be used to assess surgical performance.
Collapse
Affiliation(s)
- Mohamad Bakhaidar
- Department of Neurology and Neurosurgery, Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
- Department of Neurology and Neurosurgery, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
- Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Ahmad Alsayegh
- Department of Neurology and Neurosurgery, Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
- Department of Neurology and Neurosurgery, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
- Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Recai Yilmaz
- Department of Neurology and Neurosurgery, Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
| | - Ali M Fazlollahi
- Department of Neurology and Neurosurgery, Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
| | - Nicole Ledwos
- Department of Neurology and Neurosurgery, Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
| | - Nykan Mirchi
- Department of Neurology and Neurosurgery, Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
| | - Alexander Winkler-Schwartz
- Department of Neurology and Neurosurgery, Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
- Department of Neurology and Neurosurgery, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
| | - Lucy Luo
- Department of Neurology and Neurosurgery, Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
- Department of Orthopedic Surgery, McGill University Health Centre, Montreal, Quebec, Canada
| | - Rolando F Del Maestro
- Department of Neurology and Neurosurgery, Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
- Department of Neurology and Neurosurgery, Montreal Neurological Hospital and Institute, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
11
|
Anetzberger H, Kugler A, John D, Kopf S, Becker R, Reppenhagen S. The number of arthroscopies performed by trainees does not deduce the level of their arthroscopic proficiency. Knee Surg Sports Traumatol Arthrosc 2023; 31:4231-4238. [PMID: 37296326 DOI: 10.1007/s00167-023-07471-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 05/30/2023] [Indexed: 06/12/2023]
Abstract
PURPOSE It is reasonable to question whether the case volume is a suitable proxy for the manual competence of an arthroscopic surgeon. The aim of this study was to evaluate the correlation between the number of arthroscopies previously performed and the arthroscopic skills acquired using a standardized simulator test. METHODS A total of 97 resident and early orthopaedic surgeons who participated in arthroscopic simulator training courses were divided into five groups based on their self-reported number of arthroscopic surgeries: (1) none, (2) < 10, (3) 10 to 19, (4) 20 to 39 and (5) 40 to 100. Arthroscopic manual skills were evaluated with a simulator by means of the diagnostic arthroscopy skill score (DASS) before and after training. Seventy-five points out of 100 must be achieved to pass the test. RESULTS In the pretest, only three trainees in group 5 passed the arthroscopic skill test, and all other participants failed. Group 5 (57 ± 17 points; n = 17) scored significantly higher than the other groups (group 1: 30 ± 14, n = 20; group 2: 35 ± 14, n = 24; group 3: 35 ± 18, n = 23; and group 4: 33 ± 17, n = 13). After a two-day simulator training, trainees showed a significant increase in performance. In group 5, participants scored 81 ± 17 points, which was significantly higher than the other groups (group 1: 75 ± 16; group 2: 75 ± 14; group 3: 69 ± 15; and group 4: 73 ± 13). While self-reported arthroscopic procedures were n.s. associated with higher log odds of passing the test (p = 0.423), the points scored in the pretest were found to be a good predictor of whether a trainee would pass the test (p < 0.05). A positive correlation was observed between the points scored in the pretest and the posttest (p < 0.05, r = 0.59, r2 = 0.34). CONCLUSIONS The number of previously performed arthroscopies is not a reliable indicator of the skills level of orthopaedic residents. A reasonable alternative in the future would be to verify arthroscopic proficiency on the simulator by means of a score as a pass-fail examination. LEVEL OF EVIDENCE III.
Collapse
Affiliation(s)
- Hermann Anetzberger
- Orthopädische Gemeinschaftspraxis am OEZ, Hanauer Str. 65, 80993, Munich, Germany.
| | - Andreas Kugler
- Zentrum für Gelenkchirurgie im MVZ am Nordbad, Schleißheimer Straße 130, 80797, Munich, Germany
| | - Dominik John
- Gelenk.Bonn, Bonner Talweg 61, 53113, Bonn, Germany
| | - Sebastian Kopf
- Zentrum für Orthopädie und Unfallchirurgie, Universitätsklinikum Brandenburg der Medizinischen Hochschule Brandenburg Theodor Fontane, Hochstraße 29, 14770, Brandenburg an der Havel, Germany
| | - Roland Becker
- Zentrum für Orthopädie und Unfallchirurgie, Universitätsklinikum Brandenburg der Medizinischen Hochschule Brandenburg Theodor Fontane, Hochstraße 29, 14770, Brandenburg an der Havel, Germany
| | - Stephan Reppenhagen
- Orthopädische Klinik König-Ludwig-Haus, Brettreichstraße 11, 97074, Würzburg, Germany
| |
Collapse
|
12
|
De Groote R, Puliatti S, Amato M, Mazzone E, Larcher A, Farinha R, Paludo A, Desender L, Hubert N, Cleynenbreugel BV, Bunting BP, Mottrie A, Gallagher AG, Rosiello G, Uvin P, Decoene J, Tuyten T, D’Hondt M, Chatzopoulos C, De Troyer B, Turri F, Dell’Oglio P, Liakos N, Andrea Bravi C, Lambert E, Andras I, Di Maida F, Everaerts W. Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial. ANNALS OF SURGERY OPEN 2023; 4:e307. [PMID: 37746611 PMCID: PMC10513364 DOI: 10.1097/as9.0000000000000307] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Accepted: 06/03/2023] [Indexed: 09/26/2023] Open
Abstract
Objective To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity. Background GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations. Methods In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves. Results The PBP group made 42% fewer objectively assessed performance errors than the conventional group (P < 0.001) and scored 15% better on the GEARS assessment (P = 0.033). The mean interrater reliability for binary metrics and GEARS was 0.87 and 0.38, respectively. Binary total error metrics AUC was 97% and for GEARS 85%. With a sensitivity threshold of 0.8, false positives rates were 3% and 25% for, respectively, the binary and GEARS assessments. Conclusions Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.
Collapse
Affiliation(s)
- Ruben De Groote
- From the ORSI Academy, Ghent, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
| | - Stefano Puliatti
- From the ORSI Academy, Ghent, Belgium
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Marco Amato
- From the ORSI Academy, Ghent, Belgium
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Elio Mazzone
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy
- Vita-Salute San Raffaele University, Milan, Italy
| | - Alessandro Larcher
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy
- Vita-Salute San Raffaele University, Milan, Italy
| | | | - Artur Paludo
- From the ORSI Academy, Ghent, Belgium
- Clinic Hospital of Porto Alegre, Urology, Porto Alegre, Brazil
| | - Liesbeth Desender
- Department of Thoracovascular Surgery, University Hospital Ghent, Ghent, Belgium
| | - Nicolas Hubert
- Department of Urology, CHR de la Citadelle, Liège, Belgium
| | | | - Brendan P. Bunting
- School of Psychology, Ulster University, Coleraine, Northern Ireland, United Kingdom
| | - Alexandre Mottrie
- From the ORSI Academy, Ghent, Belgium
- Department of Urology, OLV, Aalst, Belgium
| | - Anthony G. Gallagher
- From the ORSI Academy, Ghent, Belgium
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- School of Medicine, Faculty of Life and Health Sciences, Ulster University, Northern Ireland, United Kingdom
| | - Giuseppe Rosiello
- From the ORSI Academy, Ghent, Belgium
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy
- Vita-Salute San Raffaele University, Milan, Italy
| | - Pieter Uvin
- Department of Urology, AZ Sint-Jan, Bruges, Belgium
| | - Jasper Decoene
- Department of Urology, OLV van Lourdes Hospital, Waregem, Belgium
| | - Tom Tuyten
- Department of Urology, Jessa Hospital, Hasselt, Belgium
| | | | | | - Bart De Troyer
- Department of Urology, AZ Nikolaas, Sint-Niklaas, Belgium
| | - Filippo Turri
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | | | - Nikolaos Liakos
- Prostate Center Northwest, Department of Urology, Pediatric Urology and Uro-Oncology, St. Antonius-Hospital, Gronau, Germany
| | - Carlo Andrea Bravi
- From the ORSI Academy, Ghent, Belgium
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy
- Vita-Salute San Raffaele University, Milan, Italy
| | | | - Iulia Andras
- Department of Urology, Iuliu Hatieganu University of Medicine and Pharmacy, Cluj-Napoca, Romania
| | | | - Wouter Everaerts
- Department of Cellular and Molecular Medicine, KU Leuven, Leuven, Belgium
| |
Collapse
|
13
|
McCarthy N, Neville K, Pope A, Barry L, Livingstone V. Effectiveness of a proficiency-based progression e-learning approach to training in communication in the context of clinically deteriorating patients: a multi-arm randomised controlled trial. BMJ Open 2023; 13:e072488. [PMID: 37536965 PMCID: PMC10401258 DOI: 10.1136/bmjopen-2023-072488] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 06/08/2023] [Indexed: 08/05/2023] Open
Abstract
OBJECTIVE To determine the effectiveness of proficiency-based progression (PBP) e-learning in training in communication concerning clinically deteriorating patients. DESIGN Single-centre multi-arm randomised double-blind controlled trial with three parallel arms. RANDOMISATION, SETTING AND PARTICIPANTS A computer-generated program randomised and allocated 120 final year medical students in an Irish University into three trial groups. INTERVENTION Each group completed the standard Identification, Situation, Background, Assessment, Recommendation communication e-learning; group 1 Heath Service Executive course group (HSE) performed this alone; group 2 (PBP) performed additional e-learning using PBP scenarios with expert-determined proficiency benchmarks composed of weighted marking schemes of steps, errors and critical errors cut-offs; group 3 (S) (self-directed, no PBP) performed additional e-learning with identical scenarios to (PBP) without PBP. MAIN OUTCOME MEASURES Primary analysis was based on 114 students, comparing ability to reach expert-determined predefined proficiency benchmark in standardised low-fidelity simulation assessment, before and after completion of each group's e-learning requirements. Performance was recorded and scored by two independent blinded assessors. RESULTS Post-intervention, proficiency in each group in the low-fidelity simulation environment improved with statistically significant difference in proficiency between groups (p<0.001). Proficiency was highest in (PBP) (81.1%, 30/37). Post hoc pairwise comparisons revealed statistically significant differences between (PBP) and self-directed (S) (p<0.001) and (HSE) (p<0.001). No statistically significant difference existed between (S) and (HSE) (p=0.479). Changes in proficiency from pre-intervention to post-intervention were significantly different between the three groups (p=0.001). Post-intervention, an extra 67.6% (25/37) in (PBP) achieved proficiency in the low-fidelity simulation. Post hoc pairwise comparisons revealed statistically significant differences between (PBP) and both (S) (p=0.020) and (HSE) (p<0.001). No statistically significant difference was found between (S) and (HSE) (p=0.156). CONCLUSIONS PBP e-learning is a more effective way to train in communication concerning clinically deteriorating patients than standard e-learning or e-learning without PBP. TRIAL REGISTRATION NUMBER NCT02937597.
Collapse
Affiliation(s)
- Nora McCarthy
- Medical Education Unit, School of Medicine, University College Cork, Cork, Ireland
| | - Karen Neville
- Department of Business Information Systems, Cork University Business School, University College Cork, Cork, Ireland
| | - Andrew Pope
- Department of Business Information Systems, Cork University Business School, University College Cork, Cork, Ireland
| | - Lee Barry
- ESA-BIC, Tyndall Institute, University College Cork National University of Ireland, Cork, Ireland
| | - Vicki Livingstone
- INFANT Centre, University College Cork National University of Ireland, Cork, Ireland
| |
Collapse
|
14
|
Farinha R, Breda A, Porter J, Mottrie A, Van Cleynenbreugel B, Vander Sloten J, Mottaran A, Gallagher AG. Objective assessment of intraoperative skills for robot-assisted partial nephrectomy (RAPN). J Robot Surg 2023; 17:1401-1409. [PMID: 36689078 PMCID: PMC10374474 DOI: 10.1007/s11701-023-01521-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Accepted: 01/02/2023] [Indexed: 01/24/2023]
Abstract
RAPN training usually takes place in-vivo and methods vary across countries/institutions. No common system exists to objectively assess trainee capacity to perform RAPN at predetermined performance levels prior to in-vivo practice. The identification of objective performance metrics for RAPN training is a crucial starting point to improve training and surgical outcomes. The authors sought to examine the reliability, construct and discriminative validity of objective intraoperative performance metrics which best characterize the optimal and suboptimal performance of a reference approach for training novice RAPN surgeons. Seven Novice and 9 Experienced RAPN surgeons video recorded one or two independently performed RAPN procedures in the human. The videos were anonymized and two experienced urology surgeons were trained to reliably score RAPN performance, using previously developed metrics. The assessors were blinded to the performing surgeon, hospital and surgeon group. They independently scored surgeon RAPN performance. Novice and Experienced group performance scores were compared for procedure steps completed and errors made. Each group was divided at the median for Total Errors score, and subgroup scores (i.e., Novice HiErrs and LoErrs, Experienced HiErrs and LoErrs) were compared. The mean inter-rater reliability (IRR) for scoring was 0.95 (range 0.84-1). Compared with Novices, Experienced RAPN surgeons made 69% fewer procedural Total Errors. This difference was accentuated when the LoErr Expert RAPN surgeon's performance was compared with the HiErrs Novice RAPN surgeon's performance with an observed 170% fewer Total Errors. GEARS showed poor reliability (Mean IRR = 0.44; range 0.0-0.8), for scoring RAPN surgical performance. The RAPN procedure metrics reliably distinguish Novice and Experienced surgeon performances. They further differentiated performance levels within a group with similar experiences. Reliable and valid metrics will underpin quality-assured novice RAPN surgical training.
Collapse
Affiliation(s)
- Rui Farinha
- Orsi Academy, Proefhoevestraat 12, Melle, 9090, Ghent, Belgium.
- Department of Urology, Onze-Lieve-Vrouw Ziekenhuis, Aalst, Belgium.
- Department of Urology, São José Hospital, Lisbon, Portugal.
| | - Alberto Breda
- Department of Urology, Fundació Puigvert, Universitat Autonoma de Barcelona, Barcelona, Spain
| | - James Porter
- Swedish Urology Group, Swedish Medical Center, Seattle, WA, USA
| | - Alexandre Mottrie
- Orsi Academy, Proefhoevestraat 12, Melle, 9090, Ghent, Belgium
- Department of Urology, Onze-Lieve-Vrouw Ziekenhuis, Aalst, Belgium
| | - Ben Van Cleynenbreugel
- Department of Urology, University Hospitals Leuven, Louvain, Belgium
- Department of Development and Regeneration, KU Leuven, Louvain, Belgium
| | - Jozef Vander Sloten
- Department of Mechanical Engineering, Section of Biomechanics, KU Leuven, Louvain, Belgium
| | - Angelo Mottaran
- Division of Urology, IRCCS Azienda Ospedaliero-Universitaris di Bologna, Bologna, Italy
- University of Bologna, Bologna, Italy
| | - Anthony G Gallagher
- Orsi Academy, Proefhoevestraat 12, Melle, 9090, Ghent, Belgium
- Faculty of Medicine, KU Leuven, Louvain, Belgium
- Faculty of Life and Health Sciences, Ulster University, Derry, Northern Ireland, UK
| |
Collapse
|
15
|
Mascheroni J, Stockburger M, Patwala A, Mont L, Rao A, Retzlaff H, Garweg C, Verbelen T, Gallagher AG. Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial. JAMA Netw Open 2023; 6:e2322750. [PMID: 37651144 PMCID: PMC10472192 DOI: 10.1001/jamanetworkopen.2023.22750] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Accepted: 05/22/2023] [Indexed: 09/01/2023] Open
Abstract
Importance In cardiac device implant training, there is no common system to objectively assess trainees' ability to perform tasks at predetermined performance levels before in vivo practice; therefore, patients are potentially exposed to risks related to operators' early learning curve. Objective To assess the effect on implant performance quality of novel metrics-based simulation training to proficiency (proficiency-based progression [PBP]) vs traditional simulation-based training (SBT). Design, Setting, and Participants In this prospective randomized trial, conducted between March 8, 2022 and November 24, 2022, novice implanters were randomized (blinded) 1:1 to participate in an SBT curriculum (procedural knowledge e-learning and in-person simulation training) at an international skills training center, with proficiency demonstration requirements at each training stage for advancing (PBP approach) or without the requirements. Ultimately, trainees performed a cardiac resynchronization therapy (CRT) implant using virtual reality simulation. The procedure was video-recorded and subsequently scored using previously validated metrics by 2 independent assessors blinded to group. Physicians who had already implanted more than 20 pacemakers or defibrillators and fewer than 200 CRT systems as the first operator were eligible. Thirty-two implanters from 10 countries voluntarily enrolled in the training program and were randomized; 30 (15 per group) started and completed training. Data analysis was performed from November 27 to December 22, 2022. Intervention Training with PBP vs SBT. Main Outcome and Measures The primary outcome comprised 4 objectively assessed performance metrics derived from the video-recordings: number of procedural steps completed, errors, critical errors, and all errors combined. Results Baseline experience of the 30 participants (19 [63%] male; mean [SD] number of years in implant practice, 2.0 [1.8]; median [IQR] number of implanted pacemakers or defibrillators, 47.5 [30.0-115.0]; median [IQR] number of implanted CRT systems, 3.0 [1.25-10.0]) was similar between study groups. Compared with the SBT group, the PBP group completed 27% more procedural steps (median [IQR], 31 [30-32] vs 24 [22-27]; P < .001) and made 73% fewer errors (median [IQR], 2 [1-3] vs 7 [5-8]; P < .001), 84% fewer critical errors (median [IQR], 1 [0-1] vs 3 [3-5]; P < .001), and 77% fewer all errors combined (errors plus critical errors) (median [IQR], 3 [1-3] vs 11 [8-12]; P < .001); 14 of the 15 PBP trainees (93%) demonstrated the predefined target performance level vs 0 of the 15 SBT trainees. Conclusions and Relevance In this randomized trial, the PBP approach to novice implanter training generated superior objectively assessed performance vs SBT. If implemented broadly and systematically, PBP training may ensure safe and effective performance standards before trainees proceed to (supervised) in vivo practice. Future studies are needed to verify implications on procedure-related patient complications. Trial Registration ClinicalTrials.gov Identifier: NCT05952908.
Collapse
Affiliation(s)
- Jorio Mascheroni
- Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium
- Department of Cardiac Rhythm Management Training and Education, Medtronic International Trading Sàrl, Tolochenaz, Switzerland
| | - Martin Stockburger
- Department of Cardiology and Internal Medicine, Havelland Kliniken, Nauen, Germany
- Department of Cardiology and Angiology, Charité–Universitätsmedizin Berlin, Berlin, Germany
| | - Ashish Patwala
- Department of Cardiology, University Hospital of North Midlands, Stoke-on-Trent, United Kingdom
| | - Lluís Mont
- Department of Cardiology, Hospital Clinic, Universitat de Barcelona, Barcelona, Catalonia, Spain
- Institut d’Investigacions Biomèdiques August Pi i Sunyer, Barcelona, Catalonia, Spain
- Centro de Investigación Biomédica en Red Cardiovascular, Madrid, Spain
| | - Archana Rao
- Department of Cardiology, Liverpool Heart and Chest Hospital, Liverpool, United Kingdom
| | | | - Christophe Garweg
- Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium
- Department of Cardiology, UZ Leuven, Leuven, Belgium
| | - Tom Verbelen
- Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium
- Department of Cardiac Surgery, UZ Leuven, Leuven, Belgium
| | - Anthony G. Gallagher
- Orsi Academy, Melle, Belgium
- Faculty of Medicine, KU Leuven, Leuven, Belgium
- School of Medicine, Faculty of Life and Health Sciences, Ulster University, Londonderry, United Kingdom
| |
Collapse
|
16
|
D’Andrea M, Musio A, Colasanti R, Mongardi L, Fuschillo D, Lofrese G, Tosatto L. A novel, reusable, realistic neurosurgical training simulator for cerebrovascular bypass surgery: Iatrotek ® bypass simulator validation study and literature review. Front Surg 2023; 10:1048083. [PMID: 36843992 PMCID: PMC9947354 DOI: 10.3389/fsurg.2023.1048083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 01/18/2023] [Indexed: 02/11/2023] Open
Abstract
Background Microanastomosis is a challenging technique requiring continuous training to be mastered. Several models have been proposed, but few effectively reflect a real bypass surgery; even fewer are reusable, most are not easily accessible, and the setting is often quite long. We aim to validate a simplified, ready-to-use, reusable, ergonomic bypass simulator. Methods Twelve novice and two expert neurosurgeons completed eight End-to-End (EE), eight End-to-Side (ES), and eight Side-to-Side (SS) microanastomoses using 2-mm synthetic vessels. Data on time to perform bypass (TPB), number of sutures and time required to stop potential leaks were collected. After the last training, participants completed a Likert Like Survey for bypass simulator evaluation. Each participant was assessed using the Northwestern Objective Microanastomosis Assessment Tool (NOMAT). Results When comparing the first and last attempts, an improvement of the mean TPB was registered in both groups for the three types of microanastomosis. The improvement was always statistically significant in the novice group, while in the expert group, it was only significant for ES bypass. The NOMAT score improved in both groups, displaying statistical significance in the novices for EE bypass. The mean number of leakages, and the relative time for their resolution, also tended to progressively reduce in both groups by increasing the attempts. The Likert score expressed by the experts was slightly higher (25 vs. 24.58 by the novices). Conclusions Our proposed bypass training model may represent a simplified, ready-to-use, reusable, ergonomic, and efficient system to improve eye-hand coordination and dexterity in performing microanastomoses.
Collapse
Affiliation(s)
- Marcello D’Andrea
- Department of Neurosurgery, Maurizio Bufalini Hospital, Cesena, Italy
| | - Antonio Musio
- Department of Ferrara – Neurosurgery, Sant ‘Anna University Hospital, Ferrara, Italy,Correspondence: Antonio Musio
| | | | - Lorenzo Mongardi
- Department of Ferrara – Neurosurgery, Sant ‘Anna University Hospital, Ferrara, Italy
| | - Dalila Fuschillo
- Department of Neurosurgery, Maurizio Bufalini Hospital, Cesena, Italy
| | - Giorgio Lofrese
- Department of Neurosurgery, Maurizio Bufalini Hospital, Cesena, Italy
| | - Luigino Tosatto
- Department of Neurosurgery, Maurizio Bufalini Hospital, Cesena, Italy
| |
Collapse
|
17
|
Younes MM, Larkins K, To G, Burke G, Heriot A, Warrier S, Mohan H. What are clinically relevant performance metrics in robotic surgery? A systematic review of the literature. J Robot Surg 2022; 17:335-350. [PMID: 36190655 PMCID: PMC10076398 DOI: 10.1007/s11701-022-01457-y] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Accepted: 09/17/2022] [Indexed: 10/10/2022]
Abstract
A crucial element of any surgical training program is the ability to provide procedure-specific, objective, and reliable measures of performance. During robotic surgery, objective clinically relevant performance metrics (CRPMs) can provide tailored contextual feedback and correlate with clinical outcomes. This review aims to define CRPMs, assess their validity in robotic surgical training and compare CRPMs to existing measures of robotic performance. A systematic search of Medline and Embase databases was conducted in May 2022 following the PRISMA guidelines. The search terms included Clinically Relevant Performance Metrics (CRPMs) OR Clinically Relevant Outcome Measures (CROMs) AND robotic surgery. The study settings, speciality, operative context, study design, metric details, and validation status were extracted and analysed. The initial search yielded 116 citations, of which 6 were included. Citation searching identified 3 additional studies, resulting in 9 studies included in this review. Metrics were defined as CRPMs, CROMs, proficiency-based performance metrics and reference-procedure metrics which were developed using a modified Delphi methodology. All metrics underwent both contents and construct validation. Two studies found a strong correlation with GEARS but none correlated their metrics with patient outcome data. CRPMs are a validated and objective approach for assessing trainee proficiency. Evaluating CRPMs with other robotic-assessment tools will facilitate a multimodal metric evaluation approach to robotic surgery training. Further studies should assess the correlation with clinical outcomes. This review highlights there is significant scope for the development and validation of CRPMs to establish proficiency-based progression curricula that can be translated from a simulation setting into clinical practice.
Collapse
Affiliation(s)
- Melissa M Younes
- The University of Melbourne, 305 Grattan Street, Parkville, VIC, Australia
| | - Kirsten Larkins
- The University of Melbourne, 305 Grattan Street, Parkville, VIC, Australia. .,Peter MacCallum Cancer Centre, Melbourne, VIC, Australia.
| | - Gloria To
- The University of Melbourne, 305 Grattan Street, Parkville, VIC, Australia
| | - Grace Burke
- International Medical Robotics Academy, North Melbourne, VIC, Australia
| | - Alexander Heriot
- The University of Melbourne, 305 Grattan Street, Parkville, VIC, Australia.,Peter MacCallum Cancer Centre, Melbourne, VIC, Australia.,International Medical Robotics Academy, North Melbourne, VIC, Australia
| | - Satish Warrier
- The University of Melbourne, 305 Grattan Street, Parkville, VIC, Australia.,Peter MacCallum Cancer Centre, Melbourne, VIC, Australia.,International Medical Robotics Academy, North Melbourne, VIC, Australia.,Monash University, Clayton, VIC, Australia
| | - Helen Mohan
- The University of Melbourne, 305 Grattan Street, Parkville, VIC, Australia.,Peter MacCallum Cancer Centre, Melbourne, VIC, Australia.,Austin Health, Heidelberg, VIC, Australia
| |
Collapse
|
18
|
De Groote R, Puliatti S, Amato M, Mazzone E, Rosiello G, Farinha R, Paludo A, Desender L, Van Cleynenbreugel B, Bunting BP, Mottrie A, Gallagher AG. Proficiency-based progression training for robotic surgery skills training: a randomized clinical trial. BJU Int 2022; 130:528-535. [PMID: 37382230 DOI: 10.1111/bju.15811] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/26/2023]
Abstract
OBJECTIVE To determine whether proficiency-based progression (PBP) training leads to better robotic surgical performance compared to traditional training (TT), given that the value of PBP training for learning robotic surgical skills is unclear. MATERIALS AND METHODS The PROVESA trial is a multicentric, prospective, randomized and blinded clinical study comparing PBP training with TT for robotic suturing and knot-tying anastomosis skills. A total of 36 robotic surgery-naïve junior residents were recruited from 16 training sites and 12 residency training programmes. Participants were randomly allocated to metric-based PBP training or the current standard of care TT, and compared at the end of training. The primary outcome was percentage of participants reaching the predefined proficiency benchmark. Secondary outcomes were the numbers of procedure steps and errors made. RESULTS Of the group that received TT, 3/18 reached the proficiency benchmark versus 12/18 of the PBP group (i.e. the PBP group were ~10 times as likely to demonstrate proficiency [P = 0.006]). The PBP group demonstrated a 51% reduction in number of performance errors from baseline to the final assessment (18.3 vs 8.9). The TT group demonstrated a marginal improvement (15.94 vs 15.44) in errors made. CONCLUSIONS The PROVESA trial is the first prospective randomized controlled trial on basic skills training in robotic surgery. Implementation of a PBP training methodology resulted in superior surgical performance for robotic suturing and knot-tying anastomosis performance. Compared to TT, better surgical quality could be obtained by implementing PBP training for basic skills in robotic surgery.
Collapse
Affiliation(s)
- Ruben De Groote
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
| | - Stefano Puliatti
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Marco Amato
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Elio Mazzone
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy
- Vita-Salute San Raffaele University, Milan, Italy
| | - Giuseppe Rosiello
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy
- Vita-Salute San Raffaele University, Milan, Italy
| | - Rui Farinha
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
| | - Artur Paludo
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Urology, Clinic Hospital of Porto Alegre, Porto Alegre, Brazil
| | - Liesbeth Desender
- Department of Thoracovascular Surgery, University Hospital Ghent, Ghent, Belgium
| | | | | | - Alexandre Mottrie
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
| | - Anthony G Gallagher
- ORSI Academy, Melle, Belgium
- Faculty of Medicine, KU Leuven, Leuven, Belgium
- School of Medicine, Faculty of Life and Health Sciences, Ulster University, Coleraine, UK
| |
Collapse
|
19
|
Farinha R, Breda A, Porter J, Mottrie A, Van Cleynenbreugel B, Vander Sloten J, Mottaran A, Gallagher AG. International Expert Consensus on Metric-based Characterization of Robot-assisted Partial Nephrectomy. Eur Urol Focus 2022; 9:388-395. [PMID: 36229343 DOI: 10.1016/j.euf.2022.09.017] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2022] [Revised: 09/11/2022] [Accepted: 09/28/2022] [Indexed: 11/04/2022]
Abstract
BACKGROUND Robot-assisted partial nephrectomy (RAPN) training usually takes place in vivo, and methods vary across countries/institutions. No common system exists to objectively assess trainee ability to perform RAPN at predetermined performance levels prior to in vivo practice. The identification of objective performance metrics for RAPN training is a crucial starting point to improve training and surgical outcomes. OBJECTIVE We sought to identify objective performance metrics that best characterize a reference approach to RAPN, and obtain face and content validity from procedure experts through a modified Delphi meeting. DESIGN, SETTING, AND PARTICIPANTS During a series of online meetings, a core metrics team of three RAPN experts and a senior behavioral scientist performed a detailed task deconstruction of a transperitoneal left-sided RAPN procedure. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS Based on published guidelines, manufacturers' instructions, and unedited videos of RAPN, the team identified performance metrics that constitute an optimal approach for training purposes. The metrics were then subjected to an in-person modified international Delphi panel meeting with 19 expert surgeons. RESULTS AND LIMITATIONS Eleven procedure phases, with 64 procedure steps, 43 errors, and 39 critical errors, were identified. After the modified Delphi process, the international expert panel added 13 metrics (two steps), six were deleted, and three were modified; 100% panel consensus on the resulting metrics was obtained. Limitations are that the metrics are applicable only to left-sided RAPN cases and some might have been excluded. CONCLUSIONS Performance metrics that accurately characterize RAPN procedure were developed by a core group of experts. The metrics were then presented to and endorsed by an international panel of very experienced peers. Reliable and valid metrics underpin effective, quality-assured, structured surgical training for RAPN. PATIENT SUMMARY We organize a meeting among robot-assisted partial nephrectomy (RAPN) experts to identify and reach consensus on objective performance metrics for RAPN training. The metrics are a crucial starting point to improve and quality assure surgical training and patients' clinical outcomes.
Collapse
|
20
|
Holtmannspötter M, Crossley RA, Liebig T, Gallagher AG. Metric-Based Simulation Training to Proficiency for Endovascular Thrombectomy in Ischemic Stroke. Front Neurol 2022; 13:742263. [PMID: 36237633 PMCID: PMC9551015 DOI: 10.3389/fneur.2022.742263] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2021] [Accepted: 06/22/2022] [Indexed: 11/17/2022] Open
Abstract
Ischemic stroke is one of the leading causes of death and long-term disability in the West. Mechanical revascularization techniques are considered the standard of care for large vessel occlusive stroke. Traditional apprenticeship models involve doctors training their skills on patients. Simulation platforms have long been recognized as an alternative to this. There has however been very little robust assessment of the training outcomes achieved on some of these platforms. At best, these simulations increase understanding of the procedural process and may help improve some technical skills; at worst they may instill bad habits and poor technique. The prerequisite of any simulation process must be to teach what to do, with which devices, in the correct sequence as well as what not to do. It should provide valid metric-based feedback to the trainee that is objective, transparent, and fair for formative and summative performance feedback. It should deliver a training program that measures the performance progress of trainees against expert benchmarks—benchmarks that represent an evidence-based peer-reviewed standard. In this paper, we present a perspective for PBP training for thrombectomy based on our experience with the process of procedure characterization, metric validation, and early experience of using this approach for proficiency training. Patient outcomes are not only determined by optimal performance in the Angio Suite but also by an efficient patient procedure pathway. There will be value in utilizing the PBP training standard not only for the procedure itself but also for the constituent elements of the stroke pathway to further improve treatment outcomes for ischemic stroke patients.
Collapse
Affiliation(s)
- Markus Holtmannspötter
- Department of Neuroradiology, Nuremberg General Hospital, Paracelsus Medical University, Nuremberg, Germany
- *Correspondence: Markus Holtmannspötter
| | - Robert A. Crossley
- North Bristol National Health Service (NHS) Trust, Bristol, United Kingdom
| | - Thomas Liebig
- Department of Neuroradiology, Ludwig-Maximilians-University (LMU) Munich University Hospital, Munich, Germany
| | - Anthony G. Gallagher
- Faculty of Medicine, Katholieke Universiteit (KU) Leuven, Leuven, Belgium
- Orsi Academy, Melle, Belgium
- School of Medicine, Faculty of Life and Health Sciences, University of Ulster, Coleraine, United Kingdom
| |
Collapse
|
21
|
Puliatti S, Amato M, Mazzone E, Rosiello G, De Groote R, Berquin C, Piazza P, Farinha R, Mottrie A, Gallagher AG. Development and Validation of the Metric-Based Assessment of a Robotic Dissection Task on an Avian Model. J Surg Res 2022; 277:224-234. [DOI: 10.1016/j.jss.2022.02.056] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2021] [Revised: 02/13/2022] [Accepted: 02/23/2022] [Indexed: 01/16/2023]
|
22
|
A three-dimensional (3D) printed simulator as a feasible assessment tool for evaluating hip arthroscopy skills. Knee Surg Sports Traumatol Arthrosc 2022; 31:2030-2037. [PMID: 36038668 PMCID: PMC10090017 DOI: 10.1007/s00167-022-07125-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Accepted: 08/17/2022] [Indexed: 10/15/2022]
Abstract
PURPOSE The aims of this study were (1) to develop a three-dimensional (3D) printed simulator that facilitates the simulation of surgical skills for portal placement, intra-articular identification of anatomical structures and arthroscope navigation for hip arthroscopy and (2) to concurrently examine the feasibility of using this simulator as an assessment tool to evaluate trainees' surgical competencies. METHODS A simulator was developed using a combination of medical imaging, computer-aided design, and 3D printing. A cross-sectional study was conducted with 29 participants divided into 3 subgroups (novice, intermediate and experienced). All participants performed related skills on the simulator, and their performance was evaluated using different assessment parameters. The participants' qualitative feedback regarding the simulator was also collected. The data collated from each group of participants were subsequently compared. RESULTS Significant differences were observed between the three subgroups of participants with regard to the total checklist score (F2,26 = 11.3), total Arthroscopic Surgical Skill Evaluation score (F2,26 = 92.1), overall final global rating scale score (F2,26 = 49), number of times the participants used fluoroscopy (F2,26 = 7.4), and task completion times (F2,26 = 23.5). The participants' performance in the simulated operation was correlated with their prior clinical experience. There was mainly positive feedback with regard to the fidelity and utility of the simulator in relation to the surgeons' prior clinical experience. CONCLUSIONS This study demonstrated that a reliable hip arthroscopic simulator can be developed for use by orthopedic surgeons to evaluate their hip arthroscopic skills before performing actual surgical operations. LEVEL OF EVIDENCE Level III.
Collapse
|
23
|
Kojima KE, Graves M, Taha W, Ghidinelli M, Struelens B, Aliaga JAA, Cunningham M, Joeris A, Gallagher AG. Discrimination, reliability, sensitivity, and specificity of metric-based assessment of an unstable pertrochanteric 31A2 intramedullary nailing procedure performed by experienced and novice surgeons. Injury 2022; 53:2832-2838. [PMID: 35705426 DOI: 10.1016/j.injury.2022.05.056] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/12/2022] [Revised: 05/30/2022] [Accepted: 05/31/2022] [Indexed: 02/02/2023]
Abstract
INTRODUCTION Identifying objective performance metrics for surgical training in orthopedic surgery is imperative for effective training and patient safety. The objective of this study was to determine if an internationally agreed, metric-based objective assessment of video recordings of an unstable pertrochanteric 31A2 intramedullary nailing procedure distinguished between the performance of experienced and novice orthopedic surgeons. MATERIALS AND METHODS Previously agreed procedure metrics (i.e., 15 phases of the procedure, 75 steps, 88 errors, and 28 sentinel errors) for a closed reduction and standard cephalomedullary nail fixation with a single cephalic element of an unstable pertrochanteric 31A2 fracture. Experienced surgeons trained to assess the performance metrics with an interrater reliability (IRR) > 0.8 assessed 14 videos from 10 novice surgeons (orthopaedic residents/trainees) and 20 videos from 14 experienced surgeons (orthopaedic surgeons) blinded to group and procedure order. RESULTS The mean IRR of procedure assessments was 0.97. No statistically significant differences were observed between the two groups for Procedure Steps, Errors, Sentinel Errors, and Total Errors. A small number of Experienced surgeons made a similar number of Total Errors as the weakest performing Novices. When the scores of each group were divided at the median Total Error score, large differences were observed between the Experienced surgeons who made the fewest errors and the Novices making the most errors (p < 0.001). Experienced surgeons who made the most errors made significantly more than their Experienced peers (p < 0.003) and the best performing Novices (p < 0.001). Error metrics assessed with Area Under the Curve demonstrated good to excellent Sensitivity and Specificity (0.807-0.907). DISCUSSION Binary performance metrics previously agreed by an international Delphi meeting discriminated between the objectively assessed video-recorded performance of Experienced and Novice orthopedic surgeons when group scores were sub-divided at the median for Total Errors. Error metrics discriminated best and also demonstrated good to excellent Sensitivity and Specificity. Some very experienced surgeons performed similar to the Novice group surgeons that made most errors. CONCLUSIONS The procedure metrics used in this study reliably distinguish Novice and Experienced orthopaedic surgeons' performance and will underpin quality-assured novice training.
Collapse
Affiliation(s)
- Kodi E Kojima
- Instituto de Ortopedia, Hospital das Clinicas HCFMUSP, Faculdade de Medicina, Universidade de Sao Paulo, Sao Paulo, Brazil
| | - Matt Graves
- Hansjörg Wyss AO Medical Foundation Chair of Orthopaedic Trauma, University of Mississippi Medical Center, Jackson, MS, USA
| | - Wa'el Taha
- Prince Mohammed bin Abdulaziz Hospital, Madinah, Saudi Arabia
| | - Monica Ghidinelli
- AO Foundation, AO Education Institute, Stettbachstrasse 6, Dübendorf 8600, Switzerland
| | - Bernard Struelens
- Orthopaedics and Traumatology Department, AZ West, Ieperse Steenweg 100, Veurne 8630, Belgium
| | | | - Mike Cunningham
- AO Foundation, AO Education Institute, Stettbachstrasse 6, Dübendorf 8600, Switzerland
| | | | - Anthony G Gallagher
- ORSI Academy, Melle, Belgium; School of Medicine, Faculty of Life and Health Sciences, Ulster University, Northern Ireland, UK; Faculty of Medicine, KU Leuven, Leuven, Belgium.
| |
Collapse
|
24
|
Cai B, Duan S, Yi J, Huang W, Bay BH, Li C, Chen C. Training surgical skills on hip arthroscopy by simulation: a survey on surgeon's perspectives. Int J Comput Assist Radiol Surg 2022; 17:1813-1821. [PMID: 35831550 PMCID: PMC9468038 DOI: 10.1007/s11548-022-02708-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2021] [Accepted: 06/21/2022] [Indexed: 11/15/2022]
Abstract
Purpose The purpose of this study is to investigate the importance of general and specific surgical skills for hip arthroscopy from the perspective of surgeons in China. Concurrently, we intend to identify the preferred type of simulation that would facilitate competency of surgical trainees in performing arthroscopy and reinforce their preparation for carrying out the actual surgical procedure. Methods An online survey comprising 42 questions was developed by experts in hip arthroscopy and sent to 3 online communities whose members are arthroscopic surgeons in China. The responses collected were based on a 5-point Likert scale, with an open-ended comment section. Data were analyzed using one-way AVOVA and post hoc Tukey’s test. Results A total of 159 valid responses from 66 junior specialist surgeons, 68 consultant surgeons, and 25 senior consultant surgeons (from 130 institutions in 27 out of 34 provincial administrative districts in China) were collected. Cognitive ability was identified as the overall most important attribute for hip arthroscopic trainees to possess, while skills relevant to the treatment of femoroacetabular impingement (FAI) were considered as the most important specific skills by the surgeons surveyed. In addition, simulation using cadaveric specimens was considered the most favorable method for surgeons to practice their surgical skills. Conclusion In designing a training program for hip arthroscopy, it is essential to incorporate features that evaluate cognitive skills. It would be helpful for trainees to specifically practice skills that are often used in the treatment of some very common diseases of the hip joint, such as FAI. Using high-fidelity physical models for simulation to train skills of hip arthroscopy could be an ideal alternative and effective way to overcome problems arising from the lack of accessibility to cadaveric specimens. Supplementary Information The online version contains supplementary material available at 10.1007/s11548-022-02708-x.
Collapse
Affiliation(s)
- Bohong Cai
- Department of Industrial and Product Design, School of Design, Sichuan Fine Arts Institute, Chongqing, China
| | - Shengfeng Duan
- Department of Industrial and Product Design, School of Design, Sichuan Fine Arts Institute, Chongqing, China
| | - Jiahui Yi
- Department of Industrial and Product Design, School of Design, Sichuan Fine Arts Institute, Chongqing, China
| | - Wei Huang
- Department of Orthopedics, The First Affiliated Hospital of Chongqing Medical University, Chongqing, 400016, China
| | - Boon Huat Bay
- Department of Anatomy, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Chunbao Li
- Department of Orthopedics, The Fourth Medical Center of Chinese PLA General Hospital, Beijing, 100048, China.
| | - Cheng Chen
- Department of Orthopedics, The First Affiliated Hospital of Chongqing Medical University, Chongqing, 400016, China.
| |
Collapse
|
25
|
Quality improvement: understanding the adoption and diffusion of digital technologies related to surgical performance. INTERNATIONAL JOURNAL OF QUALITY & RELIABILITY MANAGEMENT 2022. [DOI: 10.1108/ijqrm-07-2021-0234] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
PurposeQuantifying the performance level of surgeons with digital virtual reality (VR) simulators can help ensure that quality requirements in healthcare are met. In order to better understand integration amongst quality principles, practices and technologies in the adoption and diffusion of VR simulators, the authors applied a technological innovation system (TIS) framework. The purpose of this study is to understand how the adoption and diffusion of VR surgical simulators in a Swedish healthcare context is influenced by various system factors.Design/methodology/approachIn this study, single-case holistic design based on innovation system theory was used to analyse the adoption of digital quality technologies related to surgical performance in Swedish hospitals. The case employs a mixed methods approach triangulating data longitudinally from published documents and expert interviews.FindingsAdoption of digital technologies regarding surgical performance is restricted by system factors relating to inconsistent normative and regulatory requirements for quantified performance criteria to judge surgical expertise. Addressing these systems' weaknesses with evidence-based training programmes can have a significant impact on the further development of the innovation system and can ultimately affect healthcare reliability and quality.Originality/valueThis paper explores quality management (QM) challenges in the context of digital transformation in healthcare. The paper attempts to fill the gap for TIS studies in a healthcare context and highlight the role of innovation function strength along the value chain and in relation to technology cycles to increase the understanding of adoption of digital technologies relating to surgical performance.
Collapse
|
26
|
Gallagher AG, De Groote R, Paciotti M, Mottrie A. Proficiency-based Progression Training: A Scientific Approach to Learning Surgical Skills. Eur Urol 2022; 81:394-395. [PMID: 35074249 DOI: 10.1016/j.eururo.2022.01.004] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2021] [Accepted: 01/04/2022] [Indexed: 01/22/2023]
Affiliation(s)
- Anthony G Gallagher
- ORSI Academy, Melle, Belgium; Faculty of Medicine, KU Leuven, Leuven, Belgium; School of Medicine, Faculty of Life and Health Sciences, Ulster University, Belfast, UK.
| | - Ruben De Groote
- ORSI Academy, Melle, Belgium; Faculty of Medicine, KU Leuven, Leuven, Belgium; Department of Urology, OLV, Aalst, Belgium
| | - Marco Paciotti
- ORSI Academy, Melle, Belgium; Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy; Department of Urology, IRCCS Humanitas Research Hospital, Rozzano, Italy
| | - Alexandre Mottrie
- ORSI Academy, Melle, Belgium; Department of Urology, OLV, Aalst, Belgium
| |
Collapse
|
27
|
Puliatti S, Amato M, Farinha R, Paludo A, Rosiello G, De Groote R, Mari A, Bianchi L, Piazza P, Van Cleynenbreugel B, Mazzone E, Migliorini F, Forte S, Rocco B, Kiely P, Mottrie A, Gallagher AG. Does quality assured eLearning provide adequate preparation for robotic surgical skills; a prospective, randomized and multi-center study. Int J Comput Assist Radiol Surg 2022; 17:457-465. [PMID: 34997525 PMCID: PMC8740863 DOI: 10.1007/s11548-021-02545-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Accepted: 12/10/2021] [Indexed: 11/26/2022]
Abstract
Purpose In particular after the onset of the COVID-19 pandemic, there was a precipitous rush to implement virtual and online learning strategies in surgery and medicine. It is essential to understand whether this approach is sufficient and adequate to allow the development of robotic basic surgical skills. The main aim of the authors was to verify if the quality assured eLearning is sufficient to prepare individuals to perform a basic surgical robotic task.
Methods A prospective, randomized and multi-center study was conducted in September 2020 in the ORSI Academy, International surgical robotic training center. Forty-seven participants, with no experience but a special interest in robotic surgery, were matched and randomized into four groups who underwent a didactic preparation with different formats before carrying out a robotic suturing and anastomosis task. Didactic preparation methods ranged from a complete eLearning path to peer-reviewed published manuscripts describing the suturing, knot tying and task assessment metrics. Results The primary outcome was the percentage of trainees who demonstrated the quantitatively defined proficiency benchmark after learning to complete an assisted but unaided robotic vesico-urethral anastomosis task. The quantitatively defined benchmark was based on the objectively assessed performance (i.e., procedure steps completed, errors and critical errors) of experienced robotic surgeons for a proficiency-based progression (PBP) training course. None of the trainees in this study demonstrated the proficiency benchmarks in completing the robotic surgery task. Conclusions PBP-based e-learning methodology is an effective training method avoiding critical errors in the suturing and knotting task. Quality assured online learning is insufficient preparation for robotic suturing and knot tying anastomosis skills. Trial registration ClinicalTrials.gov Identifier: NCT04541615.
Collapse
Affiliation(s)
- Stefano Puliatti
- ORSI Academy, Melle, Belgium.
- Department of Urology, OLV, Aalst, Belgium.
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy.
- Department of Urology, Hospital de Clínicas de Porto Alegre, Porto Alegre, Brazil.
| | - Marco Amato
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Rui Farinha
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
| | - Artur Paludo
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Department of Experimental, Diagnostic and Specialty Medicine (DIMES), Cardio-Nephro-Thoracic Sciences Doctorate, University of Bologna, Bologna, Italy
- Department of Urology, Hospital de Clínicas de Porto Alegre, Porto Alegre, Brazil
| | - Giuseppe Rosiello
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Vita-Salute San Raffaele University, Milan, Italy
| | - Ruben De Groote
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
| | - Andrea Mari
- Unit of Oncologic Minimally Invasive Urology and Andrology, Department of Urology, University of Florence, Careggi Hospital, Florence, Italy
| | - Lorenzo Bianchi
- Office for the Vice President for Learning and Teaching, University College Cork, Cork, Ireland
- Department of Urology, Azienda Ospedaliero-Universitaria di Bologna, Via Albertoni 15, Bologna, Italy
| | - Pietro Piazza
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Office for the Vice President for Learning and Teaching, University College Cork, Cork, Ireland
| | | | - Elio Mazzone
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Vita-Salute San Raffaele University, Milan, Italy
| | - Filippo Migliorini
- Urology Unit, Azienda Ospedaliera Universitaria Integrata di Verona, Verona, Italy
| | - Saverio Forte
- Urology and Andrology Unit II, Department of Emergency and Organ Transplantation, University of Bari, Bari, Italy
| | - Bernardo Rocco
- Urology Department, ASST Santi Paolo e Carlo - Milano, Università Statale di Milano, Milan, Italy
| | - Patrick Kiely
- ORSI Academy, Melle, Belgium
- Office for the Vice President for Learning and Teaching, University College Cork, Cork, Ireland
| | - Alexandre Mottrie
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
| | - Anthony G Gallagher
- ORSI Academy, Melle, Belgium
- Faculty of Life and Health Sciences, Ulster University, Derry, Northern Ireland, UK
- Faculty of Medicine, KU Leuven, Leuven, Belgium
| |
Collapse
|
28
|
Toale C, Morris M, Kavanagh DO. Training to proficiency in surgery using simulation: is there a moral obligation? JOURNAL OF MEDICAL ETHICS 2022; 49:medethics-2021-107678. [PMID: 34992083 DOI: 10.1136/medethics-2021-107678] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2021] [Accepted: 12/21/2021] [Indexed: 06/14/2023]
Abstract
A deontological approach to surgical ethics advocates that patients have the right to receive the best care that can be provided. The 'learning curve' in surgical skill is an observable and measurable phenomenon. Surgical training may therefore carry risk to patients. This can occur directly, through inadvertent harm, or indirectly through theatre inefficiency and associated costs. Trainee surgeon operating, however, is necessary from a utilitarian perspective, with potential risk balanced by the greater societal need to train future independent surgeons.New technology means that the surgical learning curve could take place, at least in part, outside of the operating theatre. Simulation-based deliberate practice could be used to obtain a predetermined level of proficiency in a safe environment, followed by simulation-based assessment of operative competence. Such an approach would require an overhaul of the current training paradigm and significant investment in simulator technology. This may increasingly be viewed as necessary in light of well-discussed pressures on surgical trainees and trainers.This article discusses the obligations to trainees, trainers and training bodies raised by simulation technology, and outlines the current arguments both against and in favour of a simulation-based training-to-proficiency model in surgery. The significant changes to the current training paradigm that would be required to implement such a model are also discussed.
Collapse
Affiliation(s)
- Conor Toale
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Marie Morris
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Dara O Kavanagh
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin, Ireland
| |
Collapse
|
29
|
Myers TG, Marsh JL, Nicandri G, Gorczyca J, Pellegrini VD. Contemporary Issues in the Acquisition of Orthopaedic Surgical Skills During Residency: Competency-Based Medical Education and Simulation. J Bone Joint Surg Am 2022; 104:79-91. [PMID: 34752441 DOI: 10.2106/jbjs.20.01553] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
➤ Orthopaedic education should produce surgeons who are competent to function independently and can obtain and maintain board certification. ➤ Contemporary orthopaedic training programs exist within a fixed 5-year time frame, which may not be a perfect match for each trainee. ➤ Most modern orthopaedic residencies have not yet fully adopted objective, proficiency-based, surgical skill training methods despite nearly 2 decades of evidence supporting the use of this methodology. ➤ Competency-based medical education backed by surgical simulation rooted in proficiency-based progression has the potential to address surgical skill acquisition challenges in orthopaedic surgery.
Collapse
Affiliation(s)
- Thomas G Myers
- Department of Orthopaedics, University of Rochester Medical Center, Rochester, New York
| | | | - Gregg Nicandri
- Department of Orthopaedics, University of Rochester Medical Center, Rochester, New York
| | - John Gorczyca
- Department of Orthopaedics, University of Rochester Medical Center, Rochester, New York
| | - Vincent D Pellegrini
- Department of Orthopaedics, Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire
| |
Collapse
|
30
|
Gómez Ruiz M, Tou S, Gallagher AG, Cagigas Fernández C, Cristobal Poch L, Matzel KE. OUP accepted manuscript. BJS Open 2022; 6:6583541. [PMID: 35543264 PMCID: PMC9092445 DOI: 10.1093/bjsopen/zrac041] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Revised: 02/23/2022] [Accepted: 03/04/2022] [Indexed: 11/13/2022] Open
Abstract
Background This study aimed to evaluate the use of binary metric-based (proficiency-based progression; PBP) performance assessments and global evaluative assessment of robotic skills (GEARS) of a robotic-assisted low anterior rectal resection (RA-LAR) procedure. Method A prospective study of video analysis of RA-LAR procedures was carried out using the PBP metrics with binary parameters previously developed, and GEARS. Recordings were collected from five novice surgeons (≤30 RA-LAR previously performed) and seven experienced surgeons (>30 RA-LAR previously performed). Two consultant colorectal surgeons were trained to be assessors in the use of PBP binary parameters to evaluate the procedure phases, surgical steps, errors, and critical errors in male and female patients and GEARS scores. Novice and experienced surgeons were categorized and assessed using PBP metrics and GEARS; mean scores obtained were compared for statistical purpose. Also, the inter-rater reliability (IRR) of these assessment tools was evaluated. Results Twenty unedited recordings of RA-LAR procedures were blindly assessed. Overall, using PBP metric-based assessment, a subgroup of experienced surgeons made more errors (20 versus 16, P = 0.158) and critical errors (9.2 versus 7.8, P = 0.417) than the novice group, although not significantly. However, during the critical phase of RA-LAR, experienced surgeons made significantly fewer errors than the novice group (95% CI of the difference, Lower = 0.104 – Upper = 5.155, df = 11.9, t = 2.23, p = 0.042), and a similar pattern was observed for critical errors. The PBP metric and GEARS assessment tools distinguished between the objectively assessed performance of experienced and novice colorectal surgeons performing RA-LAR (total error scores with PBP metrics, P = 0.019–0.008; GEARS scores, P = 0.029–0.025). GEARS demonstrated poor IRR (mean IRR 0.49) and weaker discrimination between groups (15–41 per cent difference). PBP binary metrics demonstrated good IRR (mean 0.94) and robust discrimination particularly for total error scores (58–64 per cent). Conclusions PBP binary metrics seem to be useful for metric-based training for surgeons learning RA-LAR procedures.
Collapse
Affiliation(s)
- Marcos Gómez Ruiz
- Colorectal Surgery Unit, General Surgery Department, Marqués de Valdecilla University Hospital, Santander, Spain
- Valdecilla virtual Hospital, Valdecilla Biomedical Research Institute (IDIVAL), Santander, Spain
| | - Samson Tou
- Department of Colorectal Surgery, University Hospitals of Derby and Burton NHS Foundation Trust, Derby, UK
- School of Medicine, Royal Derby Hospital, University of Nottingham, Derby, UK
- Correspondence to: Samson Tou, University Hospitals of Derby and Burton NHS Foundation Trust, Uttoxeter Road, Derby DE22 3NE, UK (e-mail: )
| | | | - Carmen Cagigas Fernández
- Colorectal Surgery Unit, General Surgery Department, Marqués de Valdecilla University Hospital, Santander, Spain
- Valdecilla virtual Hospital, Valdecilla Biomedical Research Institute (IDIVAL), Santander, Spain
| | - Lidia Cristobal Poch
- Colorectal Surgery Unit, General Surgery Department, Marqués de Valdecilla University Hospital, Santander, Spain
- Valdecilla virtual Hospital, Valdecilla Biomedical Research Institute (IDIVAL), Santander, Spain
| | - Klaus E. Matzel
- Section of Coloproctology, Department of Surgery, University of Erlangen-Nürnberg, FAU, Erlangen, Germany
| |
Collapse
|
31
|
O' Herlihy N, Griffin S, Gaffney R, Henn P, Khashan AS, Ring M, Gallagher A, Cahill MR. Proficiency-based progression intern training to reduce critical blood sampling errors including ‘wrong blood in tube’. HRB Open Res 2021. [DOI: 10.12688/hrbopenres.13329.2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
Background: Blood sampling errors including ‘wrong blood in tube’ (WBIT) may have adverse effects on clinical outcomes. WBIT errors occur when the blood sample in the tube is not that of the patient identified on the label. This study aims to determine the effect of proficiency-based progression (PBP) training in phlebotomy on the rate of blood sampling errors (including WBIT). Methods: A non-randomised controlled trial compared the blood sampling error rate of 43 historical controls who had not undergone PBP training in 2016 to 44 PBP trained interventional groups in 2017. In 2018, the PBP training programme was implemented and the blood sampling error rate of 46 interns was compared to the 43 historical controls in 2016. Data analysis was performed using logistic regression analysis adjusting for sample timing. Results: In 2016, 43 interns had a total blood sample error rate of 2.4%, compared to 44 interns in 2017, who had error rate of 1.2% (adjusted OR=0.50, 95% CI 0.36-0.70; <0.01). In 2018, 46 interns had an error rate of 1.9% (adjusted OR=0.89, 95% CI 0.65-1.21; p=0.46) when compared to the 2016 historical controls. There were three WBITs in 2016, three WBITs in 2017 and five WBITs in 2018. Conclusions: The study demonstrates that PBP training in phlebotomy has the potential to reduce blood sampling errors. Trial registration number: NCT03577561
Collapse
|
32
|
Kehily E, Mazzone E, Coffey N, Allen F, Gallagher A, Roberts A. Proficiency Based Progression (PBP) training- the future model for dental operative skills training?: A systematic review and meta-analysis of existing literature. J Dent 2021; 116:103906. [PMID: 34838846 DOI: 10.1016/j.jdent.2021.103906] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2021] [Revised: 11/18/2021] [Accepted: 11/22/2021] [Indexed: 11/24/2022] Open
Abstract
OBJECTIVE To evaluate the effectiveness of Proficiency-Based Progression (PBP) operative training using validated performance metrics, by comparing this to standard, conventional training methods. DATA This systematic review was conducted in accordance with the guidelines of the Transparent Reporting of Systematic Reviews and Meta-Analyses (PRISMA). Study quality was assessed using the MERSQI tool and the Cochrane Risk of Bias tool. Results were pooled using biased corrected standardized mean difference and ratio-of-means (ROM). Summary effects were evaluated using a series of fixed and random effects models. The primary outcome was the number of procedural errors performed comparing PBP and non-PBP-based training pathways. In quantitative synthesis testing for procedural errors, a pooled meta-analysis on 87 trainees was conducted using random-effects models. In a ROM analysis, PBP was estimated to reduce the mean rate of errors by 62%, when compared to standard training (ROM 0.38, 95% CI: 0.25; 0.58; p < 0.001) Sources: The electronic databases of PubMed, Embase, Web of Science, MEDLINE and Cochrane library's CENTRAL were searched from inception to 8/11/2021. Filters activated were Randomized Controlled trials, clinical trial. STUDY SELECTION 13 studies were included for review with 11 included in the quantitative synthesis from 174 potentially relevant publications identified by the search strategy. Main inclusion criteria were studies comparing standard surgical/operative training with proficiency-based simulation training using validated metrics based on expert performance. CONCLUSIONS Our meta-analysis found that PBP training improved trainees' performances, by decreasing procedural errors. There is sufficient evidence to explore PBP training for use in dental skills training. PBP training was estimated to reduce the mean rate of operative errors by 62%, when compared to standard training. Given that there is a direct correlation between operative skill and patient outcomes, these data suggest that there is sufficient evidence to explore PBP training for use in dental skills training.
Collapse
Affiliation(s)
- Elaine Kehily
- Cork University Dental School & Hospital, University College Cork, Ireland.
| | - Elio Mazzone
- Division of Oncology, Unit of Urology URI, IRCCS Ospedale San Raffaele, Via Olgettina 60, Milan 20132, MI, Italy
| | - Niamh Coffey
- Cork University Dental School & Hospital, University College Cork, Ireland
| | - Finbarr Allen
- Faculty of Dentistry, National University of Singapore, Singapore; National University Centre for Oral Health, Singapore, Singapore
| | - Anthony Gallagher
- Faculty of Medicine, KU Leuven, Belgium; Faculty of Health and Life Sciences, Ulster University, Northern Ireland; ORSI Academy, Belgium
| | - Anthony Roberts
- Cork University Dental School & Hospital, University College Cork, Ireland
| |
Collapse
|
33
|
Aydın A, Ahmed K, Abe T, Raison N, Van Hemelrijck M, Garmo H, Ahmed HU, Mukhtar F, Al-Jabir A, Brunckhorst O, Shinohara N, Zhu W, Zeng G, Sfakianos JP, Gupta M, Tewari A, Gözen AS, Rassweiler J, Skolarikos A, Kunit T, Knoll T, Moltzahn F, Thalmann GN, Lantz Powers AG, Chew BH, Sarica K, Shamim Khan M, Dasgupta P. Effect of Simulation-based Training on Surgical Proficiency and Patient Outcomes: A Randomised Controlled Clinical and Educational Trial. Eur Urol 2021; 81:385-393. [PMID: 34789393 DOI: 10.1016/j.eururo.2021.10.030] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2020] [Revised: 09/09/2021] [Accepted: 10/18/2021] [Indexed: 12/17/2022]
Abstract
BACKGROUND It is hypothesised that simulation enhances progression along the initial phase of the surgical learning curve. OBJECTIVE To evaluate whether residents undergoing additional simulation, compared to conventional training, are able to achieve proficiency sooner with better patient outcomes. DESIGN, SETTING, AND PARTICIPANTS This international, multicentre, randomised controlled trial recruited 94 urology residents with experience of zero to ten procedures and no prior exposure to simulation in ureterorenoscopy, selected as an index procedure. INTERVENTION Participants were randomised to simulation or conventional operating room training, as is the current standard globally, and followed for 25 procedures or over 18 mo. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS The number of procedures required to achieve proficiency, defined as achieving a score of ≥28 on the Objective Structured Assessment of Technical Skill (OSATS) scale over three consecutive operations, was measured. Surgical complications were evaluated as a key secondary outcome. This trial is registered at www.isrctn.com as ISCRTN 12260261. RESULTS AND LIMITATIONS A total of 1140 cases were performed by 65 participants, with proficiency achieved by 21 simulation and 18 conventional participants over a median of eight and nine procedures, respectively (hazard ratio [HR] 1.41, 95% confidence interval [CI] 0.72-2.75). More participants reached proficiency in the simulation arm in flexible ureterorenoscopy, requiring a lower number of procedures (HR 0.89, 95% CI 0.39-2.02). Significant differences were observed in overall comparison of OSATS scores between the groups (mean difference 1.42, 95% CI 0.91-1.92; p < 0.001), with fewer total complications (15 vs 37; p = 0.003) and ureteric injuries (3 vs 9; p < 0.001) in the simulation group. CONCLUSIONS Although the number of procedures required to reach proficiency was similar, simulation-based training led to higher overall proficiency scores than for conventional training. Fewer procedures were required to achieve proficiency in the complex form of the index procedure, with fewer serious complications overall. PATIENT SUMMARY This study investigated the effect of simulation training in junior surgeons and found that it may improve performance in real operating settings and reduce surgical complications for complex procedures.
Collapse
Affiliation(s)
- Abdullatif Aydın
- MRC Centre for Transplantation, King's College London, King's Health Partners, London, UK.
| | - Kamran Ahmed
- MRC Centre for Transplantation, King's College London, King's Health Partners, London, UK; Department of Urology, King's College Hospital NHS Foundation Trust, King's Health Partners, London, UK
| | - Takashige Abe
- Department of Urology, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | - Nicholas Raison
- MRC Centre for Transplantation, King's College London, King's Health Partners, London, UK
| | | | - Hans Garmo
- School of Cancer and Pharmaceutical Studies, King's College London, London, UK
| | - Hashim U Ahmed
- Department of Surgery and Cancer, Imperial College London, London, UK; Department of Urology, Imperial College Healthcare NHS Trust, London, UK
| | - Furhan Mukhtar
- MRC Centre for Transplantation, King's College London, King's Health Partners, London, UK
| | - Ahmed Al-Jabir
- MRC Centre for Transplantation, King's College London, King's Health Partners, London, UK
| | - Oliver Brunckhorst
- MRC Centre for Transplantation, King's College London, King's Health Partners, London, UK
| | - Nobuo Shinohara
- Department of Urology, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | - Wei Zhu
- Department of Urology, Minimally Invasive Surgery Centre, First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Guohua Zeng
- Department of Urology, Minimally Invasive Surgery Centre, First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - John P Sfakianos
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Mantu Gupta
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Ashutosh Tewari
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Ali Serdar Gözen
- Department of Urology, SLK Kliniken, University of Heidelberg, Heilbronn, Germany
| | - Jens Rassweiler
- Department of Urology, SLK Kliniken, University of Heidelberg, Heilbronn, Germany
| | - Andreas Skolarikos
- 2nd Department of Urology, Sismanoglio Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Thomas Kunit
- Department of Urology and Andrology, Paracelsus Medical University, Salzburg, Austria
| | - Thomas Knoll
- Department of Urology, Klinikum Sindelfingen-Böblingen, University of Tübingen, Sindelfingen, Germany
| | - Felix Moltzahn
- Department of Urology, University of Bern, Bern, Switzerland
| | | | | | - Ben H Chew
- Department of Urologic Sciences, University of British Columbia, Vancouver, BC, Canada
| | - Kemal Sarica
- Department of Urology, Biruni University Hospital, Istanbul, Turkey
| | - Muhammad Shamim Khan
- MRC Centre for Transplantation, King's College London, King's Health Partners, London, UK; Urology Centre, Guy's and St. Thomas' NHS Foundation Trust, King's Health Partners, London, UK
| | - Prokar Dasgupta
- MRC Centre for Transplantation, King's College London, King's Health Partners, London, UK; Urology Centre, Guy's and St. Thomas' NHS Foundation Trust, King's Health Partners, London, UK
| |
Collapse
|
34
|
Angelo RL, St Pierre P, Tauro J, Gallagher AG. Live Observational Objective Assessment of Operative Performance in a Cadaveric Model is Equivalent to Delayed Video-Based Assessment. Arthroscopy 2021; 37:3241-3247. [PMID: 33964394 DOI: 10.1016/j.arthro.2021.04.060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Revised: 04/15/2021] [Accepted: 04/23/2021] [Indexed: 02/02/2023]
Abstract
PURPOSE The purpose of our study was to compare real-time, live observational scoring with delayed retrospective video review of operative performance and to determine whether the evaluation method affected the attainment of proficiency benchmarks. METHODS Sixteen arthroscopy/sports medicine fellows and 2 senior residents completed training to perform arthroscopic Bankart repairs (ABRs) and arthroscopic rotator cuff repairs (ARCRs) using a proficiency-based progression curriculum. Each final operative performance for 15 randomly selected ABRs and 13 ARCRs performed on cadavers were scored live (observation during the operative performance) and on delayed video review (6-8 weeks) by 1 of 15 trained raters using validated metric-based (step and error) assessment tools. The inter-rater reliability (IRR) of live versus video review by a single rater was calculated, and changes to the trainee's attainment of the proficiency benchmarks were noted. The correlation coefficient (r) and the R2 were also calculated for the paired scores from the randomly selected performances. RESULTS No significant differences in the observed IRR agreement or the attainment of the proficiency benchmarks were found when comparing live to video assessment for either ABR or ARCR. The correlation coefficients r and R2 were considerably lower than the agreement coefficient (IRR) for rotator cuff steps (e.g., R2 = 0.74 vs. IRR = 0.97, P = 0.001); Bankart errors (R2 = 0.73 vs. IRR = 0.98, P = 0.006); and rotator cuff errors (R2 = 0.48 vs. IRR = 0.98, P = 0.0002). CONCLUSIONS Real-time live and delayed video-based scoring of operative performance are essentially equivalent for the metric-based assessments of operative performance in ABRs and ARCRs. When the IRR agreement coefficient was compared with the correlation coefficients, the former was found to have greater homogeneity and measurement precision. CLINICAL RELEVANCE Metric-based live scoring is reliable and accurate for operative performance assessment, including high-stakes evaluations.
Collapse
Affiliation(s)
| | - Pat St Pierre
- Desert Orthopedic Center, Palm Desert, California, U.S.A
| | - Joe Tauro
- Department of Orthopedic Surgery, Rutgers Medical School, Newark, New Jersey, U.S.A
| | - Anthony G Gallagher
- Faculty of Life and Health Sciences, Ulster University, Londonderry, U.K., ORSI Academy, Melle, Belgium
| |
Collapse
|
35
|
Mitigating Surgical Skill Decay in Orthopaedics Using Virtual Simulation Learning. JOURNAL OF THE AMERICAN ACADEMY OF ORTHOPAEDIC SURGEONS GLOBAL RESEARCH AND REVIEWS 2021; 5:01979360-202110000-00010. [PMID: 34637404 DOI: 10.5435/jaaosglobal-d-21-00193] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Accepted: 08/10/2021] [Indexed: 01/22/2023]
Abstract
BACKGROUND The COVID-19 pandemic has interrupted orthopaedic training structures for both surgeons and trainees. The concept of skill decay must be considered during inactivity of elective practice. The purpose of this study was to provide an evidence-based curriculum in association with immersive virtual reality (iVR) to prevent skill decay during periods of training cessation and beyond. METHODS A review of pertinent literature for orthopaedic surgical skill decay was performed. Early experience by faculty instructors and residency and fellowship program directors was gathered from multiple institutions with experience in virtual training methods including iVR. A proposed curriculum for cognitive and manual skill acquisition during COVID-19 was produced from qualitative narrative group opinion. RESULTS Skill decay can occur on the order of days to months and is dependent on the initial skill level. A novel curriculum for structured continuing medical education during and after periods of surgical disruption including e-learning, virtual meetings, and iVR simulators was produced from expert opinion and based on competency-based curriculum standards. CONCLUSION Skill decay mitigation strategies should use best available evidence technologies and course structures that satisfy advanced learning concepts. The virtual curriculum including iVR simulators may provide cost-effective solutions to training.
Collapse
|
36
|
Mohamed H, McAuliffe N, O'Connor R, Ceballos Salazar A, Zohaib Aslam M, Kallidaikurichi Srinivasan K, Iohom G, Shorten G. Proficiency-based progression training: implementing a novel approach to training for epidural analgesia in labour. Int J Obstet Anesth 2021; 48:103213. [PMID: 34500191 DOI: 10.1016/j.ijoa.2021.103213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/29/2020] [Revised: 07/04/2021] [Accepted: 08/10/2021] [Indexed: 10/20/2022]
Abstract
BACKGROUND Epidural insertion is a challenging anaesthetic procedural skill to learn and may require up to 75 attempts to achieve competency. Proficiency-based progression (PbP) training based on unambiguously defined metrics was associated with a 53% reduction in epidural failure rate. The aim of this observational study was to examine the feasibility of implementation of innovative PbP training for labour epidural insertion performed by novices in a busy tertiary hospital. METHODS All trainees who were scheduled to commence their obstetric anaesthesia training were invited to participate. Novices undertook intensive PbP training with one-to-one supervision by an anaesthetist trained in PbP. Trainees proceeded to the clinical phase only after attaining the pre-defined proficiency benchmark. All subsequent attempts at labour epidural catheter placement were evaluated. RESULTS All 12 novice trainees who were scheduled for their initial exposure to obstetric anaesthesia completed PbP training in epidural catheter insertion successfully. The average duration of the training courses was 70 (SD 11) min. Trainee characteristics were broadly similar. They performed a total of 180 labour epidural catheter placements with an overall epidural failure rate of 12.2% (22/180). The proportion of supervisor takeover was 6% (11/179). The incidence of complications was 4% (8/180) and difficulty in epidural catheter insertion due to patient factors was 16% (29/180). Patient satisfaction rates were 80% (satisfied or very satisfied), with 20% unsatisfied with their experience of epidural insertion. CONCLUSION In our experience, PbP training in epidural placement is feasible within existing departmental resources in a busy tertiary teaching hospital setting.
Collapse
Affiliation(s)
- H Mohamed
- Department of Anaesthesia and Intensive Care, Faculty of Medicine, Cairo University, Cairo, Egypt; Department of Anaesthesia and Intensive Care, Leeds Teaching Hospitals, Leeds, UK.
| | - N McAuliffe
- Department of Anaesthesia and Intensive Care, Cork University Hospital, Cork, Ireland
| | - R O'Connor
- Department of Anaesthesia, University College Cork, Cork, Ireland
| | - A Ceballos Salazar
- Department of Anaesthesia and Intensive Care, Royal Papworth Hospital, Cambridge, UK
| | - M Zohaib Aslam
- Department of Anaesthesia and Intensive Care, Cork University Hospital, Cork, Ireland
| | | | - G Iohom
- Department of Anaesthesia and Intensive Care, Cork University Hospital, Cork, Ireland; Department of Anaesthesia, University College Cork, Cork, Ireland
| | - G Shorten
- Department of Anaesthesia and Intensive Care, Cork University Hospital, Cork, Ireland; Department of Anaesthesia, University College Cork, Cork, Ireland; Insight II SFI Research Centre, Ireland
| |
Collapse
|
37
|
Mazzone E, Puliatti S, Amato M, Bunting B, Rocco B, Montorsi F, Mottrie A, Gallagher AG. A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes. Ann Surg 2021; 274:281-289. [PMID: 33630473 DOI: 10.1097/sla.0000000000004650] [Citation(s) in RCA: 58] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
Abstract
OBJECTIVE To analyze all published prospective, randomized, and blinded clinical studies on the proficiency-based progression (PBP) training using objective performance metrics. BACKGROUND The benefit of PBP methodology to learning clinical skills in comparison to conventional training is not settled. METHODS Search of PubMed, Cochrane library's Central, EMBASE, MEDLINE, and Scopus databases, from inception to 1st March 2020. Two independent reviewers extracted the data. The Medical Education Research Study Quality Instrument (MERSQI) was used to assess the methodological quality of included studies. Results were pooled using biased corrected standardized mean difference and ratio-of-means. Summary effects were evaluated using a series of fixed and random effects models. The primary outcome was the number of procedural errors performed comparing PBP and non-PBP-based training pathways. Secondary outcomes were the number of procedural steps completed and the time to complete the task/procedure. RESULTS From the initial pool of 468 studies, 12 randomized clinical studies with a total of 239 participants were included in the analysis. In comparison to the non-PBP training, ratio-of-means results showed that PBP training reduced the number of performance errors by 60% (P < 0.001) and procedural time by 15% (P = 0.003) and increased the number of steps performed by 47% (P < 0.001). CONCLUSIONS AND RELEVANCE Our systematic review and meta-analysis confirms that PBP training in comparison to conventional or quality assured training improved trainees' performances, by decreasing procedural errors and procedural time, while increasing the number of correct steps taken when compared to standard simulation-based training.
Collapse
Affiliation(s)
- Elio Mazzone
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy
- ORSI Academy, Melle, Belgium
| | - Stefano Puliatti
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Marco Amato
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Brendan Bunting
- School of Medicine, Faculty of Life and Health Sciences, Ulster University, Northern Ireland, UK
| | - Bernardo Rocco
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Francesco Montorsi
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy
| | - Alexandre Mottrie
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
| | - Anthony G Gallagher
- ORSI Academy, Melle, Belgium
- School of Medicine, Faculty of Life and Health Sciences, Ulster University, Northern Ireland, UK
- Faculty of Medicine, KU Leuven, Leuven, Belgium
| |
Collapse
|
38
|
Chen IHA, Ghazi A, Sridhar A, Stoyanov D, Slack M, Kelly JD, Collins JW. Evolving robotic surgery training and improving patient safety, with the integration of novel technologies. World J Urol 2021; 39:2883-2893. [PMID: 33156361 PMCID: PMC8405494 DOI: 10.1007/s00345-020-03467-7] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2020] [Accepted: 09/21/2020] [Indexed: 12/18/2022] Open
Abstract
INTRODUCTION Robot-assisted surgery is becoming increasingly adopted by multiple surgical specialties. There is evidence of inherent risks of utilising new technologies that are unfamiliar early in the learning curve. The development of standardised and validated training programmes is crucial to deliver safe introduction. In this review, we aim to evaluate the current evidence and opportunities to integrate novel technologies into modern digitalised robotic training curricula. METHODS A systematic literature review of the current evidence for novel technologies in surgical training was conducted online and relevant publications and information were identified. Evaluation was made on how these technologies could further enable digitalisation of training. RESULTS Overall, the quality of available studies was found to be low with current available evidence consisting largely of expert opinion, consensus statements and small qualitative studies. The review identified that there are several novel technologies already being utilised in robotic surgery training. There is also a trend towards standardised validated robotic training curricula. Currently, the majority of the validated curricula do not incorporate novel technologies and training is delivered with more traditional methods that includes centralisation of training services with wet laboratories that have access to cadavers and dedicated training robots. CONCLUSIONS Improvements to training standards and understanding performance data have good potential to significantly lower complications in patients. Digitalisation automates data collection and brings data together for analysis. Machine learning has potential to develop automated performance feedback for trainees. Digitalised training aims to build on the current gold standards and to further improve the 'continuum of training' by integrating PBP training, 3D-printed models, telementoring, telemetry and machine learning.
Collapse
Affiliation(s)
- I-Hsuan Alan Chen
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, University College London, London, UK.
- Department of Surgery, Division of Urology, Kaohsiung Veterans General Hospital, No. 386, Dazhong 1st Rd., Zuoying District, Kaohsiung, 81362, Taiwan.
- Wellcome/ESPRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK.
| | - Ahmed Ghazi
- Department of Urology, Simulation Innovation Laboratory, University of Rochester, New York, USA
| | - Ashwin Sridhar
- Division of Uro-Oncology, University College London Hospital, London, UK
| | - Danail Stoyanov
- Wellcome/ESPRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK
| | | | - John D Kelly
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, University College London, London, UK
- Wellcome/ESPRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK
- Division of Uro-Oncology, University College London Hospital, London, UK
| | - Justin W Collins
- Division of Surgery and Interventional Science, Research Department of Targeted Intervention, University College London, London, UK.
- Wellcome/ESPRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK.
- Division of Uro-Oncology, University College London Hospital, London, UK.
| |
Collapse
|
39
|
O' Herlihy N, Griffin S, Gaffney R, Henn P, Khashan AS, Ring M, Gallagher A, Cahill MR. Proficiency-based progression intern training to reduce critical blood sampling errors including ‘wrong blood in tube’. HRB Open Res 2021. [DOI: 10.12688/hrbopenres.13329.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
Background: Blood sampling errors including ‘wrong blood in tube’ (WBIT) may have adverse effects on clinical outcomes. WBIT errors occur when the blood sample in the tube is not that of the patient identified on the label. This study aims to determine the effect of proficiency-based progression (PBP) training in phlebotomy on the rate of blood sampling errors (including WBIT). Methods: A non-randomised controlled trial compared the blood sampling error rate of 43 historical controls who had not undergone PBP training in 2016 to 44 PBP trained interventional groups in 2017. In 2018, the PBP training programme was implemented and the blood sampling error rate of 46 interns was compared to the 43 historical controls in 2016. Data analysis was performed using logistic regression analysis adjusting for sample timing. Results: In 2016, 43 interns had a total blood sample error rate of 2.4%, compared to 44 interns in 2017, who had error rate of 1.2% (adjusted OR=0.50, 95% CI 0.36-0.70; <0.01). In 2018, 46 interns had an error rate of 1.9% (adjusted OR=0.89, 95% CI 0.65-1.21; p=0.46) when compared to the 2016 historical controls. There were three WBITs in 2016, three WBITs in 2017 and five WBITs in 2018. Conclusions: The study demonstrates that PBP training in phlebotomy has the potential to reduce blood sampling errors. Trial registration number: NCT03577561
Collapse
|
40
|
James HK, Chapman AW, Pattison GTR, Fisher JD, Griffin DR. Analysis of Tools Used in Assessing Technical Skills and Operative Competence in Trauma and Orthopaedic Surgical Training: A Systematic Review. JBJS Rev 2021; 8:e1900167. [PMID: 33006464 PMCID: PMC7360100 DOI: 10.2106/jbjs.rvw.19.00167] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
BACKGROUND Robust assessment of skills acquisition and surgical performance during training is vital to ensuring operative competence among orthopaedic surgeons. A move to competency-based surgical training requires the use of tools that can assess surgical skills objectively and systematically. The aim of this systematic review was to describe the evidence for the utility of assessment tools used in evaluating operative performance in trauma and orthopaedic surgical training. METHODS We performed a comprehensive literature search of MEDLINE, Embase, and Google Scholar databases to June 2019. From eligible studies we abstracted data on study aim, assessment format (live theater or simulated setting), skills assessed, and tools or metrics used to assess surgical performance. The strengths, limitations, and psychometric properties of the assessment tools are reported on the basis of previously defined utility criteria. RESULTS One hundred and five studies published between 1990 and 2019 were included. Forty-two studies involved open orthopaedic surgical procedures, and 63 involved arthroscopy. The majority (85%) were used in the simulated environment. There was wide variation in the type of assessment tools in used, the strengths and weaknesses of which are assessor and setting-dependent. CONCLUSIONS Current technical skills-assessment tools in trauma and orthopaedic surgery are largely procedure-specific and limited to research use in the simulated environment. An objective technical skills-assessment tool that is suitable for use in the live operative theater requires development and validation, to ensure proper competency-based assessment of surgical performance and readiness for unsupervised clinical practice. CLINICAL RELEVANCE Trainers and trainees can gain further insight into the technical skills assessment tools that they use in practice through the utility evidence provided.
Collapse
Affiliation(s)
- Hannah K James
- 1Clinical Trials Unit, Warwick Medical School, Coventry, United Kingdom 2Department of Trauma & Orthopedic Surgery, University Hospitals Coventry & Warwickshire, Coventry, United Kingdom
| | | | | | | | | |
Collapse
|
41
|
Eren MB, Bilgiç E. Novice Surgeon Portal Preference to Visualize the Femoral Anterior Cruciate Ligament Footprint: The Accessory Medial Portal Offers Improved Visualization. Arthrosc Sports Med Rehabil 2021; 3:e815-e822. [PMID: 34195649 PMCID: PMC8220608 DOI: 10.1016/j.asmr.2020.12.013] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Accepted: 12/29/2020] [Indexed: 01/22/2023] Open
Abstract
Purpose To evaluate participant opinions on the appropriateness of different viewing angles by asking 8 questions, using visual content techniques. Methods Survey information from participants was used in this study. We used images of a patient who was operated on for symptomatic knee instability. Participants were asked whether the visual angle could determine the femoral tunnel entry point or evaluate anterior cruciate ligament (ACL) integrity. Results Questionnaires were completed by 40 participants. When all responses were evaluated, participant opinions on the accessory medial portal (AMP) were more positive than opinions on images taken from the anterolateral portal (ALP). These observations were statistically significant (P < .005). Both ACL integrity assessments and femoral tunnel entry site questions were evaluated, we observed that participant opinions on the AMP for both types of evaluation were more positive than opinions on the ALP. This observation was statistically significant (P < .005). When evaluations with photos and video transcripts were compared, no significant differences in terms of participant opinions were observed (P < .005). Conclusions AMP use may be a valuable tool for assistant and postgraduate education, as it offers a more suitable view angle for ACL reconstruction. Level of evidence Level V, survey study.
Collapse
Affiliation(s)
- Mehmet Burtaç Eren
- Department of Orthopaedics and Traumatology, Gaziosmanpasa University School of Medicine, Tokat, Turkey
| | - Erkal Bilgiç
- Department of Orthopaedics and Traumatology, Gaziosmanpasa University School of Medicine, Tokat, Turkey
| |
Collapse
|
42
|
Mottrie A, Mazzone E, Wiklund P, Graefen M, Collins JW, De Groote R, Dell’Oglio P, Puliatti S, Gallagher AG. Objective assessment of intraoperative skills for robot-assisted radical prostatectomy (RARP): results from the ERUS Scientific and Educational Working Groups Metrics Initiative. BJU Int 2021; 128:103-111. [PMID: 33251703 PMCID: PMC8359192 DOI: 10.1111/bju.15311] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
Abstract
OBJECTIVE To develop and seek consensus from procedure experts on the metrics that best characterise a reference robot-assisted radical prostatectomy (RARP) and determine if the metrics distinguished between the objectively assessed RARP performance of experienced and novice urologists, as identifying objective performance metrics for surgical training in robotic surgery is imperative for patient safety. MATERIALS AND METHODS In Study 1, the metrics, i.e. 12 phases of the procedure, 81 steps, 245 errors and 110 critical errors for a reference RARP were developed and then presented to an international Delphi panel of 19 experienced urologists. In Study 2, 12 very experienced surgeons (VES) who had performed >500 RARPs and 12 novice urology surgeons performed a RARP, which was video recorded and assessed by two experienced urologists blinded as to subject and group. Percentage agreement between experienced urologists for the Delphi meeting and Mann-Whitney U- and Kruskal-Wallis tests were used for construct validation of the newly identified RARP metrics. RESULTS At the Delphi panel, consensus was reached on the appropriateness of the metrics for a reference RARP. In Study 2, the results showed that the VES performed ~4% more procedure steps and made 72% fewer procedure errors than the novices (P = 0.027). Phases VIIa and VIIb (i.e. neurovascular bundle dissection) best discriminated between the VES and novices. LIMITATIONS VES whose performance was in the bottom half of their group demonstrated considerable error variability and made five-times as many errors as the other half of the group (P = 0.006). CONCLUSIONS The international Delphi panel reached high-level consensus on the RARP metrics that reliably distinguished between the objectively scored procedure performance of VES and novices. Reliable and valid performance metrics of RARP are imperative for effective and quality assured surgical training.
Collapse
Affiliation(s)
- Alexandre Mottrie
- Orsi AcademyMelleBelgium
- Department of UrologyOnze Lieve Vrouw Hospital (OLV)AalstBelgium
| | - Elio Mazzone
- Orsi AcademyMelleBelgium
- Department of UrologyOnze Lieve Vrouw Hospital (OLV)AalstBelgium
- Division of Oncology/Unit of UrologyURIL’Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Ospedale San RaffaeleMilanItaly
- Vita‐Salute San Raffaele UniversityMilanItaly
| | - Peter Wiklund
- Department of UrologyKarolinska InstitutetStockholmSweden
- Department of UrologyIcahn School of Medicine at Mount Sinai Health SystemNew YorkNYUSA
| | - Markus Graefen
- Martini‐Klinik Prostate Cancer CenterUniversity Hospital Hamburg‐EppendorfHamburgGermany
| | - Justin W. Collins
- Orsi AcademyMelleBelgium
- Department of Uro‐oncologyUniversity College London Hospital (UCLH)LondonUK
| | - Ruben De Groote
- Orsi AcademyMelleBelgium
- Department of UrologyOnze Lieve Vrouw Hospital (OLV)AalstBelgium
| | - Paolo Dell’Oglio
- Orsi AcademyMelleBelgium
- Department of UrologyASST Grande Ospedale Metropolitano NiguardaMilanItaly
| | - Stefano Puliatti
- Orsi AcademyMelleBelgium
- Department of UrologyOnze Lieve Vrouw Hospital (OLV)AalstBelgium
- Department of UrologyUniversity of Modena and Reggio EmiliaModenaItaly
| | - Anthony G. Gallagher
- Orsi AcademyMelleBelgium
- Faculty of Life and Health SciencesUlster UniversityNorthern IrelandUK
| |
Collapse
|
43
|
Harris JD. Editorial Commentary: Virtual Reality Simulation Can Help Arthroscopic Hip Preservation Surgeons at All Levels of Training and Practice-This is How. Arthroscopy 2021; 37:1867-1871. [PMID: 34090570 DOI: 10.1016/j.arthro.2021.03.002] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Accepted: 03/04/2021] [Indexed: 02/02/2023]
Abstract
Virtual reality (VR) simulation has enormous potential utility in technically demanding manual activities. Hip arthroscopy is a perfect example of a challenging surgical technique with an extensive learning curve. The literature has recently consistently demonstrated that both career and annual maintenance case volume significantly influences patient-reported outcomes and risk of revision surgery and complications. Current residency and fellowship programs do not sufficiently prepare trainees to meet or exceed experience thresholds, so augmentation of training is necessary. A significant strength of VR simulation includes its ability to practice without limits. Unfortunately, hip models are limited to simple tasks, without full surgery models yet available simulating routine arthroscopic hip preservation procedures like labral repair, cam and pincer correction, capsular repair. Advanced techniques like labral reconstruction or augmentation, protrusio acetabulae, extensive cam morphology, revision surgery, peritrochanteric space endoscopy, and deep gluteal space endoscopy are not yet available for simulation. VR simulation can probably achieve competence for most, if not all, surgeons; possibly achieve proficiency; and unlikely to achieve mastery. The use of machine learning and artificial intelligence can process vast quantities of photo and video data to generate high-fidelity, lifelike surgical simulation. The near future will incorporate and assimilate these technologies cost-effectively for training programs and surgeons. Our patients will benefit.
Collapse
|
44
|
Beaudoin A, Larrivée S, McRae S, Leiter J, Stranges G. Module-Based Arthroscopic Knee Simulator Training Improves Technical Skills in Naive Learners: A Randomized Trial. Arthrosc Sports Med Rehabil 2021; 3:e757-e764. [PMID: 34195642 PMCID: PMC8220613 DOI: 10.1016/j.asmr.2021.01.016] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2020] [Accepted: 01/24/2021] [Indexed: 01/22/2023] Open
Abstract
Purpose To compare the effectiveness, in comparison to a control group (C), of module-based training (MBT) and traditional learning (TL) as a means of acquiring arthroscopic skills on an arthroscopic surgery simulator. Methods Thirty health sciences students with no previous arthroscopy experience were recruited and randomized into 1 of 3 groups: MBT, TL, or C (1:1:1 ratio). Participants in MBT were required to independently practice on a VirtaMed ArthroS simulator (VirtaMed AG, Zurich, Switzerland) for a minimum of 2 hours per week, whereas TL received one-on-one coaching by a senior orthopaedic resident for 15 minutes per week. The control group received no training. All groups were assessed at baseline and at 4 weeks based on objective measures generated by the surgical simulator (procedure time, camera path length, meniscus cutting score, detailed visualization, safety score and total score), and subjective ratings scales (Objective Assessment of Arthroscopic Skill [OAAS] global assessment form, and Competency-Based Assessment form). Results Participants in the MBT group trained on average 113 min/week whereas the TL group trained on average 24 min/week. Three-way repeated-measures analysis of variance showed significant group by time interactions for procedure time (P = .006), camera path length (P = .008), safety score (P = .013), total score (P = .003), OAAS form (P < .001), and Competency-Based Assessment form (P < .001). MBT group was superior to C group for procedure time (P = .02), camera path length (P = .003), total score (P = .004), and OAAS form (P = .021), but there were no significant post-hoc differences between MBT and TL groups, or TL and C groups after Bonferroni correction. Total practice time explained 37.5% of the final simulator total score variance. Conclusions Knee arthroscopy simulation training with self-learning modules can improve skills in areas such as procedure time, camera path length, and total score in untrained participants compared with a control group. Clinical Relevance Module-based simulation training provides additional training time and improves technical skills in naive health science students. It is hoped that this effect can be preserved and applied to junior resident developing in a busy residency program.
Collapse
Affiliation(s)
| | | | - Sheila McRae
- University of Manitoba, Pan Am Clinic Foundation, Winnipeg
| | - Jeff Leiter
- University of Manitoba, Oak Bluff, Manitoba, Canada
| | | |
Collapse
|
45
|
Hauschild J, Rivera JC, Johnson AE, Burns TC, Roach CJ. Shoulder Arthroscopy Simulator Training Improves Surgical Procedure Performance: A Controlled Laboratory Study. Orthop J Sports Med 2021; 9:23259671211003873. [PMID: 33997080 PMCID: PMC8113660 DOI: 10.1177/23259671211003873] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Accepted: 11/23/2020] [Indexed: 01/22/2023] Open
Abstract
Background: Previous simulation studies evaluated either dry lab (DL) or virtual reality (VR) simulation, correlating simulator training with the performance of arthroscopic tasks. However, these studies did not compare simulation training with specific surgical procedures. Purpose/Hypothesis: To determine the effectiveness of a shoulder arthroscopy simulator program in improving performance during arthroscopic anterior labral repair. It was hypothesized that both DL and VR simulation methods would improve procedure performance; however, VR simulation would be more effective based on the validated Arthroscopic Surgery Skill Evaluation Tool (ASSET) Global Rating Scale. Study Design: Controlled laboratory study. Methods: Enrolled in the study were 38 orthopaedic residents at a single institution, postgraduate years (PGYs) 1 to 5. Each resident completed a pretest shoulder stabilization procedure on a cadaveric model and was then randomized into 1 of 2 groups: VR or DL simulation. Participants then underwent a 4-week arthroscopy simulation program and completed a posttest. Sports medicine–trained orthopaedic surgeons graded the participants on completeness of the surgical repair at the time of the procedure, and a single, blinded orthopaedic surgeon, using the ASSET Global Rating Scale, graded participants’ arthroscopy skills. The procedure step and ASSET grades were compared between simulator groups and between PGYs using paired t tests. Results: There was no significant difference between the groups in pretest performance in either the procedural steps or ASSET scores. Overall procedural step scores improved after combining both types of simulator training (P = .0424) but not in the individual simulation groups. The ASSET scores improved across both DL (P = .0045) and VR (P = .0003), with no significant difference between the groups. Conclusion: A 4-week simulation program can improve arthroscopic skills and performance during a specific surgical procedure. This study provides additional evidence regarding the benefits of simulator training in orthopaedic surgery for both novice and experienced arthroscopic surgeons. There was no statistically significant difference between the VR and DL models, which disproved the authors’ hypothesis that the VR simulator would be the more effective simulation tool. Clinical Relevance: There may be a role for simulator training in the teaching of arthroscopic skills and learning of specific surgical procedures.
Collapse
|
46
|
Mul J, Melchior P, Seravalli E, Saunders D, Bolle S, Cameron AL, Gurtner K, Harrabi S, Lassen-Ramshad Y, Lavan N, Magelssen H, Mandeville H, Boterberg T, Kroon PS, Kotte AN, Hoeben BA, van Rossum PS, van Grotel M, Graf N, van den Heuvel-Eibrink MM, Rübe C, Janssens GO. Inter-clinician delineation variation for a new highly-conformal flank target volume in children with renal tumors: A SIOP-Renal Tumor Study Group international multicenter exercise. Clin Transl Radiat Oncol 2021; 28:39-47. [PMID: 33796796 PMCID: PMC7995478 DOI: 10.1016/j.ctro.2021.03.001] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2020] [Revised: 03/02/2021] [Accepted: 03/03/2021] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND AND PURPOSE Recently, the SIOP-RTSG developed a highly-conformal flank target volume definition for children with renal tumors. The aims of this study were to evaluate the inter-clinician delineation variation of this new target volume definition in an international multicenter setting and to explore the necessity of quality assurance. MATERIALS AND METHODS Six pediatric renal cancer cases were transferred to ten radiation oncologists from seven European countries ('participants'). These participants delineated the pre- and postoperative Gross Tumor Volume (GTVpre/post), and Clinical Target Volume (CTV) during two test phases (case 1-2 and 3-4), followed by guideline refinement and a quality assurance phase (case 5-6). Reference target volumes (TVref) were established by three experienced radiation oncologists. The Dice Similarity Coefficient between the reference and participants (DSCref/part) was calculated per case. Delineations of case 5-6 were graded by four independent reviewers as 'per protocol' (0-4 mm), 'minor deviation' (5-9 mm) or 'major deviation' (≥10 mm) from the delineation guideline using 18 standardized criteria. Also, a major deviation resulting in underestimation of the CTVref was regarded as an unacceptable variation. RESULTS A total of 57/60 delineation sets were completed. The median DSCref/part for the CTV was 0.55 without improvement after sequential cases (case 3-4 vs. case 5-6: p = 0.15). For case 5-6, a major deviation was found for 5/18, 12/17, 18/18 and 4/9 collected delineations of the GTVpre, GTVpost, CTV-T and CTV-N, respectively. An unacceptable variation from the CTVref was found for 7/9 participants for case 5 and 6/9 participants for case 6. CONCLUSION This international multicenter delineation exercise demonstrates that the new consensus for highly-conformal postoperative flank target volume delineation leads to geometrical variation among participants. Moreover, standardized review showed an unacceptable delineation variation in the majority of the participants. These findings strongly suggest the need for additional training and centralized pre-treatment review when this target volume delineation approach is implemented on a larger scale.
Collapse
Key Words
- AA, abdominal aorta
- AP/PA, Anterior-Posterior/Posterior-Anterior
- CT, Computed Tomography
- CTV-N, Clinical Target Volume of the lymph node area
- CTV-T, Clinical Target Volume of the primary Tumor
- DICOM, Digital Imaging and Communications in Medicine
- DSC, Dice Similarity Coefficient
- Flank target volume
- GTVpre/post, pre- and postoperative Gross Tumor Volume respectively
- HR, High-Risk
- Highly-conformal radiotherapy
- IGRT, Image-Guided Radiotherapy
- IMRT, Intensity-Modulated Radiotherapy
- IR, Intermediate-Risk
- IVC, inferior vena cava
- Inter-clinician variation
- MRI, Magnetic Resonance Imaging
- OAR, organs at risk
- Pediatric renal tumors
- Quality assurance
- RT, radiotherapy
- RTOG, Radiation Oncology Group
- RTSG, Renal Tumor Study Group
- SIOP, International Society for Pediatric Oncology
- TVintersect, intersect target volume
- TVref, reference target volumes
- WT, Wilms’ tumor
- Wilms tumor
- n.a., not applicable
- part, participant
- ref, reference
Collapse
Affiliation(s)
- Joeri Mul
- Princess Máxima Center for Pediatric Oncology, Utrecht, the Netherlands
- Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Patrick Melchior
- Dept. of Radiation Oncology, Saarland University Hospital, Homburg, Germany
| | - Enrica Seravalli
- Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Daniel Saunders
- Dept. of Clinical Oncology, The Christie Hospital, Manchester, United Kingdom
| | - Stephanie Bolle
- Dept. of Radiation Oncology, Gustave Roussy, Villejuif, France
| | - Alison L. Cameron
- Bristol Cancer Institute, University Hospitals, Bristol, United Kingdom
| | - Kristin Gurtner
- Dept. of Radiation Oncology, University Hospital Carl Gustav Carus, Dresden, Germany
| | - Semi Harrabi
- Dept. of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | | | - Naomi Lavan
- St. Luke’s Radiation Oncology Network, Dublin, Ireland
| | | | - Henry Mandeville
- Dept. of Clinical Oncology, The Royal Marsden NHS Foundation Trust, Sutton, United Kingdom
| | - Tom Boterberg
- Dept. of Radiation Oncology, Ghent University Hospital, Ghent, Belgium
| | - Petra S. Kroon
- Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Alexis N.T.J. Kotte
- Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Bianca A.W. Hoeben
- Princess Máxima Center for Pediatric Oncology, Utrecht, the Netherlands
- Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Peter S.N. van Rossum
- Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | | | - Norbert Graf
- Dept. of Pediatric Oncology, Saarland University Hospital, Homburg, Germany
| | | | - Christian Rübe
- Dept. of Radiation Oncology, Saarland University Hospital, Homburg, Germany
| | - Geert O. Janssens
- Princess Máxima Center for Pediatric Oncology, Utrecht, the Netherlands
- Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| |
Collapse
|
47
|
Mascheroni J, Mont L, Stockburger M, Patwala A, Retzlaff H, Gallagher AG. The imperative of consistency and proficiency in cardiac devices implant skills training. Open Heart 2021; 8:e001629. [PMID: 33972405 PMCID: PMC8112444 DOI: 10.1136/openhrt-2021-001629] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/21/2021] [Indexed: 11/15/2022] Open
Affiliation(s)
- Jorio Mascheroni
- College of Medicine and Health, University College Cork, Cork, Ireland
| | - Lluís Mont
- Department of Cardiology, Hospital Clinic, Universitat de Barcelona, IDIBAPS, Barcelona, Spain
| | - Martin Stockburger
- Department of Cardiology and Internal Medicine, Havelland Kliniken, Nauen, Germany
- Department of Cardiology and Angiology, Charité - Universitaetsmedizin Berlin, Berlin, Germany
| | - Ashish Patwala
- Department of Cardiology, University Hospital of North Midlands, Stoke-on-Trent, UK
| | | | - Anthony G Gallagher
- Orsi Academy, Melle, Belgium
- Faculty of Medicine, KU Leuven, Leuven, Belgium
- School of Medicine, Faculty of Life and Health Sciences, Ulster University, Londonderry, UK
| |
Collapse
|
48
|
Editorial Commentary: Proficiency-Based Progression Surgical Simulation Training Is an Efficient Adjunct to High-Volume Clinical Experience. Arthroscopy 2021; 37:1107-1109. [PMID: 33812515 DOI: 10.1016/j.arthro.2021.02.013] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/12/2021] [Accepted: 02/12/2021] [Indexed: 02/02/2023]
Abstract
The optimal way to train a future surgeon has been debated for years, with strategies ranging from the well-known "see one, do one, teach one" approach to more novel approaches that rely on metrics and proficiency. Recent research shows that surgical training with a proficiency-based progression curriculum is an efficient strategy for teaching arthroscopy procedural skills, and, further, may improve patient safety by reducing the technical errors that might otherwise occur before proficiency is achieved. While every surgical specialty has its nuances that must be mastered to provide safe, effective, and efficient care, for a variety of reasons, the skills needed to perform arthroscopy are incredibly difficult to learn, let alone achieve proficiency or master. "On-the-job" training for orthopaedic residents has become more difficult in today's fast-paced, work hour-limited, volume-rewarded society. Proficiency-based progression is a piece of the puzzle, but for now, it is not a complete substitute for high-volume, clinical experience and exposure to the countless variables that may affect a "real-life" surgical procedure.
Collapse
|
49
|
Angelo RL, St Pierre P, Tauro J, Gallagher AG. A Proficiency-Based Progression Simulation Training Curriculum to Acquire the Skills Needed in Performing Arthroscopic Bankart and Rotator Cuff Repairs-Implementation and Impact. Arthroscopy 2021; 37:1099-1106.e5. [PMID: 33359814 DOI: 10.1016/j.arthro.2020.11.040] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/02/2020] [Revised: 11/12/2020] [Accepted: 11/13/2020] [Indexed: 02/02/2023]
Abstract
PURPOSE To investigate the impact of a proficiency-based progression (PBP) curriculum employed to teach trainees in the skills needed to demonstrate proficiency for an arthroscopic Bankart repair (ABR) and an arthroscopic rotator cuff repair (ARCR) by objectively comparing pre- and immediate postcourse performances. METHODS In a prospective study, 16 arthroscopy/sports medicine fellows and 2 senior residents (complete group: N = 18) were randomly assigned to perform a precourse cadaveric ABR (Bankart subgroup: N = 6), ARCR (cuff subgroup: N = 6), or basic skills on a shoulder simulator (N = 6). After completing a PBP training curriculum, all 18 registrants performed both an ABR and ARCR scored in real time by trained raters using previously validated metrics. RESULTS The Bankart subgroup made 58% fewer objectively assessed errors at the completion of the course than at baseline (P = .004, confidence interval -1.449 to -0.281), and performance variability was substantially reduced (standard deviation = 5.89 vs 2.81). The cuff subgroup also made 58% fewer errors (P = .001, confidence interval -1.376 to 0.382) and showed a similar reduction in performance variability (standard deviation = 5.42 vs 2.1). Only one subject's precourse baseline performance met the proficiency benchmark compared with 89% and 83% of the all registrants on the final ABR and ARCR cadaveric assessments, respectively. CONCLUSIONS The results of this study reject the null hypothesis. They demonstrate that the implementation of a PBP simulation curriculum to train the skills necessary to perform arthroscopic Bankart and rotator cuff repairs results in a large and statistically significant improvement in the trainee's ability to meet the 2 related performance benchmarks. Proficiency was demonstrated by 89% and 83% of the trainees for an ABR and an ARCR, respectively, in a two- and one-half day course. CLINICAL RELEVANCE Surgical training employing a PBP curriculum is efficient, effective, and has the potential to improve patient safety.
Collapse
Affiliation(s)
| | - Pat St Pierre
- Desert Orthopedic Center, Shoulder and Elbow Service, Palm Desert, California, U.S.A
| | - Joe Tauro
- Orthopedic Surgery, Rutgers Medical School, Newark, New Jersey, U.S.A
| | - Anthony G Gallagher
- Faculty of Life and Health Sciences, Ulster University, Londonderry, United Kingdom; ORSI Academy, Melle, Belgium
| | | |
Collapse
|
50
|
Lakhani S, Selim OA, Saeed MZ. Arthroscopic Simulation: The Future of Surgical Training: A Systematic Review. JBJS Rev 2021; 9:01874474-202103000-00006. [PMID: 33750750 DOI: 10.2106/jbjs.rvw.20.00076] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
Abstract
BACKGROUND Arthroscopic simulation has rapidly evolved recently with the introduction of higher-fidelity simulation models, such as virtual reality simulators, which provide trainees an environment to practice skills without causing undue harm to patients. Simulation training also offers a uniform approach to learn surgical skills with immediate feedback. The aim of this article is to review the recent research investigating the use of arthroscopy simulators in training and the teaching of surgical skills. METHODS A systematic review of the Embase, MEDLINE, and Cochrane Library databases for English-language articles published before December 2019 was conducted. The search terms included arthroscopy or arthroscopic in combination with simulation or simulator. RESULTS We identified a total of 44 relevant studies involving benchtop or virtually simulated ankle, knee, shoulder, and hip arthroscopy environments. The majority of these studies demonstrated construct and transfer validity; considerably fewer studies demonstrated content and face validity. CONCLUSIONS Our review indicates that there is a considerable evidence base regarding the use of arthroscopy simulators for training purposes. Further work should focus on the development of a more uniform simulator training course that can be compared with current intraoperative training in large-scale trials with long-term follow-up at tertiary centers.
Collapse
Affiliation(s)
- Saad Lakhani
- Division of Surgical & Interventional Sciences, University College London, London, United Kingdom
| | - Omar A Selim
- Division of Surgical & Interventional Sciences, University College London, London, United Kingdom
| | | |
Collapse
|