|
1
|
Küper A, Lodde GC, Livingstone E, Schadendorf D, Krämer N. Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists. J Med Internet Res 2025; 27:e58660. [PMID: 40184614 DOI: 10.2196/58660] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2024] [Revised: 12/06/2024] [Accepted: 03/08/2025] [Indexed: 04/06/2025] Open
Abstract
BACKGROUND Artificial intelligence (AI)-enabled decision support systems are critical tools in medical practice; however, their reliability is not absolute, necessitating human oversight for final decision-making. Human reliance on such systems can vary, influenced by factors such as individual psychological factors and physician experience. OBJECTIVE This study aimed to explore the psychological factors influencing subjective trust and reliance on medical AI's advice, specifically examining relative AI reliance and relative self-reliance to assess the appropriateness of reliance. METHODS A survey was conducted with 223 dermatologists, which included lesion image classification tasks and validated questionnaires assessing subjective trust, propensity to trust technology, affinity for technology interaction, control beliefs, need for cognition, as well as queries on medical experience and decision confidence. RESULTS A 2-tailed t test revealed that participants' accuracy improved significantly with AI support (t222=-3.3; P<.001; Cohen d=4.5), but only by an average of 1% (1/100). Reliance on AI was stronger for correct advice than for incorrect advice (t222=4.2; P<.001; Cohen d=0.1). Notably, participants demonstrated a mean relative AI reliance of 10.04% (139/1384) and a relative self-reliance of 85.6% (487/569), indicating a high level of self-reliance but a low level of AI reliance. Propensity to trust technology influenced AI reliance, mediated by trust (indirect effect=0.024, 95% CI 0.008-0.042; P<.001), and medical experience negatively predicted AI reliance (indirect effect=-0.001, 95% CI -0.002 to -0.001; P<.001). CONCLUSIONS The findings highlight the need to design AI support systems in a way that assists less experienced users with a high propensity to trust technology to identify potential AI errors, while encouraging experienced physicians to actively engage with system recommendations and potentially reassess initial decisions.
Collapse
Affiliation(s)
- Alisa Küper
- Social Psychology: Media and Communication, University of Duisburg-Essen, Duisburg, Germany
| | | | | | - Dirk Schadendorf
- Department of Dermatology, University Hospital Essen, Essen, Germany
| | - Nicole Krämer
- Social Psychology: Media and Communication, University of Duisburg-Essen, Duisburg, Germany
| |
Collapse
|
|
2
|
Spota A, Hassanpour A, Gomez D, Al-Sukhni E. Use of risk assessment tools in emergency general surgery: a cross-sectional survey of surgeons and trainees. Updates Surg 2025; 77:605-613. [PMID: 39825020 DOI: 10.1007/s13304-025-02089-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2024] [Accepted: 01/07/2025] [Indexed: 01/20/2025]
Abstract
The applicability of risk assessment tools (RATs) for preoperative risk assessment (PRA) in Emergency General Surgery (EGS) is unclear. Limited knowledge of surgeons' approach to risk assessment is available. We investigated how Canadian surgeons approach PRA for EGS and their awareness of available RATs. Canadian Association of General Surgeons members were invited to complete an online cross-sectional survey. Descriptive statistics were reported. Of 278 respondents, 70% were attending surgeons (44% had 5-10 years in practice, 43% > 10 years), 5% fellows, and 25% residents. Most worked in medium-/large-volume centers (89%) and teaching hospitals (77%). During preoperative risk assessment, 2/3 of respondents reported applying clinical experience/instinct and referring to literature, while 55% used RATs. The best-known and used tools were the ACS-NSQIP calculator (68% and 59%) and the Emergency Surgery Acuity Score (ESAS, 66% and 47%, respectively). Surgeons were divided regarding the accuracy of RAT estimates, with 47% considering them generally accurate and 49% inaccurate. Trainees reported greater interest in major morbidity risk (86% vs. 65%) and probability of supported discharge (45% vs. 29%) than surgeons. Among participants not using RATs, 41% indicated they are scarcely accessible in the EGS context, while 33% found them cumbersome and time-consuming. RATs are underused in favor of personal judgment. The use of RATs may facilitate decision-making in elderly/complex patients and help reduce variability in practice, particularly for trainees and less-experienced surgeons. A greater effort in education is needed to spread the culture of RATs for PRA.
Collapse
Affiliation(s)
- Andrea Spota
- Department of Surgery, University Health Network, 200 Elizabeth St, 10 Eaton North, Room 216, Toronto, ON, M5G 2C4, Canada.
| | - Amir Hassanpour
- Department of Surgery, University Health Network, 200 Elizabeth St, 10 Eaton North, Room 216, Toronto, ON, M5G 2C4, Canada
| | - David Gomez
- Department of Surgery, St. Michael's Hospital-Unity Health, Toronto, ON, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Eisar Al-Sukhni
- Department of Surgery, University Health Network, 200 Elizabeth St, 10 Eaton North, Room 216, Toronto, ON, M5G 2C4, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
|
3
|
Henn J, Vandemeulebroucke T, Hatterscheidt S, Dohmen J, Kalff JC, van Wynsberghe A, Matthaei H. German surgeons' perspective on the application of artificial intelligence in clinical decision-making. Int J Comput Assist Radiol Surg 2025:10.1007/s11548-025-03326-z. [PMID: 39907950 DOI: 10.1007/s11548-025-03326-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2024] [Accepted: 01/14/2025] [Indexed: 02/06/2025]
Abstract
PURPOSE Artificial intelligence (AI) is transforming clinical decision-making (CDM). This application of AI should be a conscious choice to avoid technological determinism. The surgeons' perspective is needed to guide further implementation. METHODS We conducted an online survey among German surgeons, focusing on digitalization and AI in CDM, specifically for acute abdominal pain (AAP). The survey included Likert items and scales. RESULTS We analyzed 263 responses. Seventy-one percentage of participants were male, with a median age of 49 years (IQR 41-57). Seventy-three percentage of participants carried out a senior role, with a median of 22 years of work experience (IQR 13-28). AI in CDM was seen as helpful for workload management (48%) but not for preventing unnecessary treatments (32%). Safety (95%), evidence (94%), and usability (96%) were prioritized over costs (43%) for the implementation. Concerns included the loss of practical CDM skills (81%) and ethical issues like transparency (52%), patient trust (45%), and physician integrity (44%). Traditional CDM for AAP was seen as experience-based (93%) and not standardized (31%), whereas AI was perceived to assist with urgency triage (60%) and resource management (59%). On median, generation Y showed more confidence in AI for CDM (P = 0.001), while participants working in primary care hospitals were less confident (P = 0.021). CONCLUSION Participants saw the potential of AI for organizational tasks but are hesitant about its use in CDM. Concerns about trust and performance need to be addressed through education and critical evaluation. In the future, AI might provide sufficient decision support but will not replace the human component.
Collapse
Affiliation(s)
- Jonas Henn
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany.
- Bonn Surgical Technology Center (BOSTER), University Hospital Bonn, Bonn, Germany.
| | - Tijs Vandemeulebroucke
- Bonn Sustainable AI Lab, Institute of Science and Ethics, University of Bonn, Bonn, Germany
| | - Simon Hatterscheidt
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
- Bonn Surgical Technology Center (BOSTER), University Hospital Bonn, Bonn, Germany
| | - Jonas Dohmen
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
- Bonn Surgical Technology Center (BOSTER), University Hospital Bonn, Bonn, Germany
| | - Jörg C Kalff
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
- Bonn Surgical Technology Center (BOSTER), University Hospital Bonn, Bonn, Germany
| | - Aimee van Wynsberghe
- Bonn Sustainable AI Lab, Institute of Science and Ethics, University of Bonn, Bonn, Germany
| | - Hanno Matthaei
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
- Bonn Surgical Technology Center (BOSTER), University Hospital Bonn, Bonn, Germany
| |
Collapse
|
|
4
|
Balch JA, Ruppert MM, Guan Z, Buchanan TR, Abbott KL, Shickel B, Bihorac A, Liang M, Upchurch GR, Tignanelli CJ, Loftus TJ. Risk-Specific Training Cohorts to Address Class Imbalance in Surgical Risk Prediction. JAMA Surg 2024; 159:1424-1431. [PMID: 39382865 PMCID: PMC11465118 DOI: 10.1001/jamasurg.2024.4299] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2024] [Accepted: 08/05/2024] [Indexed: 10/10/2024]
Abstract
Importance Machine learning tools are increasingly deployed for risk prediction and clinical decision support in surgery. Class imbalance adversely impacts predictive performance, especially for low-incidence complications. Objective To evaluate risk-prediction model performance when trained on risk-specific cohorts. Design, Setting, and Participants This cross-sectional study performed from February 2024 to July 2024 deployed a deep learning model, which generated risk scores for common postoperative complications. A total of 109 445 inpatient operations performed at 2 University of Florida Health hospitals from June 1, 2014, to May 5, 2021 were examined. Exposures The model was trained de novo on separate cohorts for high-risk, medium-risk, and low-risk Common Procedure Terminology codes defined empirically by incidence of 5 postoperative complications: (1) in-hospital mortality; (2) prolonged intensive care unit (ICU) stay (≥48 hours); (3) prolonged mechanical ventilation (≥48 hours); (4) sepsis; and (5) acute kidney injury (AKI). Low-risk and high-risk cutoffs for complications were defined by the lower-third and upper-third prevalence in the dataset, except for mortality, cutoffs for which were set at 1% or less and greater than 3%, respectively. Main Outcomes and Measures Model performance metrics were assessed for each risk-specific cohort alongside the baseline model. Metrics included area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), F1 scores, and accuracy for each model. Results A total of 109 445 inpatient operations were examined among patients treated at 2 University of Florida Health hospitals in Gainesville (77 921 procedures [71.2%]) and Jacksonville (31 524 procedures [28.8%]). Median (IQR) patient age was 58 (43-68) years, and median (IQR) Charlson Comorbidity Index score was 2 (0-4). Among 109 445 operations, 55 646 patients were male (50.8%), and 66 495 patients (60.8%) underwent a nonemergent, inpatient operation. Training on the high-risk cohort had variable impact on AUROC, but significantly improved AUPRC (as assessed by nonoverlapping 95% confidence intervals) for predicting mortality (0.53; 95% CI, 0.43-0.64), AKI (0.61; 95% CI, 0.58-0.65), and prolonged ICU stay (0.91; 95% CI, 0.89-0.92). It also significantly improved F1 score for mortality (0.42; 95% CI, 0.36-0.49), prolonged mechanical ventilation (0.55; 95% CI, 0.52-0.58), sepsis (0.46; 95% CI, 0.43-0.49), and AKI (0.57; 95% CI, 0.54-0.59). After controlling for baseline model performance on high-risk cohorts, AUPRC increased significantly for in-hospital mortality only (0.53; 95% CI, 0.42-0.65 vs 0.29; 95% CI, 0.21-0.40). Conclusion and Relevance In this cross-sectional study, by training separate models using a priori knowledge for procedure-specific risk classes, improved performance in standard evaluation metrics was observed, especially for low-prevalence complications like in-hospital mortality. Used cautiously, this approach may represent an optimal training strategy for surgical risk-prediction models.
Collapse
Affiliation(s)
- Jeremy A. Balch
- Department of Surgery, University of Florida, Gainesville
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville
- Intelligent Clinical Care Center, College of Medicine, University of Florida, Gainesville
| | | | - Ziyuan Guan
- Intelligent Clinical Care Center, College of Medicine, University of Florida, Gainesville
| | | | | | - Benjamin Shickel
- Intelligent Clinical Care Center, College of Medicine, University of Florida, Gainesville
| | - Azra Bihorac
- Intelligent Clinical Care Center, College of Medicine, University of Florida, Gainesville
| | - Muxuan Liang
- College of Medicine, University of Florida, Gainesville
| | | | | | - Tyler J. Loftus
- Department of Surgery, University of Florida, Gainesville
- Intelligent Clinical Care Center, College of Medicine, University of Florida, Gainesville
| |
Collapse
|
|
5
|
Kiwan O, Al-Kalbani M, Rafie A, Hijazi Y. Artificial intelligence in plastic surgery, where do we stand? JPRAS Open 2024; 42:234-243. [PMID: 39435018 PMCID: PMC11491964 DOI: 10.1016/j.jpra.2024.09.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2024] [Accepted: 09/05/2024] [Indexed: 10/23/2024] Open
Abstract
Since the pandemic, artificial intelligence (AI) has been integrated into several fields and everyday life as well. Healthcare is not an exception. Plastic surgery is a key focus area of this technological revolution, with hundreds of studies and reviews already published on the use of AI in plastics. This review summarizes the entirety of the available literature from 2020 to provide a comprehensive overview on AI innovation in plastic surgery. A systematic literature review (following the PRISMA guidelines) of all studies and papers that examined the application of AI in plastic surgery was carried out using Medline, Cochrane, Embase, and Google Scholar. Outcomes of interest included the growing role of AI in clinical consultations, diagnosing potentials, surgical planning, intraoperative, and post-operative uses. Ninety-six studies were included in this review; six examined the role of AI in consultations, fifteen used AI in diagnoses and assessments, seventeen involved AI in surgical planning, fifteen reported on AI use in post-operative predictions and management, and nine involved administrations and documentation. This comprehensive review of available literature found AI to be capable of transforming care throughout the entire patient journey. Certain challenges and concerns persist, but a collaborative effort can solve these issues to bring about a new era of medicine, where AI aids doctors in the pursuit of optimal patient care.
Collapse
Affiliation(s)
- Omar Kiwan
- Faculty of Biology, Medicine and Health, University of Manchester, United Kingdom
| | - Mohammed Al-Kalbani
- Faculty of Biology, Medicine and Health, University of Manchester, United Kingdom
| | - Arash Rafie
- Plastic and Reconstructive Department, Lancashire Teaching Hospitals NHS Foundation, United Kingdom
| | - Yasser Hijazi
- Plastic and Reconstructive Department, Lancashire Teaching Hospitals NHS Foundation, United Kingdom
| |
Collapse
|
|
6
|
Fritz BA, King CR, Abdelhack M, Chen Y, Kronzer A, Abraham J, Tripathi S, Ben Abdallah A, Kannampallil T, Budelier TP, Helsten D, Montes de Oca A, Mehta D, Sontha P, Higo O, Kerby P, Gregory SH, Wildes TS, Avidan MS. Effect of machine learning models on clinician prediction of postoperative complications: the Perioperative ORACLE randomised clinical trial. Br J Anaesth 2024; 133:1042-1050. [PMID: 39261226 PMCID: PMC11488162 DOI: 10.1016/j.bja.2024.08.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2024] [Revised: 07/19/2024] [Accepted: 08/07/2024] [Indexed: 09/13/2024] Open
Abstract
BACKGROUND Anaesthesiologists might be able to mitigate risk if they know which patients are at greatest risk for postoperative complications. This trial examined the impact of machine learning models on clinician risk assessment. METHODS This single-centre, prospective, randomised clinical trial enrolled surgical patients aged ≥18 yr. Anaesthesiologists and nurse anaesthetists providing remote telemedicine support reviewed electronic health records with (assisted group) or without (unassisted group) reviewing machine learning predictions. Clinicians predicted the likelihood of postoperative 30-day all-cause mortality and postoperative acute kidney injury (AKI) within 7 days. The primary outcome was area under the receiver operating characteristic curve (AUROC) for clinician predictions of mortality and AKI, comparing AUROCs between assisted and unassisted assessments. RESULTS We analysed 5071 patients (mean [range] age: 58 [18-100] yr; 52% female) assessed by 89 clinicians. Of these, 98 (2.2%) patients died within 30 days of surgery and 450 (11.1%) patients sustained AKI. Clinician predictions agreed with the models more strongly in the assisted vs unassisted group (weighted kappa 0.75 vs 0.62 for death, mean difference: 0.13 [95% CI 0.10-0.17]; and 0.79 vs 0.54 for AKI, mean difference: 0.25 [95% CI 0.21-0.29]). Clinical prediction of death was similar between the assisted (AUROC 0.793) and unassisted (AUROC 0.780) groups (mean difference: 0.013 [95% CI -0.070 to 0.097]; P=0.76). Prediction of AKI had an AUROC of 0.734 in the assisted group vs 0.688 in the unassisted group (difference 0.046 [95% CI -0.003 to 0.091]; P=0.06). CONCLUSIONS Clinician performance was not improved by machine learning assistance. Further work is needed to clarify the role of machine learning in real-time perioperative risk stratification. CLINICAL TRIAL REGISTRATION NCT05042804.
Collapse
Affiliation(s)
- Bradley A Fritz
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA.
| | - Christopher R King
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Mohamed Abdelhack
- Department of Computer Science and Engineering, Washington University McKelvey School of Engineering, Saint Louis, MO, USA; Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health, Toronto, ON, Canada
| | - Yixin Chen
- Department of Computer Science and Engineering, Washington University McKelvey School of Engineering, Saint Louis, MO, USA
| | - Alex Kronzer
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Joanna Abraham
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA; Institute for Informatics, Data Science, and Biostatistics, Washington University School of Medicine, Saint Louis, MO, USA
| | - Sandhya Tripathi
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Arbi Ben Abdallah
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Thomas Kannampallil
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA; Institute for Informatics, Data Science, and Biostatistics, Washington University School of Medicine, Saint Louis, MO, USA
| | - Thaddeus P Budelier
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Daniel Helsten
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Arianna Montes de Oca
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Divya Mehta
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Pratyush Sontha
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Omokhaye Higo
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Paul Kerby
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Stephen H Gregory
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Troy S Wildes
- Department of Anesthesiology, University of Nebraska Medical Center, Omaha, NE, USA
| | - Michael S Avidan
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| |
Collapse
|
|
7
|
Al Harbi M, Alotaibi A, Alanazi A, Alsughayir F, Alharbi D, Bin Qassim A, Alkhwaiter T, Olayan L, Al Zaid M, Alsabani M. Perspectives toward the application of Artificial Intelligence in anesthesiology-related practices in Saudi Arabia: A cross-sectional study of physicians views. Health Sci Rep 2024; 7:e70099. [PMID: 39410950 PMCID: PMC11473377 DOI: 10.1002/hsr2.70099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2024] [Revised: 09/03/2024] [Accepted: 09/05/2024] [Indexed: 10/19/2024] Open
Abstract
Background and Aims The use of Artificial Intelligence (AI) relies on computer science and large datasets, with the technology mimicking human intelligence as it makes logical decisions. This study aims to assess the perceptions and experiences of anesthesiology practitioners toward AI and identify its benefits to healthcare professionals and patients, along with current and future applications of AI. Methods This cross-sectional descriptive online survey study was disseminated to physicians who work in anesthesiology practice in Saudi Arabia. Descriptive statistics were used to report the characteristics of the respondents and summarize the results of the survey. Results There were 109 responses, with 85.32% being male, 35.78% being aged 40-49 years, and 69.72% being consultant anesthesiologists. The majority of participants (73.39%) believed that AI could be used in multiple settings related to anesthesiology practice. Participants also believed that AI could facilitate access to data (76.15%), enable precise decision-making (75.23%), reduce medical errors (55.04%), reduce workload and shortage of healthcare personnel (53.21%), and allow healthcare personnel to focus on more demanding cases (69.72%). In addition, the majority of participants believed that AI can be beneficial to patients, in which 69.72% believed that AI can improve patient access to care, 77.06% believed that AI can facilitate patient education, and 65.14% believed that AI can guide patients during treatment. Lastly, 70.64% believed that AI would be beneficial to anesthesiology practices in the future. However, 61.47% claimed that their workplace has no plan for adopting AI. Conclusions The anesthesiologists showed generally positive attitudes towards AI, in spite of its limited utilization and implementation challenges. Strong beliefs exist about AI's future potential in anesthesia care and postgraduate education.
Collapse
Affiliation(s)
- Mohammed Al Harbi
- Department of Anesthesia Ministry of National Guard Health Affairs Riyadh Saudi Arabia
- King Abdullah International Medical Research Centre Riyadh Saudi Arabia
- College of Medicine King Saud bin Abdulaziz University for Health Sciences Riyadh Saudi Arabia
| | - Ahmed Alotaibi
- King Abdullah International Medical Research Centre Riyadh Saudi Arabia
- Anesthesia Technology Department, College of Applied Medical Sciences King Saud bin Abdulaziz University for Health Sciences Riyadh Saudi Arabia
| | - Amal Alanazi
- College of Medicine King Saud bin Abdulaziz University for Health Sciences Riyadh Saudi Arabia
| | - Fatimah Alsughayir
- College of Medicine King Saud bin Abdulaziz University for Health Sciences Riyadh Saudi Arabia
| | - Deema Alharbi
- College of Medicine University of Tabuk Tabuk Saudi Arabia
| | - Ahmad Bin Qassim
- College of Medicine Imam Mohammad ibn Saud Islamic University Riyadh Saudi Arabia
| | - Talal Alkhwaiter
- College of Medicine Imam Mohammad ibn Saud Islamic University Riyadh Saudi Arabia
| | - Lafi Olayan
- King Abdullah International Medical Research Centre Riyadh Saudi Arabia
- Anesthesia Technology Department, College of Applied Medical Sciences King Saud bin Abdulaziz University for Health Sciences Riyadh Saudi Arabia
| | - Manal Al Zaid
- King Abdullah International Medical Research Centre Riyadh Saudi Arabia
- College of Medicine King Saud bin Abdulaziz University for Health Sciences Riyadh Saudi Arabia
- Department of Surgery Ministry of National Guard Health Affairs Riyadh Saudi Arabia
| | - Mohmad Alsabani
- King Abdullah International Medical Research Centre Riyadh Saudi Arabia
- Anesthesia Technology Department, College of Applied Medical Sciences King Saud bin Abdulaziz University for Health Sciences Riyadh Saudi Arabia
| |
Collapse
|
|
8
|
Shukla A, Chaudhary R, Nayyar N. Role of artificial intelligence in gastrointestinal surgery. Artif Intell Cancer 2024; 5:97317. [DOI: 10.35713/aic.v5.i2.97317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/28/2024] [Revised: 07/11/2024] [Accepted: 07/17/2024] [Indexed: 09/05/2024] Open
Abstract
Artificial intelligence is rapidly evolving and its application is increasing day-by-day in the medical field. The application of artificial intelligence is also valuable in gastrointestinal diseases, by calculating various scoring systems, evaluating radiological images, preoperative and intraoperative assistance, processing pathological slides, prognosticating, and in treatment responses. This field has a promising future and can have an impact on many management algorithms. In this minireview, we aimed to determine the basics of artificial intelligence, the role that artificial intelligence may play in gastrointestinal surgeries and malignancies, and the limitations thereof.
Collapse
Affiliation(s)
- Ankit Shukla
- Department of Surgery, Dr Rajendra Prasad Government Medical College, Kangra 176001, Himachal Pradesh, India
| | - Rajesh Chaudhary
- Department of Renal Transplantation, Dr Rajendra Prasad Government Medical College, Kangra 176001, India
| | - Nishant Nayyar
- Department of Radiology, Dr Rajendra Prasad Government Medical College, Kangra 176001, Himachal Pradesh, India
| |
Collapse
|
|
9
|
Moosavi A, Huang S, Vahabi M, Motamedivafa B, Tian N, Mahmood R, Liu P, Sun CL. Prospective Human Validation of Artificial Intelligence Interventions in Cardiology: A Scoping Review. JACC. ADVANCES 2024; 3:101202. [PMID: 39372457 PMCID: PMC11450923 DOI: 10.1016/j.jacadv.2024.101202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/22/2024] [Revised: 07/09/2024] [Accepted: 07/11/2024] [Indexed: 10/08/2024]
Abstract
Background Despite the potential of artificial intelligence (AI) in enhancing cardiovascular care, its integration into clinical practice is limited by a lack of evidence on its effectiveness with respect to human experts or gold standard practices in real-world settings. Objectives The purpose of this study was to identify AI interventions in cardiology that have been prospectively validated against human expert benchmarks or gold standard practices, assessing their effectiveness, and identifying future research areas. Methods We systematically reviewed Scopus and MEDLINE to identify peer-reviewed publications that involved prospective human validation of AI-based interventions in cardiology from January 2015 to December 2023. Results Of 2,351 initial records, 64 studies were included. Among these studies, 59 (92.2%) were published after 2020. A total of 11 (17.2%) randomized controlled trials were published. AI interventions in 44 articles (68.75%) reported definite clinical or operational improvements over human experts. These interventions were mostly used in imaging (n = 14, 21.9%), ejection fraction (n = 10, 15.6%), arrhythmia (n = 9, 14.1%), and coronary artery disease (n = 12, 18.8%) application areas. Convolutional neural networks were the most common predictive model (n = 44, 69%), and images were the most used data type (n = 38, 54.3%). Only 22 (34.4%) studies made their models or data accessible. Conclusions This review identifies the potential of AI in cardiology, with models often performing equally well as human counterparts for specific and clearly scoped tasks suitable for such models. Nonetheless, the limited number of randomized controlled trials emphasizes the need for continued validation, especially in real-world settings that closely examine joint human AI decision-making.
Collapse
Affiliation(s)
- Amirhossein Moosavi
- Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada
- University of Ottawa Heart Institute, University of Ottawa, Ottawa, Ontario, Canada
| | - Steven Huang
- University of Ottawa Heart Institute, University of Ottawa, Ottawa, Ontario, Canada
| | - Maryam Vahabi
- Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada
- University of Ottawa Heart Institute, University of Ottawa, Ottawa, Ontario, Canada
| | - Bahar Motamedivafa
- Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada
- University of Ottawa Heart Institute, University of Ottawa, Ottawa, Ontario, Canada
| | - Nelly Tian
- Marshall School of Business, University of Southern California, Los Angeles, California, USA
| | - Rafid Mahmood
- Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada
| | - Peter Liu
- University of Ottawa Heart Institute, University of Ottawa, Ottawa, Ontario, Canada
| | - Christopher L.F. Sun
- Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada
- University of Ottawa Heart Institute, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
|
10
|
Tan HJ, Spratte BN, Deal AM, Heiling HM, Nazzal EM, Meeks W, Fang R, Teal R, Vu MB, Bennett AV, Blalock SJ, Chung AE, Gotz D, Nielsen ME, Reuland DS, Harris AH, Basch E. Clinical Decision Support for Surgery: A Mixed Methods Study on Design and Implementation Perspectives From Urologists. Urology 2024; 190:15-23. [PMID: 38697362 PMCID: PMC11344670 DOI: 10.1016/j.urology.2024.04.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 04/08/2024] [Accepted: 04/20/2024] [Indexed: 05/05/2024]
Abstract
OBJECTIVE To assess urologist attitudes toward clinical decision support (CDS) embedded into the electronic health record (EHR) and define design needs to facilitate implementation and impact. With recent advances in big data and artificial intelligence (AI), enthusiasm for personalized, data-driven tools to improve surgical decision-making has grown, but the impact of current tools remains limited. METHODS A sequential explanatory mixed methods study from 2019 to 2020 was performed. First, survey responses from the 2019 American Urological Association Annual Census evaluated attitudes toward an automatic CDS tool that would display risk/benefit data. This was followed by the purposeful sampling of 25 urologists and qualitative interviews assessing perspectives on CDS impact and design needs. Bivariable, multivariable, and coding-based thematic analysis were applied and integrated. RESULTS Among a weighted sample of 12,366 practicing urologists, the majority agreed CDS would help decision-making (70.9%, 95% CI 68.7%-73.2%), aid patient counseling (78.5%, 95% CI 76.5%-80.5%), save time (58.1%, 95% CI 55.7%-60.5%), and improve patient outcomes (42.9%, 95% CI 40.5%-45.4%). More years in practice was negatively associated with agreement (P <.001). Urologists described how CDS could bolster evidence-based care, personalized medicine, resource utilization, and patient experience. They also identified multiple implementation barriers and provided suggestions on form, functionality, and visual design to improve usefulness and ease of use. CONCLUSION Urologists have favorable attitudes toward the potential for clinical decision support in the EHR. Smart design will be critical to ensure effective implementation and impact.
Collapse
Affiliation(s)
- Hung-Jui Tan
- Department of Urology, School of Medicine, University of North Carolina, Chapel Hill, NC; Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC.
| | - Brooke N Spratte
- Department of Urology, School of Medicine, University of North Carolina, Chapel Hill, NC
| | - Allison M Deal
- Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC
| | - Hillary M Heiling
- Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC
| | - Elizabeth M Nazzal
- Department of Urology, School of Medicine, University of North Carolina, Chapel Hill, NC
| | - William Meeks
- American Urological Association Data Management and Statistical Services
| | - Raymond Fang
- American Urological Association Data Management and Statistical Services
| | - Randall Teal
- Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC; Connected Health Applications and Interventions Core, University of North Carolina, Chapel Hill, NC
| | - Maihan B Vu
- Connected Health Applications and Interventions Core, University of North Carolina, Chapel Hill, NC; Center for Health Promotion and Disease Prevention, University of North Carolina, Chapel Hill, NC
| | - Antonia V Bennett
- Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC; Department of Health Policy & Management, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC
| | - Susan J Blalock
- Pharmaceutical Outcomes & Policy, Eshelman School of Pharmacy, University of North Carolina, Chapel Hill, NC
| | - Arlene E Chung
- Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC; Department of Bioinformatics, Duke University, Durham, NC
| | - David Gotz
- Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC; School of Information and Library Science, University of North Carolina, Chapel Hill, NC
| | - Matthew E Nielsen
- Department of Urology, School of Medicine, University of North Carolina, Chapel Hill, NC; Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC; Department of Health Policy & Management, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC
| | - Daniel S Reuland
- Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC; Department of Medicine, School of Medicine, University of North Carolina, Chapel Hill, NC
| | - Alex Hs Harris
- Department of Surgery, School of Medicine, Stanford University, Palo Alto, CA
| | - Ethan Basch
- Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC; Department of Health Policy & Management, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC; Department of Medicine, School of Medicine, University of North Carolina, Chapel Hill, NC
| |
Collapse
|
|
11
|
Harris J, Ahluwalia V, Xu K, Romeo D, Fritz C, Rajasekaran K. The efficacy of the National Surgical Quality Improvement Program surgical risk calculator in head and neck surgery: A meta-analysis. Head Neck 2024; 46:1718-1726. [PMID: 38576311 DOI: 10.1002/hed.27765] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Revised: 03/16/2024] [Accepted: 03/26/2024] [Indexed: 04/06/2024] Open
Abstract
BACKGROUND The National Surgical Quality Improvement Program surgical risk calculator (SRC) estimates the risk for postoperative complications. This meta-analysis assesses the efficacy of the SRC in the field of head and neck surgery. METHODS A systematic review identified studies comparing the SRC's predictions to observed outcomes following head and neck surgeries. Predictive accuracy was assessed using receiver operating characteristic curves (AUCs) and Brier scoring. RESULTS Nine studies totaling 1774 patients were included. The SRC underpredicted the risk of all outcomes (including any complication [observed (ob) = 35.9%, predicted (pr) = 21.8%] and serious complication [ob = 28.7%, pr = 17.0%]) except mortality (ob = 0.37%, pr = 1.55%). The observed length of stay was more than twice the predicted length (p < 0.02). Discrimination was acceptable for postoperative pneumonia (AUC = 0.778) and urinary tract infection (AUC = 0.782) only. Predictive accuracy was low for all outcomes (Brier scores ≥0.01) and comparable for patients with and without free-flap reconstructions. CONCLUSION The SRC is an ineffective instrument for predicting outcomes in head and neck surgery.
Collapse
Affiliation(s)
- Jacob Harris
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Vinayak Ahluwalia
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Katherine Xu
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Dominic Romeo
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Christian Fritz
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Karthik Rajasekaran
- Department of Otorhinolaryngology, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
|
12
|
Fritz BA, King CR, Abdelhack M, Chen Y, Kronzer A, Abraham J, Tripathi S, Abdallah AB, Kannampallil T, Budelier TP, Helsten D, Montes de Oca A, Mehta D, Sontha P, Higo O, Kerby P, Gregory SH, Wildes TS, Avidan MS. Effect of Machine Learning on Anaesthesiology Clinician Prediction of Postoperative Complications: The Perioperative ORACLE Randomised Clinical Trial. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.05.22.24307754. [PMID: 38826471 PMCID: PMC11142290 DOI: 10.1101/2024.05.22.24307754] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2024]
Abstract
Background Anaesthesiology clinicians can implement risk mitigation strategies if they know which patients are at greatest risk for postoperative complications. Although machine learning models predicting complications exist, their impact on clinician risk assessment is unknown. Methods This single-centre randomised clinical trial enrolled patients age ≥18 undergoing surgery with anaesthesiology services. Anaesthesiology clinicians providing remote intraoperative telemedicine support reviewed electronic health records with (assisted group) or without (unassisted group) also reviewing machine learning predictions. Clinicians predicted the likelihood of postoperative 30-day all-cause mortality and postoperative acute kidney injury within 7 days. Area under the receiver operating characteristic curve (AUROC) for the clinician predictions was determined. Results Among 5,071 patient cases reviewed by 89 clinicians, the observed incidence was 2% for postoperative death and 11% for acute kidney injury. Clinician predictions agreed with the models more strongly in the assisted versus unassisted group (weighted kappa 0.75 versus 0.62 for death [difference 0.13, 95%CI 0.10-0.17] and 0.79 versus 0.54 for kidney injury [difference 0.25, 95%CI 0.21-0.29]). Clinicians predicted death with AUROC of 0.793 in the assisted group and 0.780 in the unassisted group (difference 0.013, 95%CI -0.070 to 0.097). Clinicians predicted kidney injury with AUROC of 0.734 in the assisted group and 0.688 in the unassisted group (difference 0.046, 95%CI -0.003 to 0.091). Conclusions Although there was evidence that the models influenced clinician predictions, clinician performance was not statistically significantly different with and without machine learning assistance. Further work is needed to clarify the role of machine learning in real-time perioperative risk stratification. Trial Registration ClinicalTrials.gov NCT05042804.
Collapse
Affiliation(s)
- Bradley A Fritz
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Christopher R King
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Mohamed Abdelhack
- Department of Computer Science and Engineering, Washington University McKelvey School of Engineering, Saint Louis, USA
- Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health, Toronto, Canada
| | - Yixin Chen
- Department of Computer Science and Engineering, Washington University McKelvey School of Engineering, Saint Louis, USA
| | - Alex Kronzer
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Joanna Abraham
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
- Institute for Informatics, Data Science and Biostatistics, Washington University School of Medicine, Saint Louis, USA
| | - Sandhya Tripathi
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Arbi Ben Abdallah
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Thomas Kannampallil
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
- Institute for Informatics, Data Science and Biostatistics, Washington University School of Medicine, Saint Louis, USA
| | - Thaddeus P Budelier
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Daniel Helsten
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Arianna Montes de Oca
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Divya Mehta
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Pratyush Sontha
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Omokhaye Higo
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Paul Kerby
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Stephen H. Gregory
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Troy S. Wildes
- Department of Anesthesiology, University of Nebraska Medical Center, Omaha, USA
| | - Michael S Avidan
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| |
Collapse
|
|
13
|
Luțenco V, Țocu G, Guliciuc M, Moraru M, Candussi IL, Dănilă M, Luțenco V, Dimofte F, Mihailov OM, Mihailov R. New Horizons of Artificial Intelligence in Medicine and Surgery. J Clin Med 2024; 13:2532. [PMID: 38731061 PMCID: PMC11084145 DOI: 10.3390/jcm13092532] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2024] [Revised: 04/06/2024] [Accepted: 04/23/2024] [Indexed: 05/13/2024] Open
Abstract
Background: Ideas about Artificial intelligence appeared about half a century ago, but only now is it becoming an essential element of everyday life. The data provided are becoming a bigger pool and we need artificial intelligence that will help us with its superhuman powers. Its interaction with medicine is improving more and more, with medicine being a domain that continues to be perfected. Materials and Methods: The most important databases were used to perform this detailed search that addresses artificial intelligence in the medical and surgical fields. Discussion: Machine learning, deep learning, neural networks and computer vision are some of the mechanisms that are becoming a trend in healthcare worldwide. Developed countries such as Japan, France and Germany have already implemented artificial intelligence in their medical systems. The help it gives is in medical diagnosis, patient monitoring, personalized therapy and workflow optimization. Artificial intelligence will help surgeons to perfect their skills, to standardize techniques and to choose the best surgical techniques. Conclusions: The goal is to predict complications, reduce diagnostic times, diagnose complex pathologies, guide surgeons intraoperatively and reduce medical errors. We are at the beginning of this, and the potential is enormous, but we must not forget the impediments that may appear and slow down its implementation.
Collapse
Affiliation(s)
- Valerii Luțenco
- Surgery I Clinic, Emergency Hospital “Sf. Ap. Andrei”, 800578 Galați, Romania; (V.L.); (R.M.)
| | - George Țocu
- Faculty of Medicine and Pharmacy, “Dunărea de Jos” University of Galati, 800008 Galați, Romania; (M.G.); (M.M.); (I.L.C.); (M.D.); (F.D.)
| | - Mădălin Guliciuc
- Faculty of Medicine and Pharmacy, “Dunărea de Jos” University of Galati, 800008 Galați, Romania; (M.G.); (M.M.); (I.L.C.); (M.D.); (F.D.)
| | - Monica Moraru
- Faculty of Medicine and Pharmacy, “Dunărea de Jos” University of Galati, 800008 Galați, Romania; (M.G.); (M.M.); (I.L.C.); (M.D.); (F.D.)
| | - Iuliana Laura Candussi
- Faculty of Medicine and Pharmacy, “Dunărea de Jos” University of Galati, 800008 Galați, Romania; (M.G.); (M.M.); (I.L.C.); (M.D.); (F.D.)
- Clinical Children Emergency Hospital “Sf. Ioan”, 060011 Galați, Romania;
| | - Marius Dănilă
- Faculty of Medicine and Pharmacy, “Dunărea de Jos” University of Galati, 800008 Galați, Romania; (M.G.); (M.M.); (I.L.C.); (M.D.); (F.D.)
- Clinical Children Emergency Hospital “Sf. Ioan”, 060011 Galați, Romania;
| | - Verginia Luțenco
- Clinical Children Emergency Hospital “Sf. Ioan”, 060011 Galați, Romania;
| | - Florentin Dimofte
- Faculty of Medicine and Pharmacy, “Dunărea de Jos” University of Galati, 800008 Galați, Romania; (M.G.); (M.M.); (I.L.C.); (M.D.); (F.D.)
| | - Oana Mariana Mihailov
- Faculty of Medicine and Pharmacy, “Dunărea de Jos” University of Galati, 800008 Galați, Romania; (M.G.); (M.M.); (I.L.C.); (M.D.); (F.D.)
| | - Raul Mihailov
- Surgery I Clinic, Emergency Hospital “Sf. Ap. Andrei”, 800578 Galați, Romania; (V.L.); (R.M.)
- Faculty of Medicine and Pharmacy, “Dunărea de Jos” University of Galati, 800008 Galați, Romania; (M.G.); (M.M.); (I.L.C.); (M.D.); (F.D.)
| |
Collapse
|
|
14
|
Kowadlo G, Mittelberg Y, Ghomlaghi M, Stiglitz DK, Kishore K, Guha R, Nazareth J, Weinberg L. Development and validation of 'Patient Optimizer' (POP) algorithms for predicting surgical risk with machine learning. BMC Med Inform Decis Mak 2024; 24:70. [PMID: 38468330 DOI: 10.1186/s12911-024-02463-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Accepted: 02/20/2024] [Indexed: 03/13/2024] Open
Abstract
BACKGROUND Pre-operative risk assessment can help clinicians prepare patients for surgery, reducing the risk of perioperative complications, length of hospital stay, readmission and mortality. Further, it can facilitate collaborative decision-making and operational planning. OBJECTIVE To develop effective pre-operative risk assessment algorithms (referred to as Patient Optimizer or POP) using Machine Learning (ML) that predict the development of post-operative complications and provide pilot data to inform the design of a larger prospective study. METHODS After institutional ethics approval, we developed a base model that encapsulates the standard manual approach of combining patient-risk and procedure-risk. In an automated process, additional variables were included and tested with 10-fold cross-validation, and the best performing features were selected. The models were evaluated and confidence intervals calculated using bootstrapping. Clinical expertise was used to restrict the cardinality of categorical variables (e.g. pathology results) by including the most clinically relevant values. The models were created with logistic regression (LR) and extreme gradient-boosted trees using XGBoost (Chen and Guestrin, 2016). We evaluated performance using the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Data was obtained from a metropolitan university teaching hospital from January 2015 to July 2020. Data collection was restricted to adult patients undergoing elective surgery. RESULTS A total of 11,475 adult admissions were included. The performance of XGBoost and LR was very similar across endpoints and metrics. For predicting the risk of any post-operative complication, kidney failure and length-of-stay (LOS), POP with XGBoost achieved an AUROC (95%CI) of 0.755 (0.744, 0.767), 0.869 (0.846, 0.891) and 0.841 (0.833, 0.847) respectively and AUPRC of 0.651 (0.632, 0.669), 0.336 (0.282, 0.390) and 0.741 (0.729, 0.753) respectively. For 30-day readmission and in-patient mortality, POP with XGBoost achieved an AUROC (95%CI) of 0.610 (0.587, 0.635) and 0.866 (0.777, 0.943) respectively and AUPRC of 0.116 (0.104, 0.132) and 0.031 (0.015, 0.072) respectively. CONCLUSION The POP algorithms effectively predicted any post-operative complication, kidney failure and LOS in the sample population. A larger study is justified to improve the algorithm to better predict complications and length of hospital stay. A larger dataset may also improve the prediction of additional specific complications, readmission and mortality.
Collapse
Affiliation(s)
| | | | | | - Daniel K Stiglitz
- Atidia Health, Melbourne, Australia
- Department of Anaesthesiology and Perioperative Medicine, Alfred Health, Melbourne, Australia
| | - Kartik Kishore
- Data Analytics Research and Evaluation Centre, Austin Health, Melbourne, Australia
| | - Ranjan Guha
- Department of Anaesthesia, Austin Health, Heidelberg, Australia
| | - Justin Nazareth
- Department of Anaesthesia, Austin Health, Heidelberg, Australia
| | - Laurence Weinberg
- Department of Anaesthesia, Austin Health, Heidelberg, Australia
- Department of Critical Care, The University of Melbourne, Austin Health, Heidelberg, Australia
| |
Collapse
|
|
15
|
Evans RP, Bryant LD, Russell G, Absolom K. Trust and acceptability of data-driven clinical recommendations in everyday practice: A scoping review. Int J Med Inform 2024; 183:105342. [PMID: 38266426 DOI: 10.1016/j.ijmedinf.2024.105342] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 12/08/2023] [Accepted: 01/14/2024] [Indexed: 01/26/2024]
Abstract
BACKGROUND Increasing attention is being given to the analysis of large health datasets to derive new clinical decision support systems (CDSS). However, few data-driven CDSS are being adopted into clinical practice. Trust in these tools is believed to be fundamental for acceptance and uptake but to date little attention has been given to defining or evaluating trust in clinical settings. OBJECTIVES A scoping review was conducted to explore how and where acceptability and trustworthiness of data-driven CDSS have been assessed from the health professional's perspective. METHODS Medline, Embase, PsycInfo, Web of Science, Scopus, ACM Digital, IEEE Xplore and Google Scholar were searched in March 2022 using terms expanded from: "data-driven" AND "clinical decision support" AND "acceptability". Included studies focused on healthcare practitioner-facing data-driven CDSS, relating directly to clinical care. They included trust or a proxy as an outcome, or in the discussion. The preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScR) is followed in the reporting of this review. RESULTS 3291 papers were screened, with 85 primary research studies eligible for inclusion. Studies covered a diverse range of clinical specialisms and intended contexts, but hypothetical systems (24) outnumbered those in clinical use (18). Twenty-five studies measured trust, via a wide variety of quantitative, qualitative and mixed methods. A further 24 discussed themes of trust without it being explicitly evaluated, and from these, themes of transparency, explainability, and supporting evidence were identified as factors influencing healthcare practitioner trust in data-driven CDSS. CONCLUSION There is a growing body of research on data-driven CDSS, but few studies have explored stakeholder perceptions in depth, with limited focused research on trustworthiness. Further research on healthcare practitioner acceptance, including requirements for transparency and explainability, should inform clinical implementation.
Collapse
Affiliation(s)
- Ruth P Evans
- University of Leeds, Woodhouse Lane, Leeds LS2 9JT, UK.
| | | | - Gregor Russell
- Bradford District Care Trust, Bradford, New Mill, Victoria Rd, BD18 3LD, UK.
| | - Kate Absolom
- University of Leeds, Woodhouse Lane, Leeds LS2 9JT, UK.
| |
Collapse
|
|
16
|
Feinstein M, Katz D, Demaria S, Hofer IS. Remote Monitoring and Artificial Intelligence: Outlook for 2050. Anesth Analg 2024; 138:350-357. [PMID: 38215713 PMCID: PMC10794024 DOI: 10.1213/ane.0000000000006712] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/14/2024]
Abstract
Remote monitoring and artificial intelligence will become common and intertwined in anesthesiology by 2050. In the intraoperative period, technology will lead to the development of integrated monitoring systems that will integrate multiple data streams and allow anesthesiologists to track patients more effectively. This will free up anesthesiologists to focus on more complex tasks, such as managing risk and making value-based decisions. This will also enable the continued integration of remote monitoring and control towers having profound effects on coverage and practice models. In the PACU and ICU, the technology will lead to the development of early warning systems that can identify patients who are at risk of complications, enabling early interventions and more proactive care. The integration of augmented reality will allow for better integration of diverse types of data and better decision-making. Postoperatively, the proliferation of wearable devices that can monitor patient vital signs and track their progress will allow patients to be discharged from the hospital sooner and receive care at home. This will require increased use of telemedicine, which will allow patients to consult with doctors remotely. All of these advances will require changes to legal and regulatory frameworks that will enable new workflows that are different from those familiar to today's providers.
Collapse
Affiliation(s)
- Max Feinstein
- Department of Anesthesiology Pain and Perioperative Medicine, Icahn School of Medicine at Mount Sinai
| | - Daniel Katz
- Department of Anesthesiology Pain and Perioperative Medicine, Icahn School of Medicine at Mount Sinai
| | - Samuel Demaria
- Department of Anesthesiology Pain and Perioperative Medicine, Icahn School of Medicine at Mount Sinai
| | - Ira S. Hofer
- Department of Anesthesiology Pain and Perioperative Medicine, Icahn School of Medicine at Mount Sinai
| |
Collapse
|
|
17
|
Karalis VD. The Integration of Artificial Intelligence into Clinical Practice. APPLIED BIOSCIENCES 2024; 3:14-44. [DOI: 10.3390/applbiosci3010002] [Citation(s) in RCA: 24] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/06/2025]
Abstract
The purpose of this literature review is to provide a fundamental synopsis of current research pertaining to artificial intelligence (AI) within the domain of clinical practice. Artificial intelligence has revolutionized the field of medicine and healthcare by providing innovative solutions to complex problems. One of the most important benefits of AI in clinical practice is its ability to investigate extensive volumes of data with efficiency and precision. This has led to the development of various applications that have improved patient outcomes and reduced the workload of healthcare professionals. AI can support doctors in making more accurate diagnoses and developing personalized treatment plans. Successful examples of AI applications are outlined for a series of medical specialties like cardiology, surgery, gastroenterology, pneumology, nephrology, urology, dermatology, orthopedics, neurology, gynecology, ophthalmology, pediatrics, hematology, and critically ill patients, as well as diagnostic methods. Special reference is made to legal and ethical considerations like accuracy, informed consent, privacy issues, data security, regulatory framework, product liability, explainability, and transparency. Finally, this review closes by critically appraising AI use in clinical practice and its future perspectives. However, it is also important to approach its development and implementation cautiously to ensure ethical considerations are met.
Collapse
Affiliation(s)
- Vangelis D. Karalis
- Department of Pharmacy, School of Health Sciences, National and Kapodistrian University of Athens, 15784 Athens, Greece
- Institute of Applied and Computational Mathematics, Foundation for Research and Technology Hellas (FORTH), 70013 Heraklion, Greece
| |
Collapse
|
|
18
|
Arina P, Kaczorek MR, Hofmaenner DA, Pisciotta W, Refinetti P, Singer M, Mazomenos EB, Whittle J. Prediction of Complications and Prognostication in Perioperative Medicine: A Systematic Review and PROBAST Assessment of Machine Learning Tools. Anesthesiology 2024; 140:85-101. [PMID: 37944114 PMCID: PMC11146190 DOI: 10.1097/aln.0000000000004764] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Indexed: 11/12/2023]
Abstract
BACKGROUND The utilization of artificial intelligence and machine learning as diagnostic and predictive tools in perioperative medicine holds great promise. Indeed, many studies have been performed in recent years to explore the potential. The purpose of this systematic review is to assess the current state of machine learning in perioperative medicine, its utility in prediction of complications and prognostication, and limitations related to bias and validation. METHODS A multidisciplinary team of clinicians and engineers conducted a systematic review using the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) protocol. Multiple databases were searched, including Scopus, Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane Library, PubMed, Medline, Embase, and Web of Science. The systematic review focused on study design, type of machine learning model used, validation techniques applied, and reported model performance on prediction of complications and prognostication. This review further classified outcomes and machine learning applications using an ad hoc classification system. The Prediction model Risk Of Bias Assessment Tool (PROBAST) was used to assess risk of bias and applicability of the studies. RESULTS A total of 103 studies were identified. The models reported in the literature were primarily based on single-center validations (75%), with only 13% being externally validated across multiple centers. Most of the mortality models demonstrated a limited ability to discriminate and classify effectively. The PROBAST assessment indicated a high risk of systematic errors in predicted outcomes and artificial intelligence or machine learning applications. CONCLUSIONS The findings indicate that the development of this field is still in its early stages. This systematic review indicates that application of machine learning in perioperative medicine is still at an early stage. While many studies suggest potential utility, several key challenges must be first overcome before their introduction into clinical practice. EDITOR’S PERSPECTIVE
Collapse
Affiliation(s)
- Pietro Arina
- Bloomsbury Institute of Intensive Care Medicine and Human Physiology and Performance Laboratory, Centre for Perioperative Medicine, Department of Targeted Intervention, University College London, London, United Kingdom
| | - Maciej R. Kaczorek
- Wellcome/EPSRC Centre of Interventional and Surgical Sciences and Department of Medical Physics and Biomedical Engineering, University College London, London, United Kingdom
| | - Daniel A. Hofmaenner
- Bloomsbury Institute of Intensive Care Medicine, University College London, London, United Kingdom; and Institute of Intensive Care Medicine, University Hospital Zurich, Zurich, Switzerland
| | - Walter Pisciotta
- Bloomsbury Institute of Intensive Care Medicine, University College London, London, United Kingdom
| | - Patricia Refinetti
- Human Physiology and Performance Laboratory, Centre for Perioperative Medicine, Department of Targeted Intervention, University College London, London, United Kingdom
| | - Mervyn Singer
- Bloomsbury Institute of Intensive Care Medicine, University College London, London, United Kingdom
| | - Evangelos B. Mazomenos
- Wellcome/EPSRC Centre of Interventional and Surgical Sciences and Department of Medical Physics and Biomedical Engineering, University College London, London, United Kingdom
| | - John Whittle
- Human Physiology and Performance Laboratory, Centre for Perioperative Medicine, Department of Targeted Intervention, University College London, London, United Kingdom
| |
Collapse
|
|
19
|
Kolasa K, Admassu B, Hołownia-Voloskova M, Kędzior KJ, Poirrier JE, Perni S. Systematic reviews of machine learning in healthcare: a literature review. Expert Rev Pharmacoecon Outcomes Res 2024; 24:63-115. [PMID: 37955147 DOI: 10.1080/14737167.2023.2279107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Accepted: 10/31/2023] [Indexed: 11/14/2023]
Abstract
INTRODUCTION The increasing availability of data and computing power has made machine learning (ML) a viable approach to faster, more efficient healthcare delivery. METHODS A systematic literature review (SLR) of published SLRs evaluating ML applications in healthcare settings published between1 January 2010 and 27 March 2023 was conducted. RESULTS In total 220 SLRs covering 10,462 ML algorithms were reviewed. The main application of AI in medicine related to the clinical prediction and disease prognosis in oncology and neurology with the use of imaging data. Accuracy, specificity, and sensitivity were provided in 56%, 28%, and 25% SLRs respectively. Internal and external validation was reported in 53% and less than 1% of the cases respectively. The most common modeling approach was neural networks (2,454 ML algorithms), followed by support vector machine and random forest/decision trees (1,578 and 1,522 ML algorithms, respectively). EXPERT OPINION The review indicated considerable reporting gaps in terms of the ML's performance, both internal and external validation. Greater accessibility to healthcare data for developers can ensure the faster adoption of ML algorithms into clinical practice.
Collapse
Affiliation(s)
- Katarzyna Kolasa
- Division of Health Economics and Healthcare Management, Kozminski University, Warsaw, Poland
| | - Bisrat Admassu
- Division of Health Economics and Healthcare Management, Kozminski University, Warsaw, Poland
| | | | | | | | | |
Collapse
|
|
20
|
Wood MD, West NC, Fokkens C, Chen Y, Loftsgard KC, Cardinal K, Whyte SD, Portales-Casamar E, Görges M. An Individualized Postoperative Pain Risk Communication Tool for Use in Pediatric Surgery: Co-Design and Usability Evaluation. JMIR Pediatr Parent 2023; 6:e46785. [PMID: 37976087 PMCID: PMC10692877 DOI: 10.2196/46785] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 07/07/2023] [Accepted: 10/23/2023] [Indexed: 11/19/2023] Open
Abstract
BACKGROUND Risk identification and communication tools have the potential to improve health care by supporting clinician-patient or family discussion of treatment risks and benefits and helping patients make more informed decisions; however, they have yet to be tailored to pediatric surgery. User-centered design principles can help to ensure the successful development and uptake of health care tools. OBJECTIVE We aimed to develop and evaluate the usability of an easy-to-use tool to communicate a child's risk of postoperative pain to improve informed and collaborative preoperative decision-making between clinicians and families. METHODS With research ethics board approval, we conducted web-based co-design sessions with clinicians and family participants (people with lived surgical experience and parents of children who had recently undergone a surgical or medical procedure) at a tertiary pediatric hospital. Qualitative data from these sessions were analyzed thematically using NVivo (Lumivero) to identify design requirements to inform the iterative redesign of an existing prototype. We then evaluated the usability of our final prototype in one-to-one sessions with a new group of participants, in which we measured mental workload with the National Aeronautics and Space Administration (NASA) Task Load Index (TLX) and user satisfaction with the Post-Study System Usability Questionnaire (PSSUQ). RESULTS A total of 12 participants (8 clinicians and 4 family participants) attended 5 co-design sessions. The 5 requirements were identified: (A) present risk severity descriptively and visually; (B) ensure appearance and navigation are user-friendly; (C) frame risk identification and mitigation strategies in positive terms; (D) categorize and describe risks clearly; and (E) emphasize collaboration and effective communication. A total of 12 new participants (7 clinicians and 5 family participants) completed a usability evaluation. Tasks were completed quickly (range 5-17 s) and accurately (range 11/12, 92% to 12/12, 100%), needing only 2 requests for assistance. The median (IQR) NASA TLX performance score of 78 (66-89) indicated that participants felt able to perform the required tasks, and an overall PSSUQ score of 2.1 (IQR 1.5-2.7) suggested acceptable user satisfaction with the tool. CONCLUSIONS The key design requirements were identified, and that guided the prototype redesign, which was positively evaluated during usability testing. Implementing a personalized risk communication tool into pediatric surgery can enhance the care process and improve informed and collaborative presurgical preparation and decision-making between clinicians and families of pediatric patients.
Collapse
Affiliation(s)
- Michael D Wood
- Department of Anesthesiology Pharmacology & Therapeutics, The University of British Columbia, Vancouver, BC, Canada
- Research Institute, BC Children's Hospital, Vancouver, BC, Canada
| | - Nicholas C West
- Research Institute, BC Children's Hospital, Vancouver, BC, Canada
| | - Christina Fokkens
- Research Institute, BC Children's Hospital, Vancouver, BC, Canada
- School of Information, The University of British Columbia, Vancouver, BC, Canada
| | - Ying Chen
- Research Institute, BC Children's Hospital, Vancouver, BC, Canada
- School of Information, The University of British Columbia, Vancouver, BC, Canada
| | | | - Krystal Cardinal
- Research Institute, BC Children's Hospital, Vancouver, BC, Canada
| | - Simon D Whyte
- Department of Anesthesiology Pharmacology & Therapeutics, The University of British Columbia, Vancouver, BC, Canada
- Research Institute, BC Children's Hospital, Vancouver, BC, Canada
| | - Elodie Portales-Casamar
- Research Institute, BC Children's Hospital, Vancouver, BC, Canada
- Centre de recherche, Centre Hospitalier universitaire Sainte-Justine, Montreal, QC, Canada
| | - Matthias Görges
- Department of Anesthesiology Pharmacology & Therapeutics, The University of British Columbia, Vancouver, BC, Canada
- Research Institute, BC Children's Hospital, Vancouver, BC, Canada
| |
Collapse
|
|
21
|
Colunga-Lozano LE, Foroutan F, Rayner D, De Luca C, Hernández-Wolters B, Couban R, Ibrahim Q, Guyatt G. Clinical judgment shows similar and sometimes superior discrimination compared to prognostic clinical prediction models. A systematic review. J Clin Epidemiol 2023; 165:S0895-4356(23)00276-7. [PMID: 39492557 DOI: 10.1016/j.jclinepi.2023.10.016] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 10/09/2023] [Accepted: 10/21/2023] [Indexed: 11/05/2024]
Abstract
OBJECTIVES To systematically review the comparative statistical performance (discrimination and /or calibration) of prognostic clinical prediction models (CPMs) and clinician judgment (CJ). STUDY DESIGN AND SETTING We conducted a systematic review of observational studies in PubMed, Medline, Embase, and CINAHL. Eligible studies reported direct statistical comparison between prognostic CPMs and CJ. Risk of bias was assessed using the PROBAST tool. RESULTS We identified 41 studies, most with high risk of bias (39 studies). Of these, 41 studies, 39 examined discrimination and 12 studies assessed calibration. Prognostic CPMs had a median AUC of 0.73 (IQR, 0.62 - 0.81), while CJ had a median AUC of 0.71 (IQR, 0.62 - 0.81). 29 studies provided 124 discrimination metrics useful for comparative analysis. Among these, 58 (46.7%) found no significant difference between prognostic CPMs and CJ (p > 0.05); 31 (25%) favored prognostic CPMs, and 35 (28.2%) favored CJ. Four studies compared calibration, showing better performance on prognostic CPMs. CONCLUSIONS In many instances CJ frequently demonstrates comparable or superior discrimination compared to prognostic CPMs, although models outperform CJ on calibration. Studies comparing performance of prognostic CPMs and CJ require large improvements in reporting.
Collapse
Affiliation(s)
- Luis Enrique Colunga-Lozano
- Department of clinical medicine, Health science center, Universidad de Guadalajara, Guadalajara, Jalisco, México; Department of Health Research Methods, Evidence and Impact. McMaster University, Hamilton, Ontario, Canada.
| | - Farid Foroutan
- Department of Health Research Methods, Evidence and Impact. McMaster University, Hamilton, Ontario, Canada
| | - Daniel Rayner
- Department of Health Research Methods, Evidence and Impact. McMaster University, Hamilton, Ontario, Canada
| | - Christopher De Luca
- Faculty of Science, Schulich School of Medicine & Dentistry, University of Western Ontario, London, Canada
| | | | - Rachel Couban
- Department of Health Research Methods, Evidence and Impact. McMaster University, Hamilton, Ontario, Canada
| | - Quazi Ibrahim
- Department of Health Research Methods, Evidence and Impact. McMaster University, Hamilton, Ontario, Canada
| | - Gordon Guyatt
- Department of Health Research Methods, Evidence and Impact. McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
|
22
|
Mumtaz H, Saqib M, Jabeen S, Muneeb M, Mughal W, Sohail H, Safdar M, Mehmood Q, Khan MA, Ismail SM. Exploring alternative approaches to precision medicine through genomics and artificial intelligence - a systematic review. Front Med (Lausanne) 2023; 10:1227168. [PMID: 37849490 PMCID: PMC10577305 DOI: 10.3389/fmed.2023.1227168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 09/20/2023] [Indexed: 10/19/2023] Open
Abstract
The core idea behind precision medicine is to pinpoint the subpopulations that differ from one another in terms of disease risk, drug responsiveness, and treatment outcomes due to differences in biology and other traits. Biomarkers are found through genomic sequencing. Multi-dimensional clinical and biological data are created using these biomarkers. Better analytic methods are needed for these multidimensional data, which can be accomplished by using artificial intelligence (AI). An updated review of 80 latest original publications is presented on four main fronts-preventive medicine, medication development, treatment outcomes, and diagnostic medicine-All these studies effectively illustrated the significance of AI in precision medicine. Artificial intelligence (AI) has revolutionized precision medicine by swiftly analyzing vast amounts of data to provide tailored treatments and predictive diagnostics. Through machine learning algorithms and high-resolution imaging, AI assists in precise diagnoses and early disease detection. AI's ability to decode complex biological factors aids in identifying novel therapeutic targets, allowing personalized interventions and optimizing treatment outcomes. Furthermore, AI accelerates drug discovery by navigating chemical structures and predicting drug-target interactions, expediting the development of life-saving medications. With its unrivaled capacity to comprehend and interpret data, AI stands as an invaluable tool in the pursuit of enhanced patient care and improved health outcomes. It's evident that AI can open a new horizon for precision medicine by translating complex data into actionable information. To get better results in this regard and to fully exploit the great potential of AI, further research is required on this pressing subject.
Collapse
Affiliation(s)
| | | | | | - Muhammad Muneeb
- Department of Medicine, Dow University of Health Sciences, Karachi, Pakistan
| | - Wajiha Mughal
- Department of Medicine, Dow University of Health Sciences, Karachi, Pakistan
| | - Hassan Sohail
- Department of Medicine, Dow University of Health Sciences, Karachi, Pakistan
| | - Myra Safdar
- Armed Forces Institute of Cardiology and National Institute of Heart Diseases (AFIC-NIHD), Rawalpindi, Pakistan
| | - Qasim Mehmood
- Department of Medicine, King Edward Medical University, Lahore, Pakistan
| | - Muhammad Ahsan Khan
- Department of Medicine, Dow University of Health Sciences, Karachi, Pakistan
| | | |
Collapse
|
|
23
|
Henn J, Hatterscheidt S, Sahu A, Buness A, Dohmen J, Arensmeyer J, Feodorovici P, Sommer N, Schmidt J, Kalff JC, Matthaei H. Machine Learning for Decision-Support in Acute Abdominal Pain - Proof of Concept and Central Considerations. Zentralbl Chir 2023; 148:376-383. [PMID: 37562397 DOI: 10.1055/a-2125-1559] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/12/2023]
Abstract
Acute abdominal pain is a common presenting symptom in the emergency department and represents heterogeneous causes and diagnoses. There is often a decision to be made regarding emergency surgical care. Machine learning (ML) could be used here as a decision-support and relieve the time and personnel resource shortage.Patients with acute abdominal pain presenting to the Department of Surgery at Bonn University Hospital in 2020 and 2021 were retrospectively analyzed. Clinical parameters as well as laboratory values were used as predictors. After randomly splitting into a training and test data set (ratio 80 to 20), three ML algorithms were comparatively trained and validated. The entire procedure was repeated 20 times.A total of 1357 patients were identified and included in the analysis, with one in five (n = 276, 20.3%) requiring emergency abdominal surgery within 24 hours. Patients operated on were more likely to be male (p = 0.026), older (p = 0.006), had more gastrointestinal symptoms (nausea: p < 0.001, vomiting p < 0.001) as well as a more recent onset of pain (p < 0.001). Tenderness (p < 0.001) and guarding (p < 0.001) were more common in surgically treated patients and blood analyses showed increased inflammation levels (white blood cell count: p < 0.001, CRP: p < 0.001) and onset of organ dysfunction (creatinine: p < 0.014, quick p < 0.001). Of the three trained algorithms, the tree-based methods (h2o random forest and cforest) showed the best performance. The algorithms classified patients, i.e., predicted surgery, with a median AUC ROC of 0.81 and 0.79 and AUC PRC of 0.56 in test sets.A proof-of-concept was achieved with the development of an ML model for predicting timely surgical therapy for acute abdomen. The ML algorithm can be a valuable tool in decision-making. Especially in the context of heavily used medical resources, the algorithm can help to use these scarce resources more effectively. Technological progress, especially regarding artificial intelligence, increasingly enables evidence-based approaches in surgery but requires a strictly interdisciplinary approach. In the future, the use and handling of ML should be integrated into surgical training.
Collapse
Affiliation(s)
- Jonas Henn
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Bonn, Germany
| | - Simon Hatterscheidt
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Bonn, Germany
| | - Anshupa Sahu
- Institute for Medical Biometry, Informatics and Epidemiology, University Hospital Bonn, Bonn, Germany
- Institute for Genomic Statistics and Bioinformatics, University Hospital Bonn, Bonn, Germany
| | - Andreas Buness
- Institute for Medical Biometry, Informatics and Epidemiology, University Hospital Bonn, Bonn, Germany
- Institute for Genomic Statistics and Bioinformatics, University Hospital Bonn, Bonn, Germany
| | - Jonas Dohmen
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Bonn, Germany
| | - Jan Arensmeyer
- Division of Thoracic Surgery, Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Bonn, Germany
| | - Philipp Feodorovici
- Division of Thoracic Surgery, Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Bonn, Germany
| | - Nils Sommer
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Bonn, Germany
| | - Joachim Schmidt
- Division of Thoracic Surgery, Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Bonn, Germany
- Department of Thoracic Surgery, Helios Hospital Bonn Rhein-Sieg, Bonn, Germany
| | - Jörg C Kalff
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Bonn, Germany
| | - Hanno Matthaei
- Department of General, Visceral, Thoracic and Vascular Surgery, University Hospital Bonn, Bonn, Germany
| |
Collapse
|
|
24
|
Marwaha JS, Beaulieu-Jones BR, Berrigan M, Yuan W, Odom SR, Cook CH, Scott BB, Gupta A, Parsons CS, Seshadri AJ, Brat GA. Quantifying the Prognostic Value of Preoperative Surgeon Intuition: Comparing Surgeon Intuition and Clinical Risk Prediction as Derived from the American College of Surgeons NSQIP Risk Calculator. J Am Coll Surg 2023; 236:1093-1103. [PMID: 36815715 PMCID: PMC10192014 DOI: 10.1097/xcs.0000000000000658] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
Abstract
BACKGROUND Surgical risk prediction models traditionally use patient attributes and measures of physiology to generate predictions about postoperative outcomes. However, the surgeon's assessment of the patient may be a valuable predictor, given the surgeon's ability to detect and incorporate factors that existing models cannot capture. We compare the predictive utility of surgeon intuition and a risk calculator derived from the American College of Surgeons (ACS) NSQIP. STUDY DESIGN From January 10, 2021 to January 9, 2022, surgeons were surveyed immediately before performing surgery to assess their perception of a patient's risk of developing any postoperative complication. Clinical data were abstracted from ACS NSQIP. Both sources of data were independently used to build models to predict the likelihood of a patient experiencing any 30-day postoperative complication as defined by ACS NSQIP. RESULTS Preoperative surgeon assessment was obtained for 216 patients. NSQIP data were available for 9,182 patients who underwent general surgery (January 1, 2017 to January 9, 2022). A binomial regression model trained on clinical data alone had an area under the receiver operating characteristic curve (AUC) of 0.83 (95% CI 0.80 to 0.85) in predicting any complication. A model trained on only preoperative surgeon intuition had an AUC of 0.70 (95% CI 0.63 to 0.78). A model trained on surgeon intuition and a subset of clinical predictors had an AUC of 0.83 (95% CI 0.77 to 0.89). CONCLUSIONS Preoperative surgeon intuition alone is an independent predictor of patient outcomes; however, a risk calculator derived from ACS NSQIP is a more robust predictor of postoperative complication. Combining intuition and clinical data did not strengthen prediction.
Collapse
Affiliation(s)
- Jayson S Marwaha
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
- the Department of Biomedical Informatics, Harvard Medical School, Boston, MA (Marwaha, Beaulieu-Jones, Yuan, Brat)
| | - Brendin R Beaulieu-Jones
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
- the Department of Biomedical Informatics, Harvard Medical School, Boston, MA (Marwaha, Beaulieu-Jones, Yuan, Brat)
| | - Margaret Berrigan
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
| | - William Yuan
- the Department of Biomedical Informatics, Harvard Medical School, Boston, MA (Marwaha, Beaulieu-Jones, Yuan, Brat)
| | - Stephen R Odom
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
| | - Charles H Cook
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
| | - Benjamin B Scott
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
| | - Alok Gupta
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
| | - Charles S Parsons
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
| | - Anupamaa J Seshadri
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
| | - Gabriel A Brat
- From the Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA (Marwaha, Beaulieu-Jones, Berrigan, Odom, Cook, Scott, Gupta, Parsons, Seshadri, Brat)
- the Department of Biomedical Informatics, Harvard Medical School, Boston, MA (Marwaha, Beaulieu-Jones, Yuan, Brat)
| |
Collapse
|
|
25
|
Balch JA, Ruppert MM, Shickel B, Ozrazgat-Baslanti T, Tighe PJ, Efron PA, Upchurch GR, Rashidi P, Bihorac A, Loftus TJ. Building an automated, machine learning-enabled platform for predicting post-operative complications. Physiol Meas 2023; 44:024001. [PMID: 36657179 PMCID: PMC9910093 DOI: 10.1088/1361-6579/acb4db] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2022] [Revised: 12/29/2022] [Accepted: 01/19/2023] [Indexed: 01/21/2023]
Abstract
Objective. In 2019, the University of Florida College of Medicine launched theMySurgeryRiskalgorithm to predict eight major post-operative complications using automatically extracted data from the electronic health record.Approach. This project was developed in parallel with our Intelligent Critical Care Center and represents a culmination of efforts to build an efficient and accurate model for data processing and predictive analytics.Main Results and Significance. This paper discusses how our model was constructed and improved upon. We highlight the consolidation of the database, processing of fixed and time-series physiologic measurements, development and training of predictive models, and expansion of those models into different aspects of patient assessment and treatment. We end by discussing future directions of the model.
Collapse
Affiliation(s)
- Jeremy A Balch
- Intelligent Critical Care Center, University of Florida, Gainesville, FL, United States of America
- Department of Surgery, University of Florida, Gainesville, Florida, United States of America
| | - Matthew M Ruppert
- Intelligent Critical Care Center, University of Florida, Gainesville, FL, United States of America
- Department of Medicine, University of Florida, Gainesville, Florida, United States of America
| | - Benjamin Shickel
- Intelligent Critical Care Center, University of Florida, Gainesville, FL, United States of America
- Department of Medicine, University of Florida, Gainesville, Florida, United States of America
| | - Tezcan Ozrazgat-Baslanti
- Intelligent Critical Care Center, University of Florida, Gainesville, FL, United States of America
- Department of Medicine, University of Florida, Gainesville, Florida, United States of America
| | - Patrick J Tighe
- Department of Anesthesiology, University of Florida, Gainesville, Florida, United States of America
| | - Philip A Efron
- Department of Surgery, University of Florida, Gainesville, Florida, United States of America
| | - Gilbert R Upchurch
- Department of Surgery, University of Florida, Gainesville, Florida, United States of America
| | - Parisa Rashidi
- Intelligent Critical Care Center, University of Florida, Gainesville, FL, United States of America
- Department of Biomedical Engineering, University of Florida, Gainesville, Florida, United States of America
| | - Azra Bihorac
- Intelligent Critical Care Center, University of Florida, Gainesville, FL, United States of America
- Department of Medicine, University of Florida, Gainesville, Florida, United States of America
| | - Tyler J Loftus
- Intelligent Critical Care Center, University of Florida, Gainesville, FL, United States of America
- Department of Surgery, University of Florida, Gainesville, Florida, United States of America
| |
Collapse
|
|
26
|
Hassan AM, Rajesh A, Asaad M, Jonas NA, Coert JH, Mehrara BJ, Butler CE. Artificial Intelligence and Machine Learning in Prediction of Surgical Complications: Current State, Applications, and Implications. Am Surg 2023; 89:25-30. [PMID: 35562124 PMCID: PMC9653510 DOI: 10.1177/00031348221101488] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Surgical complications pose significant challenges for surgeons, patients, and health care systems as they may result in patient distress, suboptimal outcomes, and higher health care costs. Artificial intelligence (AI)-driven models have revolutionized the field of surgery by accurately identifying patients at high risk of developing surgical complications and by overcoming several limitations associated with traditional statistics-based risk calculators. This article aims to provide an overview of AI in predicting surgical complications using common machine learning and deep learning algorithms and illustrates how this can be utilized to risk stratify patients preoperatively. This can form the basis for discussions on informed consent based on individualized patient factors in the future.
Collapse
Affiliation(s)
- Abbas M. Hassan
- Department of Plastic and Reconstructive Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Aashish Rajesh
- Department of Surgery, University of Texas Health Science Center, San Antonio, TX, USA
| | - Malke Asaad
- Department of Plastic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Nelson A. Jonas
- Department of Plastic and Reconstructive Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - J. Henk. Coert
- Department of Plastic and Reconstructive Surgery, University Medical Center Utrecht, Utrecht, Netherlands
| | - Babak J. Mehrara
- Department of Plastic and Reconstructive Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Charles E. Butler
- Department of Plastic and Reconstructive Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| |
Collapse
|
|
27
|
Vagliano I, Chesnaye NC, Leopold JH, Jager KJ, Abu-Hanna A, Schut MC. Machine learning models for predicting acute kidney injury: a systematic review and critical appraisal. Clin Kidney J 2022; 15:2266-2280. [PMID: 36381375 PMCID: PMC9664575 DOI: 10.1093/ckj/sfac181] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Indexed: 09/08/2023] Open
Abstract
BACKGROUND The number of studies applying machine learning (ML) to predict acute kidney injury (AKI) has grown steadily over the past decade. We assess and critically appraise the state of the art in ML models for AKI prediction, considering performance, methodological soundness, and applicability. METHODS We searched PubMed and ArXiv, extracted data, and critically appraised studies based on the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD), Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), and Prediction Model Risk of Bias Assessment Tool (PROBAST) guidelines. RESULTS Forty-six studies from 3166 titles were included. Thirty-eight studies developed a model, five developed and externally validated one, and three studies externally validated one. Flexible ML methods were used more often than deep learning, although the latter was common with temporal variables and text as predictors. Predictive performance showed an area under receiver operating curves ranging from 0.49 to 0.99. Our critical appraisal identified a high risk of bias in 39 studies. Some studies lacked internal validation, whereas external validation and interpretability of results were rarely considered. Fifteen studies focused on AKI prediction in the intensive care setting, and the US-derived Medical Information Mart for Intensive Care (MIMIC) data set was commonly used. Reproducibility was limited as data and code were usually unavailable. CONCLUSIONS Flexible ML methods are popular for the prediction of AKI, although more complex models based on deep learning are emerging. Our critical appraisal identified a high risk of bias in most models: Studies should use calibration measures and external validation more often, improve model interpretability, and share data and code to improve reproducibility.
Collapse
Affiliation(s)
- Iacopo Vagliano
- Deptartment of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Nicholas C Chesnaye
- ERA Registry, Department of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Jan Hendrik Leopold
- Deptartment of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Kitty J Jager
- ERA Registry, Department of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Ameen Abu-Hanna
- Deptartment of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| | - Martijn C Schut
- Deptartment of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands
| |
Collapse
|
|
28
|
Pecqueux M, Riediger C, Distler M, Oehme F, Bork U, Kolbinger FR, Schöffski O, van Wijngaarden P, Weitz J, Schweipert J, Kahlert C. The use and future perspective of Artificial Intelligence-A survey among German surgeons. Front Public Health 2022; 10:982335. [PMID: 36276381 PMCID: PMC9580562 DOI: 10.3389/fpubh.2022.982335] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2022] [Accepted: 09/05/2022] [Indexed: 01/25/2023] Open
Abstract
Purpose Clinical abundance of artificial intelligence has increased significantly in the last decade. This survey aims to provide an overview of the current state of knowledge and acceptance of AI applications among surgeons in Germany. Methods A total of 357 surgeons from German university hospitals, academic teaching hospitals and private practices were contacted by e-mail and asked to participate in the anonymous survey. Results A total of 147 physicians completed the survey. The majority of respondents (n = 85, 52.8%) stated that they were familiar with AI applications in medicine. Personal knowledge was self-rated as average (n = 67, 41.6%) or rudimentary (n = 60, 37.3%) by the majority of participants. On the basis of various application scenarios, it became apparent that the respondents have different demands on AI applications in the area of "diagnosis confirmation" as compared to the area of "therapy decision." For the latter category, the requirements in terms of the error level are significantly higher and more respondents view their application in medical practice rather critically. Accordingly, most of the participants hope that AI systems will primarily improve diagnosis confirmation, while they see their ethical and legal problems with regard to liability as the main obstacle to extensive clinical application. Conclusion German surgeons are in principle positively disposed toward AI applications. However, many surgeons see a deficit in their own knowledge and in the implementation of AI applications in their own professional environment. Accordingly, medical education programs targeting both medical students and healthcare professionals should convey basic knowledge about the development and clinical implementation process of AI applications in different medical fields, including surgery.
Collapse
Affiliation(s)
- Mathieu Pecqueux
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
| | - Carina Riediger
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
| | - Marius Distler
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
| | - Florian Oehme
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
| | - Ulrich Bork
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
| | - Fiona R. Kolbinger
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
- Else Kröner Fresenius Center for Digital Health (EKFZ) Dresden, Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
| | - Oliver Schöffski
- Chair of Health Management, Friedrich-Alexander-Universität Erlangen-Nürnberg, Nürnberg, Germany
| | - Peter van Wijngaarden
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, Melbourne, VIC, Australia
- Ophthalmology, Department of Surgery, University of Melbourne, Melbourne, VIC, Australia
| | - Jürgen Weitz
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
- German Cancer Consortium (DKTK), Partner Site Dresden, German Cancer Research Center (DKFZ), National Center for Tumor Diseases Dresden (NCT/UCC), Heidelberg, Germany
| | - Johannes Schweipert
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
| | - Christoph Kahlert
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Technische Universität Dresden, National Center for Tumor Diseases Dresden (NCT/UCC), Dresden, Germany
- German Cancer Consortium (DKTK), Partner Site Dresden, German Cancer Research Center (DKFZ), National Center for Tumor Diseases Dresden (NCT/UCC), Heidelberg, Germany
| |
Collapse
|
|
29
|
Filiberto AC, Efron PA, Frantz A, Bihorac A, Upchurch GR, Loftus TJ. Personalized decision-making for acute cholecystitis: Understanding surgeon judgment. Front Digit Health 2022; 4:845453. [PMID: 36339515 PMCID: PMC9632988 DOI: 10.3389/fdgth.2022.845453] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2021] [Accepted: 08/30/2022] [Indexed: 12/07/2022] Open
Abstract
Background There is sparse high-level evidence to guide treatment decisions for severe, acute cholecystitis (inflammation of the gallbladder). Therefore, treatment decisions depend heavily on individual surgeon judgment, which is highly variable and potentially amenable to personalized, data-driven decision support. We test the hypothesis that surgeons' treatment recommendations misalign with perceived risks and benefits for laparoscopic cholecystectomy (surgical removal) vs. percutaneous cholecystostomy (image-guided drainage). Methods Surgery attendings, fellows, and residents applied individual judgement to standardized case scenarios in a live, web-based survey in estimating the quantitative risks and benefits of laparoscopic cholecystectomy vs. percutaneous cholecystostomy for both moderate and severe acute cholecystitis, as well as the likelihood that they would recommend cholecystectomy. Results Surgeons predicted similar 30-day morbidity rates for laparoscopic cholecystectomy and percutaneous cholecystostomy. However, a greater proportion of surgeons predicted low (<50%) likelihood of full recovery following percutaneous cholecystostomy compared with cholecystectomy for both moderate (30% vs. 2%, p < 0.001) and severe (62% vs. 38%, p < 0.001) cholecystitis. Ninety-eight percent of all surgeons were likely or very likely to recommend cholecystectomy for moderate cholecystitis; only 32% recommended cholecystectomy for severe cholecystitis (p < 0.001). There were no significant differences in predicted postoperative morbidity when respondents were stratified by academic rank or self-reported ability to predict complications or make treatment recommendations. Conclusions Surgeon recommendations for severe cholecystitis were discordant with perceived risks and benefits of treatment options. Surgeons predicted greater functional recovery after cholecystectomy but less than one-third recommended cholecystectomy. These findings suggest opportunities to augment surgical decision-making with personalized, data-driven decision support.
Collapse
Affiliation(s)
- Amanda C. Filiberto
- Department of Surgery, University of Florida Health, Gainesville, FL, United States
| | - Philip A. Efron
- Department of Surgery, University of Florida Health, Gainesville, FL, United States
| | - Amanda Frantz
- Department of Anesthesiology, University of Florida Health, Gainesville, FL, United States
| | - Azra Bihorac
- Department of Medicine, University of Florida Health, Gainesville, FL, United States
- Intelligent Critical Care Center, University of Florida Health, Gainesville, FL, United States
| | - Gilbert R. Upchurch
- Department of Surgery, University of Florida Health, Gainesville, FL, United States
| | - Tyler J. Loftus
- Department of Surgery, University of Florida Health, Gainesville, FL, United States
- Intelligent Critical Care Center, University of Florida Health, Gainesville, FL, United States
- Correspondence: Tyler J. Loftus
| |
Collapse
|
|
30
|
Wood MD, West NC, Sreepada R, Loftsgard KC, Petersen L, Robillard J, Page P, Ridgway R, Chadha NK, Portales-Casamar E, Görges M. Identifying risk factors, patient reported experience and outcome measures, and data capture tools for an individualized pain prediction tool in pediatrics: a focus group study (Preprint). JMIR Perioper Med 2022; 5:e42341. [DOI: 10.2196/42341] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 10/25/2022] [Accepted: 10/27/2022] [Indexed: 11/05/2022] Open
|
|
31
|
Ren Y, Loftus TJ, Datta S, Ruppert MM, Guan Z, Miao S, Shickel B, Feng Z, Giordano C, Upchurch GR, Rashidi P, Ozrazgat-Baslanti T, Bihorac A. Performance of a Machine Learning Algorithm Using Electronic Health Record Data to Predict Postoperative Complications and Report on a Mobile Platform. JAMA Netw Open 2022; 5:e2211973. [PMID: 35576007 PMCID: PMC9112066 DOI: 10.1001/jamanetworkopen.2022.11973] [Citation(s) in RCA: 47] [Impact Index Per Article: 15.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/22/2023] Open
Abstract
IMPORTANCE Predicting postoperative complications has the potential to inform shared decisions regarding the appropriateness of surgical procedures, targeted risk-reduction strategies, and postoperative resource use. Realizing these advantages requires that accurate real-time predictions be integrated with clinical and digital workflows; artificial intelligence predictive analytic platforms using automated electronic health record (EHR) data inputs offer an intriguing possibility for achieving this, but there is a lack of high-level evidence from prospective studies supporting their use. OBJECTIVE To examine whether the MySurgeryRisk artificial intelligence system has stable predictive performance between development and prospective validation phases and whether it is feasible to provide automated outputs directly to surgeons' mobile devices. DESIGN, SETTING, AND PARTICIPANTS In this prognostic study, the platform used automated EHR data inputs and machine learning algorithms to predict postoperative complications and provide predictions to surgeons, previously through a web portal and currently through a mobile device application. All patients 18 years or older who were admitted for any type of inpatient surgical procedure (74 417 total procedures involving 58 236 patients) between June 1, 2014, and September 20, 2020, were included. Models were developed using retrospective data from 52 117 inpatient surgical procedures performed between June 1, 2014, and November 27, 2018. Validation was performed using data from 22 300 inpatient surgical procedures collected prospectively from November 28, 2018, to September 20, 2020. MAIN OUTCOMES AND MEASURES Algorithms for generalized additive models and random forest models were developed and validated using real-time EHR data. Model predictive performance was evaluated primarily using area under the receiver operating characteristic curve (AUROC) values. RESULTS Among 58 236 total adult patients who received 74 417 major inpatient surgical procedures, the mean (SD) age was 57 (17) years; 29 226 patients (50.2%) were male. Results reported in this article focus primarily on the validation cohort. The validation cohort included 22 300 inpatient surgical procedures involving 19 132 patients (mean [SD] age, 58 [17] years; 9672 [50.6%] male). A total of 2765 patients (14.5%) were Black or African American, 14 777 (77.2%) were White, 1235 (6.5%) were of other races (including American Indian or Alaska Native, Asian, Native Hawaiian or Pacific Islander, and multiracial), and 355 (1.9%) were of unknown race because of missing data; 979 patients (5.1%) were Hispanic, 17 663 (92.3%) were non-Hispanic, and 490 (2.6%) were of unknown ethnicity because of missing data. A greater number of input features was associated with stable or improved model performance. For example, the random forest model trained with 135 input features had the highest AUROC values for predicting acute kidney injury (0.82; 95% CI, 0.82-0.83); cardiovascular complications (0.81; 95% CI, 0.81-0.82); neurological complications, including delirium (0.87; 95% CI, 0.87-0.88); prolonged intensive care unit stay (0.89; 95% CI, 0.88-0.89); prolonged mechanical ventilation (0.91; 95% CI, 0.90-0.91); sepsis (0.86; 95% CI, 0.85-0.87); venous thromboembolism (0.82; 95% CI, 0.81-0.83); wound complications (0.78; 95% CI, 0.78-0.79); 30-day mortality (0.84; 95% CI, 0.82-0.86); and 90-day mortality (0.84; 95% CI, 0.82-0.85), with accuracy similar to surgeons' predictions. Compared with the original web portal, the mobile device application allowed efficient fingerprint login access and loaded data approximately 10 times faster. The application output displayed patient information, risk of postoperative complications, top 3 risk factors for each complication, and patterns of complications for individual surgeons compared with their colleagues. CONCLUSIONS AND RELEVANCE In this study, automated real-time predictions of postoperative complications with mobile device outputs had good performance in clinical settings with prospective validation, matching surgeons' predictive accuracy.
Collapse
Affiliation(s)
- Yuanfang Ren
- Intelligent Critical Care Center, University of Florida, Gainesville
- Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, Gainesville
| | - Tyler J. Loftus
- Intelligent Critical Care Center, University of Florida, Gainesville
- Department of Surgery, University of Florida, Gainesville
| | - Shounak Datta
- Intelligent Critical Care Center, University of Florida, Gainesville
- Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, Gainesville
| | - Matthew M. Ruppert
- Intelligent Critical Care Center, University of Florida, Gainesville
- Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, Gainesville
| | - Ziyuan Guan
- Intelligent Critical Care Center, University of Florida, Gainesville
- Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, Gainesville
| | - Shunshun Miao
- Intelligent Critical Care Center, University of Florida, Gainesville
- Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, Gainesville
| | - Benjamin Shickel
- Intelligent Critical Care Center, University of Florida, Gainesville
- Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, Gainesville
| | - Zheng Feng
- Intelligent Critical Care Center, University of Florida, Gainesville
- Department of Electrical and Computer Engineering, University of Florida, Gainesville
| | - Chris Giordano
- Intelligent Critical Care Center, University of Florida, Gainesville
- Department of Anesthesiology, University of Florida, Gainesville
| | - Gilbert R. Upchurch
- Intelligent Critical Care Center, University of Florida, Gainesville
- Department of Surgery, University of Florida, Gainesville
| | - Parisa Rashidi
- Intelligent Critical Care Center, University of Florida, Gainesville
- Department of Biomedical Engineering, University of Florida, Gainesville
| | - Tezcan Ozrazgat-Baslanti
- Intelligent Critical Care Center, University of Florida, Gainesville
- Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, Gainesville
| | - Azra Bihorac
- Intelligent Critical Care Center, University of Florida, Gainesville
- Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, Gainesville
| |
Collapse
|
|
32
|
Le ST, Liu VX, Kipnis P, Zhang J, Peng PD, Cespedes Feliciano EM. Comparison of Electronic Frailty Metrics for Prediction of Adverse Outcomes of Abdominal Surgery. JAMA Surg 2022; 157:e220172. [PMID: 35293969 PMCID: PMC8928095 DOI: 10.1001/jamasurg.2022.0172] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Importance Electronic frailty metrics have been developed for automated frailty assessment and include the Hospital Frailty Risk Score (HFRS), the Electronic Frailty Index (eFI), the 5-Factor Modified Frailty Index (mFI-5), and the Risk Analysis Index (RAI). Despite substantial differences in their construction, these 4 electronic frailty metrics have not been rigorously compared within a surgical population. Objective To characterize the associations between 4 electronic frailty metrics and to measure their predictive value for adverse surgical outcomes. Design, Setting, and Participants This retrospective cohort study used electronic health record data from patients who underwent abdominal surgery from January 1, 2010, to December 31, 2020, at 20 medical centers within Kaiser Permanente Northern California (KPNC). Participants included adults older than 50 years who underwent abdominal surgical procedures at KPNC from 2010 to 2020 that were sampled for reporting to the National Surgical Quality Improvement Program. Main Outcomes and Measures Pearson correlation coefficients between electronic frailty metrics and area under the receiver operating characteristic curve (AUROC) of univariate models and multivariate preoperative risk models for 30-day mortality, readmission, and morbidity, which was defined as a composite of mortality and major postoperative complications. Results Within the cohort of 37 186 patients, mean (SD) age, 67.9 (female, 19 127 [51.4%]), correlations between pairs of metrics ranged from 0.19 (95% CI, 0.18- 0.20) for mFI-5 and RAI 0.69 (95% CI, 0.68-0.70). Only 1085 of 37 186 (2.9%) were classified as frail based on all 4 metrics. In univariate models for morbidity, HFRS demonstrated higher predictive discrimination (AUROC, 0.71; 95% CI, 0.70-0.72) than eFI (AUROC, 0.64; 95% CI, 0.63-0.65), mFI-5 (AUROC, 0.58; 95% CI, 0.57-0.59), and RAI (AUROC, 0.57; 95% CI, 0.57-0.58). The predictive discrimination of multivariate models with age, sex, comorbidity burden, and procedure characteristics for all 3 adverse surgical outcomes improved by including HFRS into the models. Conclusions and Relevance In this cohort study, the 4 electronic frailty metrics demonstrated heterogeneous correlation and classified distinct groups of surgical patients as frail. However, HFRS demonstrated the highest predictive value for adverse surgical outcomes.
Collapse
Affiliation(s)
- Sidney T. Le
- Division of Research, Kaiser Permanente Northern California, Oakland
- Department of Surgery, University of California San Francisco-East Bay, Oakland
| | - Vincent X. Liu
- Division of Research, Kaiser Permanente Northern California, Oakland
- The Permanente Medical Group, Oakland, California
| | - Patricia Kipnis
- Division of Research, Kaiser Permanente Northern California, Oakland
| | - Jie Zhang
- Division of Research, Kaiser Permanente Northern California, Oakland
| | | | | |
Collapse
|
|
33
|
Gisladottir U, Nakikj D, Jhunjhunwala R, Panton J, Brat G, Gehlenborg N. Effective Communication of Personalized Risks and Patient Preferences During Surgical Informed Consent Using Data Visualization: Qualitative Semistructured Interview Study With Patients After Surgery. JMIR Hum Factors 2022; 9:e29118. [PMID: 35486432 PMCID: PMC9107059 DOI: 10.2196/29118] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 08/24/2021] [Accepted: 10/02/2021] [Indexed: 01/13/2023] Open
Abstract
Background There is no consensus on which risks to communicate to a prospective surgical patient during informed consent or how. Complicating the process, patient preferences may diverge from clinical assumptions and are often not considered for discussion. Such discrepancies can lead to confusion and resentment, raising the potential for legal action. To overcome these issues, we propose a visual consent tool that incorporates patient preferences and communicates personalized risks to patients using data visualization. We used this platform to identify key effective visual elements to communicate personalized surgical risks. Objective Our main focus is to understand how to best communicate personalized risks using data visualization. To contextualize patient responses to the main question, we examine how patients perceive risks before surgery (research question 1), how suitably the visual consent tool is able to present personalized surgical risks (research question 2), how well our visualizations convey those personalized surgical risks (research question 3), and how the visual consent tool could improve the informed consent process and how it can be used (research question 4). Methods We designed a visual consent tool to meet the objectives of our study. To calculate and list personalized surgical risks, we used the American College of Surgeons risk calculator. We created multiple visualization mock-ups using visual elements previously determined to be well-received for risk communication. Semistructured interviews were conducted with patients after surgery, and each of the mock-ups was presented and evaluated independently and in the context of our visual consent tool design. The interviews were transcribed, and thematic analysis was performed to identify major themes. We also applied a quantitative approach to the analysis to assess the prevalence of different perceptions of the visualizations presented in our tool. Results In total, 20 patients were interviewed, with a median age of 59 (range 29-87) years. Thematic analysis revealed factors that influenced the perception of risk (the surgical procedure, the cognitive capacity of the patient, and the timing of consent; research question 1); factors that influenced the perceived value of risk visualizations (preference for rare event communication, preference for risk visualization, and usefulness of comparison with the average; research question 3); and perceived usefulness and use cases of the visual consent tool (research questions 2 and 4). Most importantly, we found that patients preferred the visual consent tool to current text-based documents and had no unified preferences for risk visualization. Furthermore, our findings suggest that patient concerns were not often represented in existing risk calculators. Conclusions We identified key elements that influence effective visual risk communication in the perioperative setting and pointed out the limitations of the existing calculators in addressing patient concerns. Patient preference is highly variable and should influence choices regarding risk presentation and visualization.
Collapse
Affiliation(s)
- Undina Gisladottir
- Department of Biomedical Informatics, Harvard Medical School, Harvard University, Boston, MA, United States
| | - Drashko Nakikj
- Department of Biomedical Informatics, Harvard Medical School, Harvard University, Boston, MA, United States
| | - Rashi Jhunjhunwala
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA, United States
| | - Jasmine Panton
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA, United States.,Geisel School of Medicine, Dartmouth College, Hanover, NH, United States
| | - Gabriel Brat
- Department of Biomedical Informatics, Harvard Medical School, Harvard University, Boston, MA, United States.,Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA, United States
| | - Nils Gehlenborg
- Department of Biomedical Informatics, Harvard Medical School, Harvard University, Boston, MA, United States
| |
Collapse
|
|
34
|
Tulk Jesso S, Kelliher A, Sanghavi H, Martin T, Henrickson Parker S. Inclusion of Clinicians in the Development and Evaluation of Clinical Artificial Intelligence Tools: A Systematic Literature Review. Front Psychol 2022; 13:830345. [PMID: 35465567 PMCID: PMC9022040 DOI: 10.3389/fpsyg.2022.830345] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 02/09/2022] [Indexed: 12/11/2022] Open
Abstract
The application of machine learning (ML) and artificial intelligence (AI) in healthcare domains has received much attention in recent years, yet significant questions remain about how these new tools integrate into frontline user workflow, and how their design will impact implementation. Lack of acceptance among clinicians is a major barrier to the translation of healthcare innovations into clinical practice. In this systematic review, we examine when and how clinicians are consulted about their needs and desires for clinical AI tools. Forty-five articles met criteria for inclusion, of which 24 were considered design studies. The design studies used a variety of methods to solicit and gather user feedback, with interviews, surveys, and user evaluations. Our findings show that tool designers consult clinicians at various but inconsistent points during the design process, and most typically at later stages in the design cycle (82%, 19/24 design studies). We also observed a smaller amount of studies adopting a human-centered approach and where clinician input was solicited throughout the design process (22%, 5/24). A third (15/45) of all studies reported on clinician trust in clinical AI algorithms and tools. The surveyed articles did not universally report validation against the “gold standard” of clinical expertise or provide detailed descriptions of the algorithms or computational methods used in their work. To realize the full potential of AI tools within healthcare settings, our review suggests there are opportunities to more thoroughly integrate frontline users’ needs and feedback in the design process.
Collapse
Affiliation(s)
- Stephanie Tulk Jesso
- Fralin Biomedical Research Institute, Virginia Tech, Roanoke, VA, United States.,Institute for Creativity, Arts, and Technology, Blacksburg, VA, United States
| | - Aisling Kelliher
- Department of Computer Science, College of Engineering, Virginia Tech, Blacksburg, VA, United States
| | | | - Thomas Martin
- Institute for Creativity, Arts, and Technology, Blacksburg, VA, United States.,Department of Electrical and Computer Engineering, College of Engineering, Virginia Tech, Blacksburg, VA, United States
| | - Sarah Henrickson Parker
- Fralin Biomedical Research Institute, Virginia Tech, Roanoke, VA, United States.,Department of Health Systems and Implementation Science, Virginia Tech Carilion School of Medicine, Roanoke, VA, United States
| |
Collapse
|
|
35
|
Taha A, Ochs V, Kayhan LN, Enodien B, Frey DM, Krähenbühl L, Taha-Mehlitz S. Advancements of Artificial Intelligence in Liver-Associated Diseases and Surgery. MEDICINA (KAUNAS, LITHUANIA) 2022; 58:medicina58040459. [PMID: 35454298 PMCID: PMC9029673 DOI: 10.3390/medicina58040459] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 03/14/2022] [Accepted: 03/18/2022] [Indexed: 02/06/2023]
Abstract
Background and Objectives: The advancement of artificial intelligence (AI) based technologies in medicine is progressing rapidly, but the majority of its real-world applications has not been implemented. The establishment of an accurate diagnosis with treatment has now transitioned into an artificial intelligence era, which has continued to provide an amplified understanding of liver cancer as a disease and helped to proceed better with the method of procurement. This article focuses on reviewing the AI in liver-associated diseases and surgical procedures, highlighting its development, use, and related counterparts. Materials and Methods: We searched for articles regarding AI in liver-related ailments and surgery, using the keywords (mentioned below) on PubMed, Google Scholar, Scopus, MEDLINE, and Cochrane Library. Choosing only the common studies suggested by these libraries, we segregated the matter based on disease. Finally, we compiled the essence of these articles under the various sub-headings. Results: After thorough review of articles, it was observed that there was a surge in the occurrence of liver-related surgeries, diagnoses, and treatments. Parallelly, advanced computer technologies governed by AI continue to prove their efficacy in the accurate screening, analysis, prediction, treatment, and recuperation of liver-related cases. Conclusions: The continual developments and high-order precision of AI is expanding its roots in all directions of applications. Despite being novel and lacking research, AI has shown its intrinsic worth for procedures in liver surgery while providing enhanced healing opportunities and personalized treatment for liver surgery patients.
Collapse
Affiliation(s)
- Anas Taha
- Department of Biomedical Engineering, Faculty of Medicine, University of Basel, 4123 Allschwil, Switzerland
- Correspondence:
| | - Vincent Ochs
- Roche Innovation Center Basel, Department of Pharma Research & Early Development, 4070 Basel, Switzerland;
| | - Leos N. Kayhan
- Department of Surgery, Canntonal Hospital Luzern, 6004 Luzern, Switzerland;
| | - Bassey Enodien
- Department of Surgery, Wetzikon Hospital, 8620 Wetzikon, Switzerland; (B.E.); (D.M.F.)
| | - Daniel M. Frey
- Department of Surgery, Wetzikon Hospital, 8620 Wetzikon, Switzerland; (B.E.); (D.M.F.)
| | | | - Stephanie Taha-Mehlitz
- Clarunis, University Centre for Gastrointestinal and Liver Diseases, St. Clara Hospital and University Hospital Basel, 4002 Basel, Switzerland;
| |
Collapse
|
|
36
|
Henn J, Buness A, Schmid M, Kalff JC, Matthaei H. Machine learning to guide clinical decision-making in abdominal surgery-a systematic literature review. Langenbecks Arch Surg 2022; 407:51-61. [PMID: 34716472 PMCID: PMC8847247 DOI: 10.1007/s00423-021-02348-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2021] [Accepted: 10/03/2021] [Indexed: 12/16/2022]
Abstract
PURPOSE An indication for surgical therapy includes balancing benefits against risk, which remains a key task in all surgical disciplines. Decisions are oftentimes based on clinical experience while guidelines lack evidence-based background. Various medical fields capitalized the application of machine learning (ML), and preliminary research suggests promising implications in surgeons' workflow. Hence, we evaluated ML's contemporary and possible future role in clinical decision-making (CDM) focusing on abdominal surgery. METHODS Using the PICO framework, relevant keywords and research questions were identified. Following the PRISMA guidelines, a systemic search strategy in the PubMed database was conducted. Results were filtered by distinct criteria and selected articles were manually full text reviewed. RESULTS Literature review revealed 4,396 articles, of which 47 matched the search criteria. The mean number of patients included was 55,843. A total of eight distinct ML techniques were evaluated whereas AUROC was applied by most authors for comparing ML predictions vs. conventional CDM routines. Most authors (N = 30/47, 63.8%) stated ML's superiority in the prediction of benefits and risks of surgery. The identification of highly relevant parameters to be integrated into algorithms allowing a more precise prognosis was emphasized as the main advantage of ML in CDM. CONCLUSIONS A potential value of ML for surgical decision-making was demonstrated in several scientific articles. However, the low number of publications with only few collaborative studies between surgeons and computer scientists underpins the early phase of this highly promising field. Interdisciplinary research initiatives combining existing clinical datasets and emerging techniques of data processing may likely improve CDM in abdominal surgery in the future.
Collapse
Affiliation(s)
- Jonas Henn
- Department of General, Visceral, Thoracic and Vascular Surgery, University of Bonn, Bonn, Germany
| | - Andreas Buness
- Institute for Medical Biometry, Informatics and Epidemiology, University of Bonn, Bonn, Germany
- Institute for Genomic Statistics and Bioinformatics, University of Bonn, Bonn, Germany
| | - Matthias Schmid
- Institute for Medical Biometry, Informatics and Epidemiology, University of Bonn, Bonn, Germany
| | - Jörg C Kalff
- Department of General, Visceral, Thoracic and Vascular Surgery, University of Bonn, Bonn, Germany
| | - Hanno Matthaei
- Department of General, Visceral, Thoracic and Vascular Surgery, University of Bonn, Bonn, Germany.
| |
Collapse
|
|
37
|
Loftus TJ, Balch JA, Ruppert MM, Tighe PJ, Hogan WR, Rashidi P, Upchurch GR, Bihorac A. Aligning Patient Acuity With Resource Intensity After Major Surgery: A Scoping Review. Ann Surg 2022; 275:332-339. [PMID: 34261886 PMCID: PMC8750209 DOI: 10.1097/sla.0000000000005079] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
OBJECTIVE Develop unifying definitions and paradigms for data-driven methods to augment postoperative resource intensity decisions. SUMMARY BACKGROUND DATA Postoperative level-of-care assignments and frequency of vital sign and laboratory measurements (ie, resource intensity) should align with patient acuity. Effective, data-driven decision-support platforms could improve value of care for millions of patients annually, but their development is hindered by the lack of salient definitions and paradigms. METHODS Embase, PubMed, and Web of Science were searched for articles describing patient acuity and resource intensity after inpatient surgery. Study quality was assessed using validated tools. Thirty-five studies were included and assimilated according to PRISMA guidelines. RESULTS Perioperative patient acuity is accurately represented by combinations of demographic, physiologic, and hospital-system variables as input features in models that capture complex, non-linear relationships. Intraoperative physiologic data enriche these representations. Triaging high-acuity patients to low-intensity care is associated with increased risk for mortality; triaging low-acuity patients to intensive care units (ICUs) has low value and imparts harm when other, valid requests for ICU admission are denied due to resource limitations, increasing their risk for unrecognized decompensation and failure-to-rescue. Providing high-intensity care for low-acuity patients may also confer harm through unnecessary testing and subsequent treatment of incidental findings, but there is insufficient evidence to evaluate this hypothesis. Compared with data-driven models, clinicians exhibit volatile performance in predicting complications and making postoperative resource intensity decisions. CONCLUSION To optimize value, postoperative resource intensity decisions should align with precise, data-driven patient acuity assessments augmented by models that accurately represent complex, non-linear relationships among risk factors.
Collapse
Affiliation(s)
- Tyler J. Loftus
- Department of Surgery, University of Florida Health,
Gainesville, FL, USA
- Precision and Intelligent Systems in Medicine
(Prisma), University of Florida, Gainesville, FL, USA
| | - Jeremy A. Balch
- Department of Surgery, University of Florida Health,
Gainesville, FL, USA
| | - Matthew M. Ruppert
- Department of Medicine, University of Florida Health,
Gainesville, FL, USA
- Precision and Intelligent Systems in Medicine
(Prisma), University of Florida, Gainesville, FL, USA
| | - Patrick J. Tighe
- Departments of Anesthesiology, Orthopedics, and Information
Systems/Operations Management, University of Florida Health, Gainesville, FL,
USA
| | - William R. Hogan
- Department of Health Outcomes & Biomedical Informatics,
College of Medicine, University of Florida, Gainesville, FL, USA
| | - Parisa Rashidi
- Departments of Biomedical Engineering, Computer and
Information Science and Engineering, and Electrical and Computer Engineering,
University of Florida, Gainesville, Florida, USA
- Precision and Intelligent Systems in Medicine
(Prisma), University of Florida, Gainesville, FL, USA
| | | | - Azra Bihorac
- Department of Medicine, University of Florida Health,
Gainesville, FL, USA
- Precision and Intelligent Systems in Medicine
(Prisma), University of Florida, Gainesville, FL, USA
| |
Collapse
|
|
38
|
Loftus TJ, Tighe PJ, Ozrazgat-Baslanti T, Davis JP, Ruppert MM, Ren Y, Shickel B, Kamaleswaran R, Hogan WR, Moorman JR, Upchurch GR, Rashidi P, Bihorac A. Ideal algorithms in healthcare: Explainable, dynamic, precise, autonomous, fair, and reproducible. PLOS DIGITAL HEALTH 2022; 1:e0000006. [PMID: 36532301 PMCID: PMC9754299 DOI: 10.1371/journal.pdig.0000006] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
Established guidelines describe minimum requirements for reporting algorithms in healthcare; it is equally important to objectify the characteristics of ideal algorithms that confer maximum potential benefits to patients, clinicians, and investigators. We propose a framework for ideal algorithms, including 6 desiderata: explainable (convey the relative importance of features in determining outputs), dynamic (capture temporal changes in physiologic signals and clinical events), precise (use high-resolution, multimodal data and aptly complex architecture), autonomous (learn with minimal supervision and execute without human input), fair (evaluate and mitigate implicit bias and social inequity), and reproducible (validated externally and prospectively and shared with academic communities). We present an ideal algorithms checklist and apply it to highly cited algorithms. Strategies and tools such as the predictive, descriptive, relevant (PDR) framework, the Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence (SPIRIT-AI) extension, sparse regression methods, and minimizing concept drift can help healthcare algorithms achieve these objectives, toward ideal algorithms in healthcare.
Collapse
Affiliation(s)
- Tyler J. Loftus
- Department of Surgery, University of Florida Health, Gainesville, Florida, United States of America
- Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville, Florida, United States of America
| | - Patrick J. Tighe
- Departments of Anesthesiology, Orthopedics, and Information Systems/Operations Management, University of Florida Health, Gainesville, Florida, United States of America
| | - Tezcan Ozrazgat-Baslanti
- Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville, Florida, United States of America
- Department of Medicine, University of Florida Health, Gainesville, Florida, United States of America
| | - John P. Davis
- Department of Surgery, University of Virginia, Charlottesville, Virginia, United States of America
| | - Matthew M. Ruppert
- Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville, Florida, United States of America
- Department of Medicine, University of Florida Health, Gainesville, Florida, United States of America
| | - Yuanfang Ren
- Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville, Florida, United States of America
- Department of Medicine, University of Florida Health, Gainesville, Florida, United States of America
| | - Benjamin Shickel
- Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville, Florida, United States of America
- Department of Medicine, University of Florida Health, Gainesville, Florida, United States of America
| | - Rishikesan Kamaleswaran
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, Georgia, United States of America
| | - William R. Hogan
- Department of Health Outcomes & Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, United States of America
| | - J. Randall Moorman
- Department of Medicine, University of Virginia, Charlottesville, Virginia, United States of America
| | - Gilbert R. Upchurch
- Department of Surgery, University of Florida Health, Gainesville, Florida, United States of America
| | - Parisa Rashidi
- Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville, Florida, United States of America
- Departments of Biomedical Engineering, Computer and Information Science and Engineering, and Electrical and Computer Engineering, University of Florida, Gainesville, Florida, United States of America
| | - Azra Bihorac
- Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville, Florida, United States of America
- Department of Medicine, University of Florida Health, Gainesville, Florida, United States of America
| |
Collapse
|
|
39
|
Bellini V, Valente M, Bertorelli G, Pifferi B, Craca M, Mordonini M, Lombardo G, Bottani E, Del Rio P, Bignami E. Machine learning in perioperative medicine: a systematic review. JOURNAL OF ANESTHESIA, ANALGESIA AND CRITICAL CARE 2022; 2:2. [PMCID: PMC8761048 DOI: 10.1186/s44158-022-00033-y] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/18/2023]
Abstract
Background Risk stratification plays a central role in anesthetic evaluation. The use of Big Data and machine learning (ML) offers considerable advantages for collection and evaluation of large amounts of complex health-care data. We conducted a systematic review to understand the role of ML in the development of predictive post-surgical outcome models and risk stratification. Methods Following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines, we selected the period of the research for studies from 1 January 2015 up to 30 March 2021. A systematic search in Scopus, CINAHL, the Cochrane Library, PubMed, and MeSH databases was performed; the strings of research included different combinations of keywords: “risk prediction,” “surgery,” “machine learning,” “intensive care unit (ICU),” and “anesthesia” “perioperative.” We identified 36 eligible studies. This study evaluates the quality of reporting of prediction models using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist. Results The most considered outcomes were mortality risk, systemic complications (pulmonary, cardiovascular, acute kidney injury (AKI), etc.), ICU admission, anesthesiologic risk and prolonged length of hospital stay. Not all the study completely followed the TRIPOD checklist, but the quality was overall acceptable with 75% of studies (Rev #2, comm #minor issue) showing an adherence rate to TRIPOD more than 60%. The most frequently used algorithms were gradient boosting (n = 13), random forest (n = 10), logistic regression (LR; n = 7), artificial neural networks (ANNs; n = 6), and support vector machines (SVM; n = 6). Models with best performance were random forest and gradient boosting, with AUC > 0.90. Conclusions The application of ML in medicine appears to have a great potential. From our analysis, depending on the input features considered and on the specific prediction task, ML algorithms seem effective in outcomes prediction more accurately than validated prognostic scores and traditional statistics. Thus, our review encourages the healthcare domain and artificial intelligence (AI) developers to adopt an interdisciplinary and systemic approach to evaluate the overall impact of AI on perioperative risk assessment and on further health care settings as well.
Collapse
Affiliation(s)
- Valentina Bellini
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| | - Marina Valente
- General Surgery Unit, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| | - Giorgia Bertorelli
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| | - Barbara Pifferi
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| | - Michelangelo Craca
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| | - Monica Mordonini
- Department of Engineering and Architecture, University of Parma, Viale G.P.Usberti 181/A, 43124 Parma, Italy
| | - Gianfranco Lombardo
- Department of Engineering and Architecture, University of Parma, Viale G.P.Usberti 181/A, 43124 Parma, Italy
| | - Eleonora Bottani
- Department of Engineering and Architecture, University of Parma, Viale G.P.Usberti 181/A, 43124 Parma, Italy
| | - Paolo Del Rio
- General Surgery Unit, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| | - Elena Bignami
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Viale Gramsci 14, 43126 Parma, Italy
| |
Collapse
|
|
40
|
Dyas AR, Colborn KL, Bronsert MR, Henderson WG, Mason NJ, Rozeboom PD, Pradhan N, Lambert-Kerzner A, Meguid RA. Comparison of Preoperative Surgical Risk Estimated by Thoracic Surgeons Versus a Standardized Surgical Risk Prediction Tool. Semin Thorac Cardiovasc Surg 2021; 34:1378-1385. [PMID: 34785355 DOI: 10.1053/j.semtcvs.2021.11.008] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Accepted: 11/10/2021] [Indexed: 11/11/2022]
Abstract
Considerable variability exists between surgeons' assessments of a patient's individual pre-operative surgical risk. Surgical risk calculators are not routinely used despite their validation. We sought to compare thoracic surgeons' prediction of patients' risk of postoperative adverse outcomes versus a surgical risk calculator, the Surgical Risk Preoperative Assessment System (SURPAS). We developed vignettes from 30 randomly selected patients who underwent thoracic surgery in the American College of Surgeons' National Surgical Quality Improvement Program (NSQIP) database. Twelve thoracic surgeons estimated patients' preoperative risks of postoperative morbidity and mortality. These were compared to SURPAS estimates of the same vignettes. C-indices and Brier scores were calculated for the surgeons' and SURPAS estimates. Agreement between surgeon estimates was examined using intraclass correlation coefficients (ICCs). Surgeons estimated higher morbidity risk compared to SURPAS for low-risk patients (ASA classes 1-2, 11.5% vs. 5.1%, p=<0.001) and lower morbidity risk compared to SURPAS for high-risk patients (ASA class 5, 37.6% vs. 69.8%, p<0.001). This trend also occurred in high-risk patients for mortality (ASA 5, 11.1% vs. 44.3%, p<0.001). C-indices for SURPAS vs. surgeons were 0.84 vs. 0.76 (p=0.3) for morbidity and 0.98 vs. 0.85 (p=0.001) for mortality. Brier scores for SURPAS vs. surgeons were 0.1579 vs. 0.1986 for morbidity (p=0.03) and 0.0409 vs. 0.0543 for mortality (p=0.006). ICCs showed that surgeons had moderate risk agreement for morbidity (ICC=0.654) and mortality (ICC=0.507). Thoracic surgeons and patients could benefit from using a surgical risk calculator to better estimate patients' surgical risks during the informed consent process.
Collapse
Affiliation(s)
- Adam R Dyas
- Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA
| | - Kathryn L Colborn
- Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA; Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO, USA
| | - Michael R Bronsert
- Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA; Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, CO, USA
| | - William G Henderson
- Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA; Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, CO, USA; Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO, USA
| | - Nicholas J Mason
- Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA
| | - Paul D Rozeboom
- Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA
| | - Nisha Pradhan
- Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA
| | - Anne Lambert-Kerzner
- Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA; Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, CO, USA
| | - Robert A Meguid
- Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA; Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, CO, USA.
| |
Collapse
|
|
41
|
Ko A, Wren SM. Advances in Appropriate Postoperative Triage and the Role of Real-time Machine-Learning Models: The Goldilocks Dilemma. JAMA Netw Open 2021; 4:e2133843. [PMID: 34757414 DOI: 10.1001/jamanetworkopen.2021.33843] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Affiliation(s)
- Ara Ko
- Division of General Surgery, Department of Surgery, Stanford University, Stanford, California
| | - Sherry M Wren
- Division of General Surgery, Department of Surgery, Stanford University, Stanford, California
| |
Collapse
|
|
42
|
Loftus TJ, Ruppert MM, Ozrazgat-Baslanti T, Balch JA, Efron PA, Tighe PJ, Hogan WR, Rashidi P, Upchurch GR, Bihorac A. Association of Postoperative Undertriage to Hospital Wards With Mortality and Morbidity. JAMA Netw Open 2021; 4:e2131669. [PMID: 34757412 PMCID: PMC8581722 DOI: 10.1001/jamanetworkopen.2021.31669] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/13/2022] Open
Abstract
IMPORTANCE Undertriaging patients who are at increased risk for postoperative complications after surgical procedures to low-acuity hospital wards (ie, floors) rather than highly vigilant intensive care units (ICUs) may be associated with risk of unrecognized decompensation and worse patient outcomes, but evidence for these associations is lacking. OBJECTIVE To test the hypothesis that postoperative undertriage is associated with increased mortality and morbidity compared with risk-matched ICU admission. DESIGN, SETTING, AND PARTICIPANTS This longitudinal cross-sectional study was conducted using data from the University of Florida Integrated Data Repository on admissions to a university hospital. Included patients were individuals aged 18 years or older who were admitted after a surgical procedure from June 1, 2014, to August 20, 2020. Data were analyzed from April through August 2021. EXPOSURES Ward admissions were considered undertriaged if their estimated risk for hospital mortality or prolonged ICU stay (ie, ≥48 hours) was in the top quartile among all inpatient surgical procedures according to a validated machine-learning model using preoperative and intraoperative electronic health record features available at surgical procedure end time. A nearest neighbors algorithm was used to identify a risk-matched control group of ICU admissions. MAIN OUTCOMES AND MEASURES The primary outcomes of hospital mortality and morbidity were compared among appropriately triaged ward admissions, undertriaged wards admissions, and a risk-matched control group of ICU admissions. RESULTS Among 12 348 postoperative ward admissions, 11 042 admissions (89.4%) were appropriately triaged (5927 [53.7%] women; median [IQR] age, 59 [44-70] years) and 1306 admissions (10.6%) were undertriaged and matched with a control group of 2452 ICU admissions. The undertriaged group, compared with the control group, had increased median [IQR] age (64 [54-74] years vs 62 [50-73] years; P = .001) and increased proportions of women (649 [49.7%] women vs 1080 [44.0%] women; P < .001) and admitted patients with do not resuscitate orders before first surgical procedure (53 admissions [4.1%] vs 27 admissions [1.1%]); P < .001); 207 admissions that were undertriaged (15.8%) had subsequent ICU admission. In the validation cohort, hospital mortality and prolonged ICU stay estimations had areas under the receiver operating characteristic curve of 0.92 (95% CI, 0.91-0.93) and 0.92 (95% CI, 0.92-0.92), respectively. The undertriaged group, compared with the control group, had similar incidence of prolonged mechanical ventilation (32 admissions [2.5%] vs 53 admissions [2.2%]; P = .60), decreased median (IQR) total costs for admission ($26 900 [$18 400-$42 300] vs $32 700 [$22 700-$48 500]; P < .001), increased median (IQR) hospital length of stay (8.1 [5.1-13.6] days vs 6.0 [3.3-9.3] days, P < .001), and increased incidence of hospital mortality (19 admissions [1.5%] vs 17 admissions [0.7%]; P = .04), discharge to hospice (23 admissions [1.8%] vs 14 admissions [0.6%]; P < .001), unplanned intubation (45 admissions [3.4%] vs 49 admissions [2.0%]; P = .01), and acute kidney injury (341 admissions [26.1%] vs 477 admissions [19.5%]; P < .001). CONCLUSIONS AND RELEVANCE This study found that admitted patients at increased risk for postoperative complications who were undertriaged to hospital wards had increased mortality and morbidity compared with a risk-matched control group of admissions to ICUs. Postoperative undertriage was identifiable using automated preoperative and intraoperative data as features in real-time machine-learning models.
Collapse
Affiliation(s)
- Tyler J. Loftus
- Department of Surgery, University of Florida Health, Gainesville
- Precision and Intelligent Systems in Medicine Research Partnership, University of Florida, Gainesville
| | - Matthew M. Ruppert
- Precision and Intelligent Systems in Medicine Research Partnership, University of Florida, Gainesville
- Department of Medicine, University of Florida Health, Gainesville
| | - Tezcan Ozrazgat-Baslanti
- Precision and Intelligent Systems in Medicine Research Partnership, University of Florida, Gainesville
- Department of Medicine, University of Florida Health, Gainesville
| | - Jeremy A. Balch
- Department of Surgery, University of Florida Health, Gainesville
| | - Philip A. Efron
- Department of Surgery, University of Florida Health, Gainesville
| | - Patrick J. Tighe
- Department of Anesthesiology, University of Florida Health, Gainesville
- Department of Orthopaedic Surgery and Sports Medicine, University of Florida Health, Gainesville
- Department of Information Systems and Operations Management, University of Florida Health, Gainesville
| | - William R. Hogan
- Department of Health Outcomes and Biomedical Informatics, University of Florida College of Medicine, Gainesville
| | - Parisa Rashidi
- Precision and Intelligent Systems in Medicine Research Partnership, University of Florida, Gainesville
- Department of Biomedical Engineering, University of Florida, Gainesville
- Department of Computer and Information Science and Engineering, University of Florida, Gainesville
- Department of Electrical and Computer Engineering, University of Florida, Gainesville
| | | | - Azra Bihorac
- Precision and Intelligent Systems in Medicine Research Partnership, University of Florida, Gainesville
- Department of Medicine, University of Florida Health, Gainesville
| |
Collapse
|
|
43
|
Sakamoto T, Goto T, Fujiogi M, Kawarai Lefor A. Machine learning in gastrointestinal surgery. Surg Today 2021; 52:995-1007. [PMID: 34559310 DOI: 10.1007/s00595-021-02380-9] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2021] [Accepted: 07/03/2021] [Indexed: 12/11/2022]
Abstract
Machine learning (ML) is a collection of algorithms allowing computers to learn directly from data without predetermined equations. It is used widely to analyze "big data". In gastrointestinal surgery, surgeons deal with various data such as clinical parameters, surgical videos, and pathological images, to stratify surgical risk, perform safe surgery and predict patient prognosis. In the current "big data" era, the accelerating accumulation of a large amount of data drives studies using ML algorithms. Three subfields of ML are supervised learning, unsupervised learning, and reinforcement learning. In this review, we summarize applications of ML to surgical practice in the preoperative, intraoperative, and postoperative phases of care. Prediction and stratification using ML is promising; however, the current overarching concern is the availability of ML models. Information systems that can manage "big data" and integrate ML models into electronic health records are essential to incorporate ML into daily practice. ML is fundamental technology to meaningfully process data that exceeds the capacity of the human mind to comprehend. The accelerating accumulation of a large amount of data is changing the nature of surgical practice fundamentally. Artificial intelligence (AI), represented by ML, is being incorporated into daily surgical practice.
Collapse
Affiliation(s)
- Takashi Sakamoto
- Department of Gastroenterological Surgery, Gastroenterological Center, Cancer Institute Hospital, Japanese Foundation for Cancer Research, 3-8-31 Ariake, Koto, Tokyo, 135-8550, Japan. .,Department of Clinical Epidemiology and Health Economics, School of Public Health, The University of Tokyo, Bunkyo-ku, Tokyo, 113-0033, Japan.
| | - Tadahiro Goto
- Department of Clinical Epidemiology and Health Economics, School of Public Health, The University of Tokyo, Bunkyo-ku, Tokyo, 113-0033, Japan.,TXP Medical Co. Ltd, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 114-8485, Japan
| | - Michimasa Fujiogi
- Department of Emergency Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, 02114, USA.,Department of Pediatric Surgery, Graduate School of Medicine, The University of Tokyo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Alan Kawarai Lefor
- Department of Surgery, Jichi Medical University, Shimotsuke, Tochigi, 3290498, Japan
| |
Collapse
|
|
44
|
Elhage SA, Deerenberg EB, Ayuso SA, Murphy KJ, Shao JM, Kercher KW, Smart NJ, Fischer JP, Augenstein VA, Colavita PD, Heniford BT. Development and Validation of Image-Based Deep Learning Models to Predict Surgical Complexity and Complications in Abdominal Wall Reconstruction. JAMA Surg 2021; 156:933-940. [PMID: 34232255 DOI: 10.1001/jamasurg.2021.3012] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
Importance Image-based deep learning models (DLMs) have been used in other disciplines, but this method has yet to be used to predict surgical outcomes. Objective To apply image-based deep learning to predict complexity, defined as need for component separation, and pulmonary and wound complications after abdominal wall reconstruction (AWR). Design, Setting, and Participants This quality improvement study was performed at an 874-bed hospital and tertiary hernia referral center from September 2019 to January 2020. A prospective database was queried for patients with ventral hernias who underwent open AWR by experienced surgeons and had preoperative computed tomography images containing the entire hernia defect. An 8-layer convolutional neural network was generated to analyze image characteristics. Images were batched into training (approximately 80%) or test sets (approximately 20%) to analyze model output. Test sets were blinded from the convolutional neural network until training was completed. For the surgical complexity model, a separate validation set of computed tomography images was evaluated by a blinded panel of 6 expert AWR surgeons and the surgical complexity DLM. Analysis started February 2020. Exposures Image-based DLM. Main Outcomes and Measures The primary outcome was model performance as measured by area under the curve in the receiver operating curve (ROC) calculated for each model; accuracy with accompanying sensitivity and specificity were also calculated. Measures were DLM prediction of surgical complexity using need for component separation techniques as a surrogate and prediction of postoperative surgical site infection and pulmonary failure. The DLM for predicting surgical complexity was compared against the prediction of 6 expert AWR surgeons. Results A total of 369 patients and 9303 computed tomography images were used. The mean (SD) age of patients was 57.9 (12.6) years, 232 (62.9%) were female, and 323 (87.5%) were White. The surgical complexity DLM performed well (ROC = 0.744; P < .001) and, when compared with surgeon prediction on the validation set, performed better with an accuracy of 81.3% compared with 65.0% (P < .001). Surgical site infection was predicted successfully with an ROC of 0.898 (P < .001). However, the DLM for predicting pulmonary failure was less effective with an ROC of 0.545 (P = .03). Conclusions and Relevance Image-based DLM using routine, preoperative computed tomography images was successful in predicting surgical complexity and more accurate than expert surgeon judgment. An additional DLM accurately predicted the development of surgical site infection.
Collapse
Affiliation(s)
- Sharbel Adib Elhage
- Department of Surgery, Franciscus Gasthuis en Vlietland, Rotterdam, the Netherlands
| | | | - Sullivan Armando Ayuso
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| | | | - Jenny Meng Shao
- Department of Surgery, University of Pennsylvania, Philadelphia
| | - Kent Williams Kercher
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| | - Neil James Smart
- Department of Colorectal Surgery, Royal Devon and Exeter NHS Foundation Trust, Royal Devon and Exeter Hospital, Exeter, United Kingdom
| | - John Patrick Fischer
- Division of Plastic Surgery, Department of Surgery, Perelman School of Medicine, Philadelphia, Pennsylvania
| | - Vedra Abdomerovic Augenstein
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| | - Paul Dominick Colavita
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| | - B Todd Heniford
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| |
Collapse
|
|
45
|
Filiberto AC, Ozrazgat-Baslanti T, Loftus TJ, Peng YC, Datta S, Efron P, Upchurch GR, Bihorac A, Cooper MA. Optimizing predictive strategies for acute kidney injury after major vascular surgery. Surgery 2021; 170:298-303. [PMID: 33648766 PMCID: PMC8276529 DOI: 10.1016/j.surg.2021.01.030] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2020] [Revised: 01/18/2021] [Accepted: 01/23/2021] [Indexed: 12/29/2022]
Abstract
BACKGROUND Postoperative acute kidney injury is common after major vascular surgery and is associated with increased morbidity, mortality, and cost. High-performance risk stratification using a machine learning model can inform strategies that mitigate harm and optimize resource use. It is hypothesized that incorporating intraoperative data would improve machine learning model accuracy, discrimination, and precision in predicting acute kidney injury among patients undergoing major vascular surgery. METHODS A single-center retrospective cohort of 1,531 adult patients who underwent nonemergency major vascular surgery, including open aortic, endovascular aortic, and lower extremity bypass procedures, was evaluated. The validated, automated MySurgeryRisk analytics platform used electronic health record data to forecast patient-level probabilistic risk scores for postoperative acute kidney injury using random forest models with preoperative data alone and perioperative data (preoperative plus intraoperative). The MySurgeryRisk predictions were compared with each other as well as with the American Society of Anesthesiologists physical status classification. RESULTS Machine learning models using perioperative data had greater accuracy, discrimination, and precision than models using either preoperative data alone or the American Society of Anesthesiologists physical status classification (accuracy: 0.70 vs 0.64 vs 0.62, area under the receiver operating characteristics curve: 0.77 vs 0.68 vs 0.61, area under the precision-recall curve: 0.70 vs 0.58 vs 0.48). CONCLUSION In predicting acute kidney injury after major vascular surgery, machine learning approaches that incorporate dynamic intraoperative data had greater accuracy, discrimination, and precision than models using either preoperative data alone or the American Society of Anesthesiologists physical status classification. Machine learning methods have the potential for real-time identification of high-risk patients who may benefit from personalized risk-reduction strategies.
Collapse
Affiliation(s)
| | - Tezcan Ozrazgat-Baslanti
- Department of Medicine, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville FL
| | - Tyler J Loftus
- Department of Surgery, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville FL
| | - Ying-Chih Peng
- Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville FL
| | - Shounak Datta
- Department of Medicine, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville FL
| | - Philip Efron
- Department of Surgery, University of Florida, Gainesville, FL; Department of Anesthesia, University of Florida, Gainesville, FL
| | | | - Azra Bihorac
- Department of Medicine, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PrismaP), University of Florida, Gainesville FL
| | - Michol A Cooper
- Department of Surgery, University of Florida, Gainesville, FL.
| |
Collapse
|
|
46
|
Solanki SL, Pandrowala S, Nayak A, Bhandare M, Ambulkar RP, Shrikhande SV. Artificial intelligence in perioperative management of major gastrointestinal surgeries. World J Gastroenterol 2021; 27:2758-2770. [PMID: 34135552 PMCID: PMC8173379 DOI: 10.3748/wjg.v27.i21.2758] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/16/2021] [Revised: 04/06/2021] [Accepted: 04/28/2021] [Indexed: 02/06/2023] Open
Abstract
Artificial intelligence (AI) demonstrated by machines is based on reinforcement learning and revolves around the usage of algorithms. The purpose of this review was to summarize concepts, the scope, applications, and limitations in major gastrointestinal surgery. This is a narrative review of the available literature on the key capabilities of AI to help anesthesiologists, surgeons, and other physicians to understand and critically evaluate ongoing and new AI applications in perioperative management. AI uses available databases called “big data” to formulate an algorithm. Analysis of other data based on these algorithms can help in early diagnosis, accurate risk assessment, intraoperative management, automated drug delivery, predicting anesthesia and surgical complications and postoperative outcomes and can thus lead to effective perioperative management as well as to reduce the cost of treatment. Perioperative physicians, anesthesiologists, and surgeons are well-positioned to help integrate AI into modern surgical practice. We all need to partner and collaborate with data scientists to collect and analyze data across all phases of perioperative care to provide clinical scenarios and context. Careful implementation and use of AI along with real-time human interpretation will revolutionize perioperative care, and is the way forward in future perioperative management of major surgery.
Collapse
Affiliation(s)
- Sohan Lal Solanki
- Department of Anesthesiology, Critical Care and Pain, Tata Memorial Hospital, Homi Bhabha National Institute, Mumbai 400012, Maharashtra, India
| | - Saneya Pandrowala
- Gastro-Intestinal Services, Department of Surgical Oncology, Tata Memorial Hospital, Homi Bhabha National Institute, Mumbai 400012, Maharashtra, India
| | - Abhirup Nayak
- Department of Anesthesiology, Critical Care and Pain, Tata Memorial Hospital, Homi Bhabha National Institute, Mumbai 400012, Maharashtra, India
| | - Manish Bhandare
- Gastro-Intestinal Services, Department of Surgical Oncology, Tata Memorial Hospital, Homi Bhabha National Institute, Mumbai 400012, Maharashtra, India
| | - Reshma P Ambulkar
- Department of Anesthesiology, Critical Care and Pain, Tata Memorial Hospital, Homi Bhabha National Institute, Mumbai 400012, Maharashtra, India
| | - Shailesh V Shrikhande
- Gastro-Intestinal Services, Department of Surgical Oncology, Tata Memorial Hospital, Homi Bhabha National Institute, Mumbai 400012, Maharashtra, India
| |
Collapse
|
|
47
|
Yin J, Ngiam KY, Teo HH. Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review. J Med Internet Res 2021; 23:e25759. [PMID: 33885365 PMCID: PMC8103304 DOI: 10.2196/25759] [Citation(s) in RCA: 142] [Impact Index Per Article: 35.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2020] [Revised: 03/08/2021] [Accepted: 03/09/2021] [Indexed: 12/19/2022] Open
Abstract
BACKGROUND Artificial intelligence (AI) applications are growing at an unprecedented pace in health care, including disease diagnosis, triage or screening, risk analysis, surgical operations, and so forth. Despite a great deal of research in the development and validation of health care AI, only few applications have been actually implemented at the frontlines of clinical practice. OBJECTIVE The objective of this study was to systematically review AI applications that have been implemented in real-life clinical practice. METHODS We conducted a literature search in PubMed, Embase, Cochrane Central, and CINAHL to identify relevant articles published between January 2010 and May 2020. We also hand searched premier computer science journals and conferences as well as registered clinical trials. Studies were included if they reported AI applications that had been implemented in real-world clinical settings. RESULTS We identified 51 relevant studies that reported the implementation and evaluation of AI applications in clinical practice, of which 13 adopted a randomized controlled trial design and eight adopted an experimental design. The AI applications targeted various clinical tasks, such as screening or triage (n=16), disease diagnosis (n=16), risk analysis (n=14), and treatment (n=7). The most commonly addressed diseases and conditions were sepsis (n=6), breast cancer (n=5), diabetic retinopathy (n=4), and polyp and adenoma (n=4). Regarding the evaluation outcomes, we found that 26 studies examined the performance of AI applications in clinical settings, 33 studies examined the effect of AI applications on clinician outcomes, 14 studies examined the effect on patient outcomes, and one study examined the economic impact associated with AI implementation. CONCLUSIONS This review indicates that research on the clinical implementation of AI applications is still at an early stage despite the great potential. More research needs to assess the benefits and challenges associated with clinical AI applications through a more rigorous methodology.
Collapse
Affiliation(s)
- Jiamin Yin
- Department of Information Systems and Analytics, School of Computing, National University of Singapore, Singapore, Singapore
| | - Kee Yuan Ngiam
- Department of Surgery, National University Hospital, Singapore, Singapore
| | - Hock Hai Teo
- Department of Information Systems and Analytics, School of Computing, National University of Singapore, Singapore, Singapore
| |
Collapse
|
|
48
|
Felmingham CM, Adler NR, Ge Z, Morton RL, Janda M, Mar VJ. The Importance of Incorporating Human Factors in the Design and Implementation of Artificial Intelligence for Skin Cancer Diagnosis in the Real World. Am J Clin Dermatol 2021; 22:233-242. [PMID: 33354741 DOI: 10.1007/s40257-020-00574-4] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Artificial intelligence (AI) algorithms have been shown to diagnose skin lesions with impressive accuracy in experimental settings. The majority of the literature to date has compared AI and dermatologists as opponents in skin cancer diagnosis. However, in the real-world clinical setting, the clinician will work in collaboration with AI. Existing evidence regarding the integration of such AI diagnostic tools into clinical practice is limited. Human factors, such as cognitive style, personality, experience, preferences, and attitudes may influence clinicians' use of AI. In this review, we consider these human factors and the potential cognitive errors, biases, and unintended consequences that could arise when using an AI skin cancer diagnostic tool in the real world. Integrating this knowledge in the design and implementation of AI technology will assist in ensuring that the end product can be used effectively. Dermatologist leadership in the development of these tools will further improve their clinical relevance and safety.
Collapse
Affiliation(s)
- Claire M Felmingham
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia.
- Victorian Melanoma Service, Alfred Hospital, 55 Commercial Road, Melbourne, VIC, 3004, Australia.
| | - Nikki R Adler
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
| | - Zongyuan Ge
- Monash eResearch Centre, Monash University, Clayton, Australia
- Department of Electrical and Computer Systems Engineering, Faculty of Engineering, Monash University, Melbourne, VIC, Australia
- Monash-Airdoc Research Centre, Monash University, Melbourne, VIC, Australia
| | - Rachael L Morton
- NHMRC Clinical Trials Centre, Faculty of Medicine and Health, University of Sydney, Camperdown, NSW, Australia
| | - Monika Janda
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Brisbane, QLD, Australia
| | - Victoria J Mar
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
- Victorian Melanoma Service, Alfred Hospital, 55 Commercial Road, Melbourne, VIC, 3004, Australia
| |
Collapse
|
|
49
|
Jauk S, Kramer D, Avian A, Berghold A, Leodolter W, Schulz S. Technology Acceptance of a Machine Learning Algorithm Predicting Delirium in a Clinical Setting: a Mixed-Methods Study. J Med Syst 2021; 45:48. [PMID: 33646459 PMCID: PMC7921052 DOI: 10.1007/s10916-021-01727-6] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Accepted: 02/18/2021] [Indexed: 12/02/2022]
Abstract
Early identification of patients with life-threatening risks such as delirium is crucial in order to initiate preventive actions as quickly as possible. Despite intense research on machine learning for the prediction of clinical outcomes, the acceptance of the integration of such complex models in clinical routine remains unclear. The aim of this study was to evaluate user acceptance of an already implemented machine learning-based application predicting the risk of delirium for in-patients. We applied a mixed methods design to collect opinions and concerns from health care professionals including physicians and nurses who regularly used the application. The evaluation was framed by the Technology Acceptance Model assessing perceived ease of use, perceived usefulness, actual system use and output quality of the application. Questionnaire results from 47 nurses and physicians as well as qualitative results of four expert group meetings rated the overall usefulness of the delirium prediction positively. For healthcare professionals, the visualization and presented information was understandable, the application was easy to use and the additional information for delirium management was appreciated. The application did not increase their workload, but the actual system use was still low during the pilot study. Our study provides insights into the user acceptance of a machine learning-based application supporting delirium management in hospitals. In order to improve quality and safety in healthcare, computerized decision support should predict actionable events and be highly accepted by users.
Collapse
Affiliation(s)
- Stefanie Jauk
- Steiermärkische Krankenanstaltengesellschaft m.b.H. (KAGes), Information and Process Management, Graz, Austria. .,Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Auenbruggerplatz 2, 8036, Graz, Austria.
| | - Diether Kramer
- Steiermärkische Krankenanstaltengesellschaft m.b.H. (KAGes), Information and Process Management, Graz, Austria
| | - Alexander Avian
- Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Auenbruggerplatz 2, 8036, Graz, Austria
| | - Andrea Berghold
- Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Auenbruggerplatz 2, 8036, Graz, Austria
| | - Werner Leodolter
- Steiermärkische Krankenanstaltengesellschaft m.b.H. (KAGes), Information and Process Management, Graz, Austria
| | - Stefan Schulz
- Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Auenbruggerplatz 2, 8036, Graz, Austria
| |
Collapse
|
|
50
|
Balch J, Upchurch GR, Bihorac A, Loftus TJ. Bridging the artificial intelligence valley of death in surgical decision-making. Surgery 2021; 169:746-748. [PMID: 33608148 DOI: 10.1016/j.surg.2021.01.008] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2021] [Revised: 01/08/2021] [Accepted: 01/11/2021] [Indexed: 11/18/2022]
Affiliation(s)
- Jeremy Balch
- Department of Surgery, University of Florida Health, Gainesville, FL. https://twitter.com/balchja
| | - Gilbert R Upchurch
- Department of Surgery, University of Florida Health, Gainesville, FL. https://twitter.com/gru6n
| | - Azra Bihorac
- Department of Medicine, University of Florida Health, Gainesville, FL. https://twitter.com/AzraBihorac
| | - Tyler J Loftus
- Department of Surgery, University of Florida Health, Gainesville, FL.
| |
Collapse
|