1
|
Zheng Z, Luo J, Zhu Y, Du L, Lan L, Zhou X, Yang X, Huang S. Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective Study. J Med Internet Res 2025; 27:e69293. [PMID: 40266658 PMCID: PMC12059492 DOI: 10.2196/69293] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2024] [Revised: 03/27/2025] [Accepted: 03/28/2025] [Indexed: 04/24/2025] Open
Abstract
BACKGROUND Timely and accurate prediction of short-term mortality is critical in intensive care units (ICUs), where patients' conditions change rapidly. Traditional scoring systems, such as the Simplified Acute Physiology Score and Acute Physiology and Chronic Health Evaluation, rely on static variables collected within the first 24 hours of admission and do not account for continuously evolving clinical states. These systems lack real-time adaptability, interpretability, and generalizability. With the increasing availability of high-frequency electronic medical record (EMR) data, machine learning (ML) approaches have emerged as powerful tools to model complex temporal patterns and support dynamic clinical decision-making. However, existing models are often limited by their inability to handle irregular sampling and missing values, and many lack rigorous external validation across institutions. OBJECTIVE We aimed to develop a real-time, interpretable risk prediction model that continuously assesses ICU patient mortality using irregular, longitudinal EMR data, with improved performance and generalizability over traditional static scoring systems. METHODS A time-aware bidirectional attention-based long short-term memory (TBAL) model was developed using EMR data from the MIMIC-IV (Medical Information Mart for Intensive Care) and eICU Collaborative Research Database (eICU-CRD) databases, comprising 176,344 ICU stays. The model incorporated dynamic variables, including vital signs, laboratory results, and medication data, updated hourly, to perform static and continuous mortality risk assessments. External cross-validation and subgroup sensitivity analyses were conducted to evaluate robustness and fairness. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), accuracy, and F1-score. Interpretability was enhanced using integrated gradients to identify key predictors. RESULTS For the static 12-hour to 1-day mortality prediction task, the TBAL model achieved AUROCs of 95.9 (95% CI 94.2-97.5) and 93.3 (95% CI 91.5-95.3) and AUPRCs of 48.5 and 21.6 in MIMIC-IV and eICU-CRD, respectively. Accuracy and F1-scores reached 94.1 and 46.7 in MIMIC-IV and 92.2 and 28.1 in eICU-CRD. In dynamic prediction tasks, AUROCs reached 93.6 (95% CI 93.2-93.9) and 91.9 (95% CI 91.6-92.1), with AUPRCs of 41.3 and 50, respectively. The model maintained high recall for positive cases (82.6% and 79.1% in MIMIC-IV and eICU-CRD). Cross-database validation yielded AUROCs of 81.3 and 76.1, confirming generalizability. Subgroup analysis showed stable performance across age, sex, and severity strata, with top predictors including lactate, vasopressor use, and Glasgow Coma Scale score. CONCLUSIONS The TBAL model offers a robust, interpretable, and generalizable solution for dynamic real-time mortality risk prediction in ICU patients. Its ability to adapt to irregular temporal patterns and to provide hourly updated predictions positions it as a promising decision-support tool. Future work should validate its utility in prospective clinical trials and investigate its integration into real-world ICU workflows to enhance patient outcomes.
Collapse
Affiliation(s)
- Zhuo Zheng
- Department of Anesthesiology, West China Hospital of Sichuan University, Chengdu, China
| | - Jiawei Luo
- West China Biomedical Big Data Center, West China Hospital of Sichuan University, Chengdu, China
| | - Yingchao Zhu
- Department of Anesthesiology, West China Hospital of Sichuan University, Chengdu, China
| | - Lei Du
- Department of Anesthesiology, West China Hospital of Sichuan University, Chengdu, China
| | - Lan Lan
- Information Management and Data Center, Beijing Tiantan Hospital, Beijing, China
| | - Xiaobo Zhou
- Center for Computational Systems Medicine, McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, United States
| | - Xiaoyan Yang
- West China Biomedical Big Data Center, West China Hospital of Sichuan University, Chengdu, China
| | - Shixin Huang
- Department of Scientific Research, The People's Hospital of Yubei District of Chongqing City, Chongqing, China
| |
Collapse
|
2
|
Weller JF, Roedl K, Daniels R, Theile P, Müller J, Peters F, Lengerke C, Bokemeyer C, Wreede LD, Kluge S, Christopeit M. Excess mortality of critically ill patients aged ≥90 years in intensive care units: A retrospective cohort study. Eur J Intern Med 2025:S0953-6205(25)00102-5. [PMID: 40133157 DOI: 10.1016/j.ejim.2025.03.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/06/2025] [Revised: 03/03/2025] [Accepted: 03/17/2025] [Indexed: 03/27/2025]
Abstract
INTRODUCTION The number of elderly patients (≥90 years) admitted to intensive care units (ICUs) is continuously increasing. Survival estimations that differentiate into population and excess mortality are lacking but might be warranted given the higher baseline mortality in this group. METHODS We evaluated excess mortality and risk factors in 1076 ICU patients ≥90 treated at the University Medical Center Hamburg-Eppendorf, Germany, from 2008 to 2019 by relative survival analysis. The Human Mortality Database of the German population served as reference. RESULTS Population mortality - vulgo baseline mortality - accounted for 22.2 % of the observed 1-year mortality rate of 57.8 %. Within the tenth and eleventh decade of life, excess mortality was not significantly influenced by age - neither in ICU patients (HR 0.98 (0.94-1.03), p = 0.46) nor in ICU survivors (HR 0.96 (0.91-1.02), p = 0·15, each per year). Yet, SAPS-II, SOFA scores, dementia, aortic valve stenosis, myocardial infarction and other factors significantly influenced survival. The initial excess mortality rate was high after ICU admission (excess mortality risk λ=0.059 for all patients), but lower after ICU survival (λ=0.022). After around 100 days, the excess mortality risk had disappeared. DISCUSSION In the largest dataset analyzed to date, we were able show that age did not significantly affect excess mortality of ICU patients aged 90 years or older. With respect to the severity of their illness, excess mortality of patients aged 90 years and older in ICUs is within an adequate range. Briefly after patients have left the ICU alive, mortality equals that of the general population.
Collapse
Affiliation(s)
- Jan Frederic Weller
- Department of Medicine II, Hematology, Oncology, Clinical Immunology and Rheumatology, University Hospital Tübingen, Tübingen, Germany; Department of Medicine II, Oncology, Hematology, Bone Marrow Transplantation and Section Pneumology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Kevin Roedl
- Department of Intensive Care Medicine, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| | - Rikus Daniels
- Department of Intensive Care Medicine, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| | - Pauline Theile
- Department of Intensive Care Medicine, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| | - Jakob Müller
- Department of Intensive Care Medicine, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany; Department of Anesthesiology, Tabea Hospital, Hamburg, Germany
| | | | - Claudia Lengerke
- Department of Medicine II, Hematology, Oncology, Clinical Immunology and Rheumatology, University Hospital Tübingen, Tübingen, Germany
| | - Carsten Bokemeyer
- Department of Medicine II, Oncology, Hematology, Bone Marrow Transplantation and Section Pneumology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | | | - Stefan Kluge
- Department of Intensive Care Medicine, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany.
| | - Maximilian Christopeit
- Department of Medicine II, Oncology, Hematology, Bone Marrow Transplantation and Section Pneumology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
| |
Collapse
|
3
|
Mu S, Yan D, Tang J, Zheng Z. Predicting Mortality in Sepsis-Associated Acute Respiratory Distress Syndrome: A Machine Learning Approach Using the MIMIC-III Database. J Intensive Care Med 2025; 40:294-302. [PMID: 39234770 DOI: 10.1177/08850666241281060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/06/2024]
Abstract
BackgroundTo develop and validate a mortality prediction model for patients with sepsis-associated Acute Respiratory Distress Syndrome (ARDS).MethodsThis retrospective cohort study included 2466 patients diagnosed with sepsis and ARDS within 24 h of ICU admission. Demographic, clinical, and laboratory parameters were extracted from Medical Information Mart for Intensive Care III (MIMIC-III) database. Feature selection was performed using the Boruta algorithm, followed by the construction of seven ML models: logistic regression, Naive Bayes, k-nearest neighbor, support vector machine, decision tree, Random Forest, and extreme gradient boosting. Model performance was evaluated using the area under the receiver operating characteristic curve, accuracy, sensitivity, specificity, positive predictive value, and negative predictive value.ResultsThe study identified 24 variables significantly associated with mortality. The optimal ML model, a Random Forest model, demonstrated an AUC of 0.8015 in the test set, with high accuracy and specificity. The model highlighted the importance of blood urea nitrogen, age, urine output, Simplified Acute Physiology Score II, and albumin levels in predicting mortality.ConclusionsThe model's superior predictive performance underscores the potential for integrating advanced analytics into clinical decision-making processes, potentially improving patient outcomes and resource allocation in critical care settings.
Collapse
Affiliation(s)
- Shengtian Mu
- Department of Intensive Care Unit, Cancer Hospital of China Medical University, Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, Shenyang, China
| | - Dongli Yan
- Department of Intensive Care Unit, Cancer Hospital of China Medical University, Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, Shenyang, China
| | - Jie Tang
- Department of Intensive Care Unit, Cancer Hospital of China Medical University, Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, Shenyang, China
| | - Zhen Zheng
- Department of Intensive Care Unit, Cancer Hospital of China Medical University, Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, Shenyang, China
| |
Collapse
|
4
|
Michard F, Mulder MP, Gonzalez F, Sanfilippo F. AI for the hemodynamic assessment of critically ill and surgical patients: focus on clinical applications. Ann Intensive Care 2025; 15:26. [PMID: 39992575 PMCID: PMC11850697 DOI: 10.1186/s13613-025-01448-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2024] [Accepted: 02/09/2025] [Indexed: 02/25/2025] Open
Abstract
Several artificial intelligence (AI)-driven tools have emerged for the hemodynamic evaluation of critically ill and surgical patients. This article provides an overview of current developments and potential clinical applications of machine learning (ML) for blood pressure measurements, hypotension prediction, hemodynamic profiling, and echocardiography. ML algorithms have shown promise in enabling continuous, non-invasive blood pressure monitoring by analyzing pulse oximetry waveforms, though these methods require periodic calibration with traditional oscillometric brachial cuffs. Additionally, a variety of ML models have been trained to forecast impending hypotension. However, clinical research indicates that these algorithms often primarily rely on mean arterial pressure, leading to questions about their added predictive value. The issue of false-positive alerts is also significant and can result in unwarranted clinical interventions. In terms of hemodynamic profiling, ML algorithms have been proposed to automatically classify patients into specific hemodynamic endotypes. However, current evidence suggests these models tend to replicate conventional hemodynamic profiles found in medical textbooks or depicted on advanced hemodynamic monitors. This raises questions about their practical clinical utility, especially given occasional discrepancies that could impact treatment decisions. Point-of-care ultrasound (POCUS) has gained traction for evaluating cardiac function in patients experiencing circulatory shock. ML algorithms now embedded in some POCUS systems can assist by recognizing ultrasound images, guiding users for optimal imaging, automating and reducing the variability of key echocardiographic measurements. These capabilities are especially beneficial for novice operators, potentially enhancing accuracy and confidence in clinical decision-making. In conclusion, while several AI-based technologies show promise for refining hemodynamic assessment in both critically ill and surgical patients, their clinical value varies. Comprehensive validation studies and real-world testing are essential to identify which innovations will genuinely contribute to improving the quality of care.
Collapse
Affiliation(s)
| | - Marijn P Mulder
- Cardiovascular and Respiratory Physiology, University of Twente, Enschede, The Netherlands
| | - Filipe Gonzalez
- Centro Cardiovascular da Universidade de Lisboa, CCUL@RISE, Faculdade de Medicina, Universidade de Lisboa, Lisbon, Portugal
- Intensive Care Department of Hospital Garcia de Orta, Almada, Portugal
| | - Filippo Sanfilippo
- Department of Surgery and Medical-Surgical Specialties, Section of Anesthesia and Intensive Care, University of Catania, Catania, Italy
| |
Collapse
|
5
|
Rockenschaub P, Akay EM, Carlisle BG, Hilbert A, Wendland J, Meyer-Eschenbach F, Näher AF, Frey D, Madai VI. External validation of AI-based scoring systems in the ICU: a systematic review and meta-analysis. BMC Med Inform Decis Mak 2025; 25:5. [PMID: 39762808 PMCID: PMC11702098 DOI: 10.1186/s12911-024-02830-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2024] [Accepted: 12/17/2024] [Indexed: 01/11/2025] Open
Abstract
BACKGROUND Machine learning (ML) is increasingly used to predict clinical deterioration in intensive care unit (ICU) patients through scoring systems. Although promising, such algorithms often overfit their training cohort and perform worse at new hospitals. Thus, external validation is a critical - but frequently overlooked - step to establish the reliability of predicted risk scores to translate them into clinical practice. We systematically reviewed how regularly external validation of ML-based risk scores is performed and how their performance changed in external data. METHODS We searched MEDLINE, Web of Science, and arXiv for studies using ML to predict deterioration of ICU patients from routine data. We included primary research published in English before December 2023. We summarised how many studies were externally validated, assessing differences over time, by outcome, and by data source. For validated studies, we evaluated the change in area under the receiver operating characteristic (AUROC) attributable to external validation using linear mixed-effects models. RESULTS We included 572 studies, of which 84 (14.7%) were externally validated, increasing to 23.9% by 2023. Validated studies made disproportionate use of open-source data, with two well-known US datasets (MIMIC and eICU) accounting for 83.3% of studies. On average, AUROC was reduced by -0.037 (95% CI -0.052 to -0.027) in external data, with more than 0.05 reduction in 49.5% of studies. DISCUSSION External validation, although increasing, remains uncommon. Performance was generally lower in external data, questioning the reliability of some recently proposed ML-based scores. Interpretation of the results was challenged by an overreliance on the same few datasets, implicit differences in case mix, and exclusive use of AUROC.
Collapse
Affiliation(s)
- Patrick Rockenschaub
- CLAIM - Charité Lab for AI in Medicine, Charité - Universitätsmedizin Berlin, Berlin, Germany
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité Universitätsmedizin Berlin, Berlin, Germany
- Institute of Clinical Epidemiology, Public Health, Health Economics, Medical Statistics and Informatics, Medical University of Innsbruck, Innsbruck, Austria
| | - Ela Marie Akay
- CLAIM - Charité Lab for AI in Medicine, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Benjamin Gregory Carlisle
- STREAM - Studies of Translation, Ethics and Medicine, School of Population and Global Health, McGill University, Montréal, Canada
| | - Adam Hilbert
- CLAIM - Charité Lab for AI in Medicine, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Joshua Wendland
- Chair for Artificial Intelligence and Formal Methods, Faculty of Computer Science, Ruhr University, Bochum, Germany
| | - Falk Meyer-Eschenbach
- Institute of Medical Informatics, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Anatol-Fiete Näher
- Institute of Medical Informatics, Charité - Universitätsmedizin Berlin, Berlin, Germany
- Digital Global Public Health, Hasso Plattner Institute for Digital Engineering, University of Potsdam, Potsdam, Germany
| | - Dietmar Frey
- CLAIM - Charité Lab for AI in Medicine, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Vince Istvan Madai
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité Universitätsmedizin Berlin, Berlin, Germany.
- Faculty of Computing, Engineering and the Built Environment, School of Computing and Digital Technology, Birmingham City University, Birmingham, UK.
| |
Collapse
|
6
|
Zhou Y, Sun Y, Pan Y, Dai Y, Xiao Y, Yu Y. Risk prediction models for intensive care unit-acquired weakness in critically ill patients: A systematic review. Aust Crit Care 2025; 38:101066. [PMID: 39013706 DOI: 10.1016/j.aucc.2024.05.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 04/27/2024] [Accepted: 05/02/2024] [Indexed: 07/18/2024] Open
Abstract
BACKGROUND Intensive care unit (ICU)-acquired weakness (ICU-AW) is a critical complication that significantly worsens patient prognosis. It is widely thought that risk prediction models can be harnessed to guide preventive interventions. While the number of ICU-AW risk prediction models is increasing, the quality and applicability of these models in clinical practice remain unclear. OBJECTIVE The objective of this study was to systematically review published studies on risk prediction models for ICU-AW. METHODS We searched electronic databases (PubMed, Web of Science, The Cochrane Library, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), China National Knowledge Infrastructure (CNKI), China Science and Technology Periodical Database (VIP), and Wanfang Database) from inception to October 2023 for studies on ICU-AW risk prediction models. Two independent researchers screened the literature, extracted data, and assessed the risk of bias and applicability of the included studies. RESULTS A total of 2709 articles were identified. After screening, 25 articles were selected, encompassing 25 risk prediction models. The area under the curve for these models ranged from 0.681 to 0.926. Evaluation of bias risk indicated that all included models exhibited a high risk of bias, with three models demonstrating poor applicability. The top five predictors among these models were mechanical ventilation duration, age, Acute Physiology and Chronic Health Evaluation II score, blood lactate levels, and the length of ICU stay. The combined area under the curve of the ten validation models was 0.83 (95% confidence interval: 0.77-0.88), indicating a strong discriminative ability. CONCLUSIONS Overall, ICU-AW risk prediction models demonstrate promising discriminative ability. However, further optimisation is needed to address limitations, including data source heterogeneity, potential biases in study design, and the need for robust statistical validation. Future efforts should prioritise external validation of existing models or the development of high-quality predictive models with superior performance. REGISTRATION The protocol for this study is registered with the International Prospective Register of Systematic Reviews (registration number: CRD42023453187).
Collapse
Affiliation(s)
- Yue Zhou
- College of Nursing, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - YuJian Sun
- College of Nursing, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - YuFan Pan
- College of Nursing, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Yu Dai
- College of Nursing, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Yi Xiao
- College of Nursing, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - YuFeng Yu
- College of Nursing, Chengdu University of Traditional Chinese Medicine, Chengdu, China.
| |
Collapse
|
7
|
Alqahtani MM, Alanazi AMM, Algarni SS, Aljohani H, Alenezi FK, F Alotaibi T, Alotaibi M, K Alqahtani M, Alahmari M, S Alwadeai K, M Alghamdi S, Almeshari MA, Alshammari TF, Mumenah N, Al Harbi E, Al Nufaiei ZF, Alhuthail E, Alzahrani E, Alahmadi H, Alarifi A, Zaidan A, T Ismaeil T. Unveiling the Influence of AI on Advancements in Respiratory Care: Narrative Review. Interact J Med Res 2024; 13:e57271. [PMID: 39705080 PMCID: PMC11699506 DOI: 10.2196/57271] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2024] [Revised: 09/22/2024] [Accepted: 10/28/2024] [Indexed: 12/21/2024] Open
Abstract
BACKGROUND Artificial intelligence is experiencing rapid growth, with continual innovation and advancements in the health care field. OBJECTIVE This study aims to evaluate the application of artificial intelligence technologies across various domains of respiratory care. METHODS We conducted a narrative review to examine the latest advancements in the use of artificial intelligence in the field of respiratory care. The search was independently conducted by respiratory care experts, each focusing on their respective scope of practice and area of interest. RESULTS This review illuminates the diverse applications of artificial intelligence, highlighting its use in areas associated with respiratory care. Artificial intelligence is harnessed across various areas in this field, including pulmonary diagnostics, respiratory care research, critical care or mechanical ventilation, pulmonary rehabilitation, telehealth, public health or health promotion, sleep clinics, home care, smoking or vaping behavior, and neonates and pediatrics. With its multifaceted utility, artificial intelligence can enhance the field of respiratory care, potentially leading to superior health outcomes for individuals under this extensive umbrella. CONCLUSIONS As artificial intelligence advances, elevating academic standards in the respiratory care profession becomes imperative, allowing practitioners to contribute to research and understand artificial intelligence's impact on respiratory care. The permanent integration of artificial intelligence into respiratory care creates the need for respiratory therapists to positively influence its progression. By participating in artificial intelligence development, respiratory therapists can augment their clinical capabilities, knowledge, and patient outcomes.
Collapse
Affiliation(s)
- Mohammed M Alqahtani
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Respiratory Services, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Abdullah M M Alanazi
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Respiratory Services, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Saleh S Algarni
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Respiratory Services, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Hassan Aljohani
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Respiratory Services, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Faraj K Alenezi
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Anesthesia Technology Department, College of Applied Medical Sciences, King Saud Bin Abdul-Aziz University for Health Sciences, Riyadh, Saudi Arabia
| | - Tareq F Alotaibi
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Respiratory Services, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Mansour Alotaibi
- Department of Physical Therapy, Northern Border University, Arar, Saudi Arabia
| | - Mobarak K Alqahtani
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Respiratory Services, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Mushabbab Alahmari
- Department of Respiratory Therapy, College of Applied Medical Sciences, University of Bisha, Bisha, Saudi Arabia
- Health and Humanities Research Center, University of Bisha, Bisha, Saudi Arabia
| | - Khalid S Alwadeai
- Department of Rehabilitation Science, College of Applied Medical Sciences, King Saud University, Riyadh, Saudi Arabia
| | - Saeed M Alghamdi
- Clinical Technology Department, Respiratory Care Program, Faculty of Applied Medical Sciences, Umm Al-Qura University, Mekkah, Saudi Arabia
| | - Mohammed A Almeshari
- Department of Rehabilitation Science, College of Applied Medical Sciences, King Saud University, Riyadh, Saudi Arabia
| | | | - Noora Mumenah
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Respiratory Services, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Ebtihal Al Harbi
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Respiratory Services, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Ziyad F Al Nufaiei
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Jeddah, Saudi Arabia
- King Abdullah International Medical Research Center, Jeddah, Saudi Arabia
| | - Eyas Alhuthail
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Basic Sciences Department, College of Sciences and Health Professions, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
| | - Esam Alzahrani
- Department of Computer Engineering, Al-Baha University, Alaqiq, Saudi Arabia
| | - Husam Alahmadi
- Department of Respiratory Therapy, Faculty of Medical Rehabilitation Sciences, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Abdulaziz Alarifi
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Basic Sciences Department, College of Sciences and Health Professions, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
| | - Amal Zaidan
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Public Health, College of Public Health and Health Informatics, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
| | - Taha T Ismaeil
- Department of Respiratory Therapy, College of Applied Medical Sciences, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Respiratory Services, King Abdulaziz Medical City, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia
| |
Collapse
|
8
|
Eyler Dang L, Klazura G, Yap A, Ozgediz D, Bryce E, Cheung M, Fedatto M, Ameh EA. Development and Internal-External Validation of a Post-Operative Mortality Risk Calculator for Pediatric Surgical Patients in Low- and Middle- Income Countries Using Machine Learning. J Pediatr Surg 2024; 59:161883. [PMID: 39317568 DOI: 10.1016/j.jpedsurg.2024.161883] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/03/2024] [Revised: 08/09/2024] [Accepted: 08/22/2024] [Indexed: 09/26/2024]
Abstract
BACKGROUND The purpose of this study was to develop and validate a mortality risk algorithm for pediatric surgery patients treated at KidsOR sites in 14 low- and middle-income countries. METHODS A SuperLearner machine learning algorithm was trained to predict post-operative mortality by hospital discharge using the retrospectively and prospectively collected KidsOR database including patients treated at 20 KidsOR sites from June 2018 to June 2023. Algorithm performance was evaluated by internal-external cross-validated AUC and calibration. FINDINGS Of 23,905 eligible patients, 21,703 with discharge status recorded were included in the analysis, representing a post-operative mortality rate of 3.1% (671 mortality events). The candidate algorithm with the best cross-validated performance was an extreme gradient boosting model. The cross-validated AUC was 0.945 (95% CI 0.936 to 0.954) and cross-validated calibration slope and intercept were 1.01 (95% CI 0.96 to 1.06) and 0.05 (95% CI -0.10 to 0.21). For Super Learner models trained on all but one site and evaluated in the holdout site for sites with at least 25 mortality events, overall external validation AUC was 0.864 (95% CI 0.846 to 0.882) with calibration slope and intercept of 1.03 (95% CI 0.97 to 1.09) and 1.18 (95% CI 0.98 to 1.39). INTERPRETATION The KidsOR post-operative mortality risk algorithm had outstanding cross-validated discrimination and strong cross-validated calibration. Across all external validation sites, discrimination of Super Learner models trained on the remaining sites was excellent, though re-calibration may be necessary prior to use at new sites. This model has the potential to inform clinical practice and guide resource allocation at KidsOR sites world-wide. TYPE OF STUDY AND LEVEL OF EVIDENCE Observational Study, Level III.
Collapse
Affiliation(s)
| | - Greg Klazura
- University of Illinois at Chicago Department of Surgery, Chicago, IL, USA; Loyola University Medical Center Department of Surgery, Maywood, IL, USA.
| | - Ava Yap
- UCSF Center for Health Equity in Surgery and Anesthesia, San Francisco, CA, USA
| | - Doruk Ozgediz
- UCSF Center for Health Equity in Surgery and Anesthesia, San Francisco, CA, USA
| | - Emma Bryce
- Usher Institute of Population Health Sciences and Informatics at the University of Edinburgh, Edinburgh, Scotland, UK; KidsOR Research Team, Edinburgh, Scotland, UK
| | - Maija Cheung
- Yale University Medical Center, New Haven, CT, USA
| | | | | |
Collapse
|
9
|
Subbaswamy A, Sahiner B, Petrick N, Pai V, Adams R, Diamond MC, Saria S. A data-driven framework for identifying patient subgroups on which an AI/machine learning model may underperform. NPJ Digit Med 2024; 7:334. [PMID: 39572755 PMCID: PMC11582698 DOI: 10.1038/s41746-024-01275-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Accepted: 09/27/2024] [Indexed: 11/24/2024] Open
Abstract
A fundamental goal of evaluating the performance of a clinical model is to ensure it performs well across a diverse intended patient population. A primary challenge is that the data used in model development and testing often consist of many overlapping, heterogeneous patient subgroups that may not be explicitly defined or labeled. While a model's average performance on a dataset may be high, the model can have significantly lower performance for certain subgroups, which may be hard to detect. We describe an algorithmic framework for identifying subgroups with potential performance disparities (AFISP), which produces a set of interpretable phenotypes corresponding to subgroups for which the model's performance may be relatively lower. This could allow model evaluators, including developers and users, to identify possible failure modes prior to wide-scale deployment. We illustrate the application of AFISP by applying it to a patient deterioration model to detect significant subgroup performance disparities, and show that AFISP is significantly more scalable than existing algorithmic approaches.
Collapse
Affiliation(s)
- Adarsh Subbaswamy
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA.
- Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, USA.
| | - Berkman Sahiner
- Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - Nicholas Petrick
- Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - Vinay Pai
- Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - Roy Adams
- Department of Psychiatry and Behavioral Science, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Matthew C Diamond
- Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - Suchi Saria
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA.
- Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
- Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA.
- Bayesian Health, New York, NY, USA.
| |
Collapse
|
10
|
Rockenschaub P, Hilbert A, Kossen T, Elbers P, von Dincklage F, Madai VI, Frey D. The Impact of Multi-Institution Datasets on the Generalizability of Machine Learning Prediction Models in the ICU. Crit Care Med 2024; 52:1710-1721. [PMID: 38958568 PMCID: PMC11469625 DOI: 10.1097/ccm.0000000000006359] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/04/2024]
Abstract
OBJECTIVES To evaluate the transferability of deep learning (DL) models for the early detection of adverse events to previously unseen hospitals. DESIGN Retrospective observational cohort study utilizing harmonized intensive care data from four public datasets. SETTING ICUs across Europe and the United States. PATIENTS Adult patients admitted to the ICU for at least 6 hours who had good data quality. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS Using carefully harmonized data from a total of 334,812 ICU stays, we systematically assessed the transferability of DL models for three common adverse events: death, acute kidney injury (AKI), and sepsis. We tested whether using more than one data source and/or algorithmically optimizing for generalizability during training improves model performance at new hospitals. We found that models achieved high area under the receiver operating characteristic (AUROC) for mortality (0.838-0.869), AKI (0.823-0.866), and sepsis (0.749-0.824) at the training hospital. As expected, AUROC dropped when models were applied at other hospitals, sometimes by as much as -0.200. Using more than one dataset for training mitigated the performance drop, with multicenter models performing roughly on par with the best single-center model. Dedicated methods promoting generalizability did not noticeably improve performance in our experiments. CONCLUSIONS Our results emphasize the importance of diverse training data for DL-based risk prediction. They suggest that as data from more hospitals become available for training, models may become increasingly generalizable. Even so, good performance at a new hospital still depended on the inclusion of compatible hospitals during training.
Collapse
Affiliation(s)
- Patrick Rockenschaub
- Charité Lab for Artificial Intelligence in Medicine (CLAIM), CharitéUniversitätsmedizin Berlin, Berlin, Germany
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité Universitätsmedizin Berlin, Berlin, Germany
- Institute of Clinical Epidemiology, Public Health, Health Economics, Medical Statistics and Informatics, Medical University of Innsbruck, Innsbruck, Austria
| | - Adam Hilbert
- Charité Lab for Artificial Intelligence in Medicine (CLAIM), CharitéUniversitätsmedizin Berlin, Berlin, Germany
| | - Tabea Kossen
- Charité Lab for Artificial Intelligence in Medicine (CLAIM), CharitéUniversitätsmedizin Berlin, Berlin, Germany
| | - Paul Elbers
- Department of Intensive Care Medicine, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
| | - Falk von Dincklage
- Department of Anesthesia, Intensive Care, Emergency and Pain Medicine, Universitätsmedizin Greifswald, Greifswald, Germany
| | - Vince Istvan Madai
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité Universitätsmedizin Berlin, Berlin, Germany
- Faculty of Computing, Engineering and the Built Environment, School of Computing and Digital Technology, Birmingham City University, Birmingham, United Kingdom
| | - Dietmar Frey
- Charité Lab for Artificial Intelligence in Medicine (CLAIM), CharitéUniversitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
11
|
Zhang K, Han Y, Gao YX, Gu FM, Gu ZX, Liang JY, Zhao JY, Zhang T, Gao M, Cai TY, Hu R, Liu TZ, Li B, Zhang Y. Association Between Systolic Blood Pressure and in-Hospital Mortality Among Congestive Heart Failure Patients with Chronic Obstructive Pulmonary Disease in the Intensive Care Unit: A Retrospective Cohort Study. Int J Chron Obstruct Pulmon Dis 2024; 19:2023-2034. [PMID: 39291240 PMCID: PMC11407313 DOI: 10.2147/copd.s448332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Accepted: 06/12/2024] [Indexed: 09/19/2024] Open
Abstract
Background There has been a growing body of research focusing on patients with Congestive Heart Failure (CHF) and chronic obstructive pulmonary disease (COPD) admitted to the intensive care unit (ICU). However, the optimal blood pressure (BP) level for such patients remains insufficiently explored. This study aimed to investigate the associations between systolic blood pressure (SBP) and in-hospital mortality among ICU patients with both CHF and COPD. Methods This retrospective cohort study enrolled 6309 patients from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. SBP was examined as both a continuous and categorical variable, with the primary outcome being in-hospital mortality. The investigation involved multivariable logistic regression, restricted cubic spline regression, and subgroup analysis to determine the relationship between SBP and mortality. Results The cohort consisted of 6309 patients with concurrent CHF and COPD (3246 females and 3063 males), with an average age of 73.0 ± 12.5 years. The multivariate analysis revealed an inverse association between SBP and in-hospital mortality, both as a continuous variable (odds ratio = 0.99 [95% CI, 0.99~1]) and as a categorical variable (divided into quintiles). Restricted cubic spline analysis demonstrated an L-shaped relationship between SBP and mortality risk (P nonlinearity < 0.001), with an inflection point at 99.479 mmHg. Stratified analyses further supported the robustness of this correlation. Conclusion The relationship between SBP and in-hospital mortality in patients with both CHF and COPD follows an L-shaped pattern, with an inflection point at approximately 99.479 mmHg.
Collapse
Affiliation(s)
- Kai Zhang
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Yu Han
- Department of Ophthalmology, First Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Yu Xuan Gao
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Fang Ming Gu
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Zhao Xuan Gu
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Jia Ying Liang
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Jia Yu Zhao
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Tianqi Zhang
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Min Gao
- Department of Cancer Center, First Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Tian Yi Cai
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Rui Hu
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Tian Zhou Liu
- Department of Gastrointestinal Surgery, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Bo Li
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| | - Yixin Zhang
- Cardiovascular Surgery Department, Second Hospital of Jilin University, Changchun, Jilin Province, People's Republic of China
| |
Collapse
|
12
|
Huber M, Bello C, Schober P, Filipovic MG, Luedi MM. Decision Curve Analysis of In-Hospital Mortality Prediction Models: The Relative Value of Pre- and Intraoperative Data For Decision-Making. Anesth Analg 2024; 139:617-28. [PMID: 38315623 DOI: 10.1213/ane.0000000000006874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2024]
Abstract
BACKGROUND Clinical prediction modeling plays a pivotal part in modern clinical care, particularly in predicting the risk of in-hospital mortality. Recent modeling efforts have focused on leveraging intraoperative data sources to improve model performance. However, the individual and collective benefit of pre- and intraoperative data for clinical decision-making remains unknown. We hypothesized that pre- and intraoperative predictors contribute equally to the net benefit in a decision curve analysis (DCA) of in-hospital mortality prediction models that include pre- and intraoperative predictors. METHODS Data from the VitalDB database featuring a subcohort of 6043 patients were used. A total of 141 predictors for in-hospital mortality were grouped into preoperative (demographics, intervention characteristics, and laboratory measurements) and intraoperative (laboratory and monitor data, drugs, and fluids) data. Prediction models using either preoperative, intraoperative, or all data were developed with multiple methods (logistic regression, neural network, random forest, gradient boosting machine, and a stacked learner). Predictive performance was evaluated by the area under the receiver-operating characteristic curve (AUROC) and under the precision-recall curve (AUPRC). Clinical utility was examined with a DCA in the predefined risk preference range (denoted by so-called treatment threshold probabilities) between 0% and 20%. RESULTS AUROC performance of the prediction models ranged from 0.53 to 0.78. AUPRC values ranged from 0.02 to 0.25 (compared to the incidence of 0.09 in our dataset) and high AUPRC values resulted from prediction models based on preoperative laboratory values. A DCA of pre- and intraoperative prediction models highlighted that preoperative data provide the largest overall benefit for decision-making, whereas intraoperative values provide only limited benefit for decision-making compared to preoperative data. While preoperative demographics, comorbidities, and surgery-related data provide the largest benefit for low treatment thresholds up to 5% to 10%, preoperative laboratory measurements become the dominant source for decision support for higher thresholds. CONCLUSIONS When it comes to predicting in-hospital mortality and subsequent decision-making, preoperative demographics, comorbidities, and surgery-related data provide the largest benefit for clinicians with risk-averse preferences, whereas preoperative laboratory values provide the largest benefit for decision-makers with more moderate risk preferences. Our decision-analytic investigation of different predictor categories moves beyond the question of whether certain predictors provide a benefit in traditional performance metrics (eg, AUROC). It offers a nuanced perspective on for whom these predictors might be beneficial in clinical decision-making. Follow-up studies requiring larger datasets and dedicated deep-learning models to handle continuous intraoperative data are essential to examine the robustness of our results.
Collapse
Affiliation(s)
- Markus Huber
- From the Department of Anaesthesiology and Pain Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Corina Bello
- From the Department of Anaesthesiology and Pain Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Patrick Schober
- Department of Anesthesiology, Amsterdam University Medical Centres, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Mark G Filipovic
- From the Department of Anaesthesiology and Pain Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Markus M Luedi
- From the Department of Anaesthesiology and Pain Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| |
Collapse
|
13
|
Olang O, Mohseni S, Shahabinezhad A, Hamidianshirazi Y, Goli A, Abolghasemian M, Shafiee MA, Aarabi M, Alavinia M, Shaker P. Artificial Intelligence-Based Models for Prediction of Mortality in ICU Patients: A Scoping Review. J Intensive Care Med 2024:8850666241277134. [PMID: 39150821 DOI: 10.1177/08850666241277134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/18/2024]
Abstract
BACKGROUND AND OBJECTIVE Healthcare professionals may be able to anticipate more accurately a patient's timing of death and assess their possibility of recovery by implementing a real-time clinical decision support system. Using such a tool, the healthcare system can better understand a patient's condition and make more informed judgements about distributing limited resources. This scoping review aimed to analyze various death prediction AI (Artificial Intelligence) algorithms that have been used in ICU (Intensive Care Unit) patient populations. METHODS The search strategy of this study involved keyword combinations of outcome and patient setting such as mortality, survival, ICU, terminal care. These terms were used to perform database searches in MEDLINE, Embase, and PubMed up to July 2022. The variables, characteristics, and performance of the identified predictive models were summarized. The accuracy of the models was compared using their Area Under the Curve (AUC) values. RESULTS Databases search yielded an initial pool of 8271 articles. A two-step screening process was then applied: first, titles and abstracts were reviewed for relevance, reducing the pool to 429 articles. Next, a full-text review was conducted, further narrowing down the selection to 400 key studies. Out of 400 studies on different tools or models for prediction of mortality in ICUs, 16 papers focused on AI-based models which were ultimately included in this study that have deployed different AI-based and machine learning models to make a prediction about negative patient outcome. The accuracy and performance of the different models varied depending on the patient populations and medical conditions. It was found that AI models compared with traditional tools like SAP3 or APACHE IV score were more accurate in death prediction, with some models achieving an AUC of up to 92.9%. The overall mortality rate ranged from 5% to more than 60% in different studies. CONCLUSION We found that AI-based models exhibit varying performance across different patient populations. To enhance the accuracy of mortality prediction, we recommend customizing models for specific patient groups and medical contexts. By doing so, healthcare professionals may more effectively assess mortality risk and tailor treatments accordingly. Additionally, incorporating additional variables-such as genetic information-into new models can further improve their accuracy.
Collapse
Affiliation(s)
- Orkideh Olang
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Sana Mohseni
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Ali Shahabinezhad
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Yasaman Hamidianshirazi
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Amireza Goli
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Mansour Abolghasemian
- Division of Orthopedic Surgery, Department of Surgery, University of Alberta, Room 404 Community Service Centre, Royal Alexandra Hospital, 10240 Kingsway Avenue, Edmonton, Alberta, Canada, T5H 3V9
| | - Mohammad Ali Shafiee
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Mehdi Aarabi
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Mohammad Alavinia
- KITE, Toronto Rehabilitation Institute, University Health Network, 550 University Ave, Toronto, ON, Canada, M5G 2A2
| | - Pouyan Shaker
- Kansas City University, College of Osteopathic Medicine, Kansas City, MO, USA, 64106
| |
Collapse
|
14
|
Peng X, Zhu T, Chen Q, Zhang Y, Zhou R, Li K, Hao X. A simple machine learning model for the prediction of acute kidney injury following noncardiac surgery in geriatric patients: a prospective cohort study. BMC Geriatr 2024; 24:549. [PMID: 38918723 PMCID: PMC11197315 DOI: 10.1186/s12877-024-05148-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2024] [Accepted: 06/13/2024] [Indexed: 06/27/2024] Open
Abstract
BACKGROUND Surgery in geriatric patients often poses risk of major postoperative complications. Acute kidney injury (AKI) is a common complication following noncardiac surgery and is associated with increased mortality. Early identification of geriatric patients at high risk of AKI could facilitate preventive measures and improve patient prognosis. This study used machine learning methods to identify important features and predict AKI following noncardiac surgery in geriatric patients. METHODS The data for this study were obtained from a prospective cohort. Patients aged ≥ 65 years who received noncardiac surgery from June 2019 to December 2021 were enrolled. Data were split into training set (from June 2019 to March 2021) and internal validation set (from April 2021 to December 2021) by time. The least absolute shrinkage and selection operator (LASSO) regularization algorithm and the random forest recursive feature elimination algorithm (RF-RFE) were used to screen important predictors. Models were trained through extreme gradient boosting (XGBoost), random forest, and LASSO. The SHapley Additive exPlanations (SHAP) package was used to interpret the machine learning model. RESULTS The training set included 6753 geriatric patients. Of these, 250 (3.70%) patients developed AKI. The XGBoost model with RF-RFE selected features outperformed other models with an area under the precision-recall curve (AUPRC) of 0.505 (95% confidence interval [CI]: 0.369-0.626) and an area under the receiver operating characteristic curve (AUROC) of 0.806 (95%CI: 0.733-0.875). The model incorporated ten predictors, including operation site and hypertension. The internal validation set included 3808 geriatric patients, and 96 (2.52%) patients developed AKI. The model maintained good predictive performance with an AUPRC of 0.431 (95%CI: 0.331-0.524) and an AUROC of 0.845 (95%CI: 0.796-0.888) in the internal validation. CONCLUSIONS This study developed a simple machine learning model and a web calculator for predicting AKI following noncardiac surgery in geriatric patients. This model may be a valuable tool for guiding preventive measures and improving patient prognosis. TRIAL REGISTRATION The protocol of this study was approved by the Committee of Ethics from West China Hospital of Sichuan University (2019-473) with a waiver of informed consent and registered at www.chictr.org.cn (ChiCTR1900025160, 15/08/2019).
Collapse
Affiliation(s)
- Xiran Peng
- Department of Anesthesiology, National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, 610041, China
- Research Unit for Perioperative Stress Assessment and Clinical Decision, Chinese Academy of Medical Sciences (2018RU012), West China Hospital, Sichuan University, Chengdu, China
| | - Tao Zhu
- Department of Anesthesiology, National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, 610041, China
- Research Unit for Perioperative Stress Assessment and Clinical Decision, Chinese Academy of Medical Sciences (2018RU012), West China Hospital, Sichuan University, Chengdu, China
| | - Qixu Chen
- Center of Statistical Research, School of Statistics, Southwestern University of Finance and Economics, Chengdu, China
- Joint Lab of Data Science and Business Intelligence, School of Statistics, Southwestern University of Finance and Economics, Chengdu, China
| | - Yuewen Zhang
- Center of Statistical Research, School of Statistics, Southwestern University of Finance and Economics, Chengdu, China
- Joint Lab of Data Science and Business Intelligence, School of Statistics, Southwestern University of Finance and Economics, Chengdu, China
| | - Ruihao Zhou
- Department of Anesthesiology, National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, 610041, China
- Research Unit for Perioperative Stress Assessment and Clinical Decision, Chinese Academy of Medical Sciences (2018RU012), West China Hospital, Sichuan University, Chengdu, China
| | - Ke Li
- Center of Statistical Research, School of Statistics, Southwestern University of Finance and Economics, Chengdu, China.
- Joint Lab of Data Science and Business Intelligence, School of Statistics, Southwestern University of Finance and Economics, Chengdu, China.
| | - Xuechao Hao
- Department of Anesthesiology, National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, 610041, China.
- Research Unit for Perioperative Stress Assessment and Clinical Decision, Chinese Academy of Medical Sciences (2018RU012), West China Hospital, Sichuan University, Chengdu, China.
| |
Collapse
|
15
|
Armoundas AA, Narayan SM, Arnett DK, Spector-Bagdady K, Bennett DA, Celi LA, Friedman PA, Gollob MH, Hall JL, Kwitek AE, Lett E, Menon BK, Sheehan KA, Al-Zaiti SS. Use of Artificial Intelligence in Improving Outcomes in Heart Disease: A Scientific Statement From the American Heart Association. Circulation 2024; 149:e1028-e1050. [PMID: 38415358 PMCID: PMC11042786 DOI: 10.1161/cir.0000000000001201] [Citation(s) in RCA: 50] [Impact Index Per Article: 50.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/29/2024]
Abstract
A major focus of academia, industry, and global governmental agencies is to develop and apply artificial intelligence and other advanced analytical tools to transform health care delivery. The American Heart Association supports the creation of tools and services that would further the science and practice of precision medicine by enabling more precise approaches to cardiovascular and stroke research, prevention, and care of individuals and populations. Nevertheless, several challenges exist, and few artificial intelligence tools have been shown to improve cardiovascular and stroke care sufficiently to be widely adopted. This scientific statement outlines the current state of the art on the use of artificial intelligence algorithms and data science in the diagnosis, classification, and treatment of cardiovascular disease. It also sets out to advance this mission, focusing on how digital tools and, in particular, artificial intelligence may provide clinical and mechanistic insights, address bias in clinical studies, and facilitate education and implementation science to improve cardiovascular and stroke outcomes. Last, a key objective of this scientific statement is to further the field by identifying best practices, gaps, and challenges for interested stakeholders.
Collapse
|
16
|
Hu C, Gao C, Li T, Liu C, Peng Z. Explainable artificial intelligence model for mortality risk prediction in the intensive care unit: a derivation and validation study. Postgrad Med J 2024; 100:219-227. [PMID: 38244550 DOI: 10.1093/postmj/qgad144] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2023] [Revised: 12/11/2023] [Accepted: 12/18/2023] [Indexed: 01/22/2024]
Abstract
BACKGROUND The lack of transparency is a prevalent issue among the current machine-learning (ML) algorithms utilized for predicting mortality risk. Herein, we aimed to improve transparency by utilizing the latest ML explicable technology, SHapley Additive exPlanation (SHAP), to develop a predictive model for critically ill patients. METHODS We extracted data from the Medical Information Mart for Intensive Care IV database, encompassing all intensive care unit admissions. We employed nine different methods to develop the models. The most accurate model, with the highest area under the receiver operating characteristic curve, was selected as the optimal model. Additionally, we used SHAP to explain the workings of the ML model. RESULTS The study included 21 395 critically ill patients, with a median age of 68 years (interquartile range, 56-79 years), and most patients were male (56.9%). The cohort was randomly split into a training set (N = 16 046) and a validation set (N = 5349). Among the nine models developed, the Random Forest model had the highest accuracy (87.62%) and the best area under the receiver operating characteristic curve value (0.89). The SHAP summary analysis showed that Glasgow Coma Scale, urine output, and blood urea nitrogen were the top three risk factors for outcome prediction. Furthermore, SHAP dependency analysis and SHAP force analysis were used to interpret the Random Forest model at the factor level and individual level, respectively. CONCLUSION A transparent ML model for predicting outcomes in critically ill patients using SHAP methodology is feasible and effective. SHAP values significantly improve the explainability of ML models.
Collapse
Affiliation(s)
- Chang Hu
- Department of Critical Care Medicine, Zhongnan Hospital of Wuhan University, Wuhan 430071, Hubei, China
- Clinical Research Center of Hubei Critical Care Medicine, Wuhan 430071, Hubei, China
| | - Chao Gao
- Department of Critical Care Medicine, Zhongnan Hospital of Wuhan University, Wuhan 430071, Hubei, China
- Clinical Research Center of Hubei Critical Care Medicine, Wuhan 430071, Hubei, China
| | - Tianlong Li
- Department of Critical Care Medicine, Zhongnan Hospital of Wuhan University, Wuhan 430071, Hubei, China
- Clinical Research Center of Hubei Critical Care Medicine, Wuhan 430071, Hubei, China
| | - Chang Liu
- Department of Critical Care Medicine, Zhongnan Hospital of Wuhan University, Wuhan 430071, Hubei, China
- Clinical Research Center of Hubei Critical Care Medicine, Wuhan 430071, Hubei, China
| | - Zhiyong Peng
- Department of Critical Care Medicine, Zhongnan Hospital of Wuhan University, Wuhan 430071, Hubei, China
- Clinical Research Center of Hubei Critical Care Medicine, Wuhan 430071, Hubei, China
| |
Collapse
|
17
|
Lim L, Gim U, Cho K, Yoo D, Ryu HG, Lee HC. Real-time machine learning model to predict short-term mortality in critically ill patients: development and international validation. Crit Care 2024; 28:76. [PMID: 38486247 PMCID: PMC10938661 DOI: 10.1186/s13054-024-04866-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Accepted: 03/09/2024] [Indexed: 03/18/2024] Open
Abstract
BACKGROUND A real-time model for predicting short-term mortality in critically ill patients is needed to identify patients at imminent risk. However, the performance of the model needs to be validated in various clinical settings and ethnicities before its clinical application. In this study, we aim to develop an ensemble machine learning model using routinely measured clinical variables at a single academic institution in South Korea. METHODS We developed an ensemble model using deep learning and light gradient boosting machine models. Internal validation was performed using the last two years of the internal cohort dataset, collected from a single academic hospital in South Korea between 2007 and 2021. External validation was performed using the full Medical Information Mart for Intensive Care (MIMIC), eICU-Collaborative Research Database (eICU-CRD), and Amsterdam University Medical Center database (AmsterdamUMCdb) data. The area under the receiver operating characteristic curve (AUROC) was calculated and compared to that for the National Early Warning Score (NEWS). RESULTS The developed model (iMORS) demonstrated high predictive performance with an internal AUROC of 0.964 (95% confidence interval [CI] 0.963-0.965) and external AUROCs of 0.890 (95% CI 0.889-0.891) for MIMIC, 0.886 (95% CI 0.885-0.887) for eICU-CRD, and 0.870 (95% CI 0.868-0.873) for AmsterdamUMCdb. The model outperformed the NEWS with higher AUROCs in the internal and external validation (0.866 for the internal, 0.746 for MIMIC, 0.798 for eICU-CRD, and 0.819 for AmsterdamUMCdb; p < 0.001). CONCLUSIONS Our real-time machine learning model to predict short-term mortality in critically ill patients showed excellent performance in both internal and external validations. This model could be a useful decision-support tool in the intensive care units to assist clinicians.
Collapse
Affiliation(s)
- Leerang Lim
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea
| | - Ukdong Gim
- VUNO, 479 Gangnam-Daero, Seocho-gu, Seoul, 06541, Republic of Korea
| | - Kyungjae Cho
- VUNO, 479 Gangnam-Daero, Seocho-gu, Seoul, 06541, Republic of Korea
| | - Dongjoon Yoo
- VUNO, 479 Gangnam-Daero, Seocho-gu, Seoul, 06541, Republic of Korea
- Department of Critical Care Medicine and Emergency Medicine, Inha University College of Medicine, 100 Inha-ro, Michuhol-gu, Incheon, 22212, Republic of Korea
| | - Ho Geol Ryu
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea
- Department of Critical Care Medicine, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea
| | - Hyung-Chul Lee
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea.
| |
Collapse
|
18
|
Padte S, Samala Venkata V, Mehta P, Tawfeeq S, Kashyap R, Surani S. 21st century critical care medicine: An overview. World J Crit Care Med 2024; 13:90176. [PMID: 38633477 PMCID: PMC11019625 DOI: 10.5492/wjccm.v13.i1.90176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/26/2023] [Revised: 12/28/2023] [Accepted: 01/24/2024] [Indexed: 03/05/2024] Open
Abstract
Critical care medicine in the 21st century has witnessed remarkable advancements that have significantly improved patient outcomes in intensive care units (ICUs). This abstract provides a concise summary of the latest developments in critical care, highlighting key areas of innovation. Recent advancements in critical care include Precision Medicine: Tailoring treatments based on individual patient characteristics, genomics, and biomarkers to enhance the effectiveness of therapies. The objective is to describe the recent advancements in Critical Care Medicine. Telemedicine: The integration of telehealth technologies for remote patient monitoring and consultation, facilitating timely interventions. Artificial intelligence (AI): AI-driven tools for early disease detection, predictive analytics, and treatment optimization, enhancing clinical decision-making. Organ Support: Advanced life support systems, such as Extracorporeal Membrane Oxygenation and Continuous Renal Replacement Therapy provide better organ support. Infection Control: Innovative infection control measures to combat emerging pathogens and reduce healthcare-associated infections. Ventilation Strategies: Precision ventilation modes and lung-protective strategies to minimize ventilator-induced lung injury. Sepsis Management: Early recognition and aggressive management of sepsis with tailored interventions. Patient-Centered Care: A shift towards patient-centered care focusing on psychological and emotional well-being in addition to medical needs. We conducted a thorough literature search on PubMed, EMBASE, and Scopus using our tailored strategy, incorporating keywords such as critical care, telemedicine, and sepsis management. A total of 125 articles meeting our criteria were included for qualitative synthesis. To ensure reliability, we focused only on articles published in the English language within the last two decades, excluding animal studies, in vitro/molecular studies, and non-original data like editorials, letters, protocols, and conference abstracts. These advancements reflect a dynamic landscape in critical care medicine, where technology, research, and patient-centered approaches converge to improve the quality of care and save lives in ICUs. The future of critical care promises even more innovative solutions to meet the evolving challenges of modern medicine.
Collapse
Affiliation(s)
- Smitesh Padte
- Department of Research, Global Remote Research Scholars Program, St. Paul, MN 55104, United States
| | | | - Priyal Mehta
- Department of Research, Global Remote Research Scholars Program, St. Paul, MN 55104, United States
| | - Sawsan Tawfeeq
- Department of Research, Global Remote Research Scholars Program, St. Paul, MN 55104, United States
| | - Rahul Kashyap
- Department of Research, Global Remote Research Scholars Program, St. Paul, MN 55104, United States
- Department of Research, WellSpan Health, York, PA 17403, United States
- Department of Pulmonary & Critical Care Medicine, Mayo Clinic, Rochester, MN 55905, United States
| | - Salim Surani
- Department of Pulmonary & Critical Care Medicine, Mayo Clinic, Rochester, MN 55905, United States
- Department of Medicine & Pharmacology, Texas A&M University, College Station, TX 77843, United States
| |
Collapse
|
19
|
Mollura M, Chicco D, Paglialonga A, Barbieri R. Identifying prognostic factors for survival in intensive care unit patients with SIRS or sepsis by machine learning analysis on electronic health records. PLOS DIGITAL HEALTH 2024; 3:e0000459. [PMID: 38489347 PMCID: PMC10942078 DOI: 10.1371/journal.pdig.0000459] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Accepted: 02/05/2024] [Indexed: 03/17/2024]
Abstract
BACKGROUND Systemic inflammatory response syndrome (SIRS) and sepsis are the most common causes of in-hospital death. However, the characteristics associated with the improvement in the patient conditions during the ICU stay were not fully elucidated for each population as well as the possible differences between the two. GOAL The aim of this study is to highlight the differences between the prognostic clinical features for the survival of patients diagnosed with SIRS and those of patients diagnosed with sepsis by using a multi-variable predictive modeling approach with a reduced set of easily available measurements collected at the admission to the intensive care unit (ICU). METHODS Data were collected from 1,257 patients (816 non-sepsis SIRS and 441 sepsis) admitted to the ICU. We compared the performance of five machine learning models in predicting patient survival. Matthews correlation coefficient (MCC) was used to evaluate model performances and feature importance, and by applying Monte Carlo stratified Cross-Validation. RESULTS Extreme Gradient Boosting (MCC = 0.489) and Logistic Regression (MCC = 0.533) achieved the highest results for SIRS and sepsis cohorts, respectively. In order of importance, APACHE II, mean platelet volume (MPV), eosinophil counts (EoC), and C-reactive protein (CRP) showed higher importance for predicting sepsis patient survival, whereas, SOFA, APACHE II, platelet counts (PLTC), and CRP obtained higher importance in the SIRS cohort. CONCLUSION By using complete blood count parameters as predictors of ICU patient survival, machine learning models can accurately predict the survival of SIRS and sepsis ICU patients. Interestingly, feature importance highlights the role of CRP and APACHE II in both SIRS and sepsis populations. In addition, MPV and EoC are shown to be important features for the sepsis population only, whereas SOFA and PLTC have higher importance for SIRS patients.
Collapse
Affiliation(s)
- Maximiliano Mollura
- Dipartimento di Elettronica Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy
| | - Davide Chicco
- Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- Dipartimento di Informatica Sistemistica e Comunicazione, Università di Milano-Bicocca, Milan, Italy
| | - Alessia Paglialonga
- CNR-Istituto di Elettronica e di Ingegneria dell’Informazione e delle Telecomunicazioni (CNR-IEIIT), Milan, Italy
| | - Riccardo Barbieri
- Dipartimento di Elettronica Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy
| |
Collapse
|
20
|
Cho KJ, Kim KH, Choi J, Yoo D, Kim J. External Validation of Deep Learning-Based Cardiac Arrest Risk Management System for Predicting In-Hospital Cardiac Arrest in Patients Admitted to General Wards Based on Rapid Response System Operating and Nonoperating Periods: A Single-Center Study. Crit Care Med 2024; 52:e110-e120. [PMID: 38381018 PMCID: PMC10876170 DOI: 10.1097/ccm.0000000000006137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/22/2024]
Abstract
OBJECTIVES The limitations of current early warning scores have prompted the development of deep learning-based systems, such as deep learning-based cardiac arrest risk management systems (DeepCARS). Unfortunately, in South Korea, only two institutions operate 24-hour Rapid Response System (RRS), whereas most hospitals have part-time or no RRS coverage at all. This study validated the predictive performance of DeepCARS during RRS operation and nonoperation periods and explored its potential beyond RRS operating hours. DESIGN Retrospective cohort study. SETTING In this 1-year retrospective study conducted at Yonsei University Health System Severance Hospital in South Korea, DeepCARS was compared with conventional early warning systems for predicting in-hospital cardiac arrest (IHCA). The study focused on adult patients admitted to the general ward, with the primary outcome being IHCA-prediction performance within 24 hours of the alarm. PATIENTS We analyzed the data records of adult patients admitted to a general ward from September 1, 2019, to August 31, 2020. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS Performance evaluation was conducted separately for the operational and nonoperational periods of the RRS, using the area under the receiver operating characteristic curve (AUROC) as the metric. DeepCARS demonstrated a superior AUROC as compared with the Modified Early Warning Score (MEWS) and the National Early Warning Score (NEWS), both during RRS operating and nonoperating hours. Although the MEWS and NEWS exhibited varying performance across the two periods, DeepCARS showed consistent performance. CONCLUSIONS The accuracy and efficiency for predicting IHCA of DeepCARS were superior to that of conventional methods, regardless of whether the RRS was in operation. These findings emphasize that DeepCARS is an effective screening tool suitable for hospitals with full-time RRS, part-time RRS, and even those without any RRS.
Collapse
Affiliation(s)
- Kyung-Jae Cho
- Department of Research and Development, VUNO, Seoul, Republic of Korea
| | - Kwan Hyung Kim
- Department of Anesthesiology and Pain Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Jaewoo Choi
- Department of Research and Development, VUNO, Seoul, Republic of Korea
| | - Dongjoon Yoo
- Department of Research and Development, VUNO, Seoul, Republic of Korea
- Department of Critical Care Medicine and Emergency Medicine, Inha University Hospital, Incheon, Republic of Korea
| | - Jeongmin Kim
- Department of Anesthesiology and Pain Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea
- Anesthesia and Pain Research Institute, Yonsei University College of Medicine, Seoul, Republic of Korea
- Institute for Innovation in Digital Healthcare, Yonsei University, Seoul, Republic of Korea
| |
Collapse
|
21
|
Mani RK, Bhatnagar S, Butola S, Gursahani R, Mehta D, Simha S, Divatia JV, Kumar A, Iyer SK, Deodhar J, Bhat RS, Salins N, Thota RS, Mathur R, Iyer RK, Gupta S, Kulkarni P, Murugan S, Nasa P, Myatra SN. Indian Society of Critical Care Medicine and Indian Association of Palliative Care Expert Consensus and Position Statements for End-of-life and Palliative Care in the Intensive Care Unit. Indian J Crit Care Med 2024; 28:200-250. [PMID: 38477011 PMCID: PMC10926026 DOI: 10.5005/jp-journals-10071-24661] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Accepted: 02/28/2024] [Indexed: 03/14/2024] Open
Abstract
UNLABELLED End-of-life care (EOLC) exemplifies the joint mission of intensive and palliative care (PC) in their human-centeredness. The explosion of technological advances in medicine must be balanced with the culture of holistic care. Inevitably, it brings together the science and the art of medicine in their full expression. High-quality EOLC in the ICU is grounded in evidence, ethical principles, and professionalism within the framework of the Law. Expert professional statements over the last two decades in India were developed while the law was evolving. Recent landmark Supreme Court judgments have necessitated a review of the clinical pathway for EOLC outlined in the previous statements. Much empirical and interventional evidence has accumulated since the position statement in 2014. This iteration of the joint Indian Society of Critical Care Medicine-Indian Association of Palliative Care (ISCCM-IAPC) Position Statement for EOLC combines contemporary evidence, ethics, and law for decision support by the bedside in Indian ICUs. HOW TO CITE THIS ARTICLE Mani RK, Bhatnagar S, Butola S, Gursahani R, Mehta D, Simha S, et al. Indian Society of Critical Care Medicine and Indian Association of Palliative Care Expert Consensus and Position Statements for End-of-life and Palliative Care in the Intensive Care Unit. Indian J Crit Care Med 2024;28(3):200-250.
Collapse
Affiliation(s)
- Raj K Mani
- Department of Critical Care and Pulmonology, Yashoda Super Specialty Hospital, Ghaziabad, Kaushambi, Uttar Pradesh, India
| | - Sushma Bhatnagar
- Department of Onco-Anaesthesia and Palliative Medicine, All India Institute of Medical Sciences, New Delhi, India
| | - Savita Butola
- Department of Palliative Care, Border Security Force Sector Hospital, Panisagar, Tripura, India
| | - Roop Gursahani
- Department of Neurology, P. D. Hinduja National Hospital & Medical Research Centre, Mumbai, Maharashtra, India
| | - Dhvani Mehta
- Division of Health, Vidhi Centre for Legal Policy, New Delhi, India
| | - Srinagesh Simha
- Department of Palliative Care, Karunashraya, Bengaluru, Karnataka, India
| | - Jigeeshu V Divatia
- Department of Anaesthesia, Critical Care, and Pain, Tata Memorial Hospital, Homi Bhabha National Institute, Mumbai, Maharashtra, India
| | - Arun Kumar
- Department of Intensive Care, Medical Intensive Care Unit, Fortis Healthcare Ltd, Mohali, Punjab, India
| | - Shiva K Iyer
- Department of Critical Care, Bharati Vidyapeeth (Deemed to be University) Medical College, Pune, Maharashtra, India
| | - Jayita Deodhar
- Department Palliative Care, Tata Memorial Hospital, Mumbai, Maharashtra, India
| | - Rajani S Bhat
- Department of Interventional Pulmonology and Palliative Medicine, SPARSH Hospitals, Bengaluru, Karnataka, India
| | - Naveen Salins
- Department of Palliative Medicine and Supportive Care, Kasturba Medical College Manipal Academy of Higher Education, Manipal, Karnataka, India
| | - Raghu S Thota
- Department Palliative Care, Tata Memorial Hospital, Mumbai, Maharashtra, India
| | - Roli Mathur
- Department of Bioethics, Indian Council of Medical Research, Bengaluru, Karnataka, India
| | - Rajam K Iyer
- Department of Palliative Care, Bhatia Hospital; P. D. Hinduja National Hospital & Medical Research Centre, Mumbai, Maharashtra, India
| | - Sudeep Gupta
- Department of Medical Oncology, Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, India
| | | | - Sangeetha Murugan
- Department of Education and Research, Karunashraya, Bengaluru, Karnataka, India
| | - Prashant Nasa
- Department of Critical Care Medicine, NMC Specialty Hospital, Dubai, United Arab Emirates
| | - Sheila N Myatra
- Department of Anesthesiology, Critical Care and Pain, Tata Memorial Hospital, Homi Bhabha National Institute, Mumbai, India
| |
Collapse
|
22
|
Chang Junior J, Caneo LF, Turquetto ALR, Amato LP, Arita ECTC, Fernandes AMDS, Trindade EM, Jatene FB, Dossou PE, Jatene MB. Predictors of in-ICU length of stay among congenital heart defect patients using artificial intelligence model: A pilot study. Heliyon 2024; 10:e25406. [PMID: 38370176 PMCID: PMC10869777 DOI: 10.1016/j.heliyon.2024.e25406] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 01/23/2024] [Accepted: 01/25/2024] [Indexed: 02/20/2024] Open
Abstract
Objective This study aims to develop a predictive model using artificial intelligence to estimate the ICU length of stay (LOS) for Congenital Heart Defects (CHD) patients after surgery, improving care planning and resource management. Design We analyze clinical data from 2240 CHD surgery patients to create and validate the predictive model. Twenty AI models are developed and evaluated for accuracy and reliability. Setting The study is conducted in a Brazilian hospital's Cardiovascular Surgery Department, focusing on transplants and cardiopulmonary surgeries. Participants Retrospective analysis is conducted on data from 2240 consecutive CHD patients undergoing surgery. Interventions Ninety-three pre and intraoperative variables are used as ICU LOS predictors. Measurements and main results Utilizing regression and clustering methodologies for ICU LOS (ICU Length of Stay) estimation, the Light Gradient Boosting Machine, using regression, achieved a Mean Squared Error (MSE) of 15.4, 11.8, and 15.2 days for training, testing, and unseen data. Key predictors included metrics such as "Mechanical Ventilation Duration", "Weight on Surgery Date", and "Vasoactive-Inotropic Score". Meanwhile, the clustering model, Cat Boost Classifier, attained an accuracy of 0.6917 and AUC of 0.8559 with similar key predictors. Conclusions Patients with higher ventilation times, vasoactive-inotropic scores, anoxia time, cardiopulmonary bypass time, and lower weight, height, BMI, age, hematocrit, and presurgical oxygen saturation have longer ICU stays, aligning with existing literature.
Collapse
Affiliation(s)
- João Chang Junior
- Hospital Das Clínicas HCFMUSP, Universidade de São Paulo, Instituto Do Coração - InCor, Av. Dr. Enéas Carvalho de Aguiar, 44, CEP 05403-000, São Paulo, Brazil
- Escola Superior de Engenharia e Gestão - ESEG, Rua Apeninos, 960, São Paulo, Brazil
- Centro Universitário Armando Alvares Penteado - FAAP, Rua Alagoas, 903, São Paulo, Brazil
| | - Luiz Fernando Caneo
- Hospital Das Clínicas HCFMUSP, Universidade de São Paulo, Instituto Do Coração - InCor, Av. Dr. Enéas Carvalho de Aguiar, 44, CEP 05403-000, São Paulo, Brazil
| | - Aida Luiza Ribeiro Turquetto
- Hospital Das Clínicas HCFMUSP, Universidade de São Paulo, Instituto Do Coração - InCor, Av. Dr. Enéas Carvalho de Aguiar, 44, CEP 05403-000, São Paulo, Brazil
- Núcleo de Avaliação de Tecnologias da Saúde - NATS-HCFMUSP, Brazil
| | - Luciana Patrick Amato
- Hospital Das Clínicas HCFMUSP, Universidade de São Paulo, Instituto Do Coração - InCor, Av. Dr. Enéas Carvalho de Aguiar, 44, CEP 05403-000, São Paulo, Brazil
- Núcleo de Avaliação de Tecnologias da Saúde - NATS-HCFMUSP, Brazil
| | - Elisandra Cristina Trevisan Calvo Arita
- Hospital Das Clínicas HCFMUSP, Universidade de São Paulo, Instituto Do Coração - InCor, Av. Dr. Enéas Carvalho de Aguiar, 44, CEP 05403-000, São Paulo, Brazil
| | - Alfredo Manoel da Silva Fernandes
- Hospital Das Clínicas HCFMUSP, Universidade de São Paulo, Instituto Do Coração - InCor, Av. Dr. Enéas Carvalho de Aguiar, 44, CEP 05403-000, São Paulo, Brazil
| | - Evelinda Marramon Trindade
- Núcleo de Avaliação de Tecnologias da Saúde - NATS-HCFMUSP, Brazil
- Laboratório de Ensino, Pesquisa e Inovação Em Saúde - LEPIC-HCFMUSP, Superintendência / Hospital Das Clínicas da FMUSP, Rua Dr. Ovidio Pires de Campos, 225, 5°. Andar – Superintendência, Sao Paulo, Brazil
- Sao Paulo State Health Secretariat–SES-SP, Sao Paulo, Brazil
| | - Fábio Biscegli Jatene
- Hospital Das Clínicas HCFMUSP, Universidade de São Paulo, Instituto Do Coração - InCor, Av. Dr. Enéas Carvalho de Aguiar, 44, CEP 05403-000, São Paulo, Brazil
| | - Paul-Eric Dossou
- Institut Catholique des Arts et Metiers–Icam, Paris-Senart, France
| | - Marcelo Biscegli Jatene
- Hospital Das Clínicas HCFMUSP, Universidade de São Paulo, Instituto Do Coração - InCor, Av. Dr. Enéas Carvalho de Aguiar, 44, CEP 05403-000, São Paulo, Brazil
| |
Collapse
|
23
|
Drudi C, Mollura M, Lehman LWH, Barbieri R. A Reinforcement Learning Model for Optimal Treatment Strategies in Intensive Care: Assessment of the Role of Cardiorespiratory Features. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2024; 5:806-815. [PMID: 39559781 PMCID: PMC11573419 DOI: 10.1109/ojemb.2024.3367236] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 12/14/2023] [Accepted: 02/14/2024] [Indexed: 11/20/2024] Open
Abstract
Goal: The purpose of this study is to evaluate the importance of cardiorespiratory variables within a Reinforcement Learning (RL) recommendation system aimed at establishing optimal strategies for drug treatment of septic patients in the intensive care unit (ICU). Methods: We developed a RL model in order to establish drug administration strategies for septic patients using only a set of cardiorespiratory variables. We then compared this model with other RL models trained with a different set of features. We selected patients meeting the Sepsis-3 criteria from the Multi-parameter Intelligent Monitoring in Intensive Care (MIMIC III) database, resulting in a total of 20,496 ICU admissions. A Markov Decision Process (MDP) was built on the extracted discrete time-series. A policy iteration algorithm was used to obtain the optimal AI policy for the MDP. The policy performance was then evaluated using the WIS estimator. The process was repeated for each set of variables and compared to a set of baseline benchmark policies. Results: The model trained with cardiorespiratory variables outperformed all other models considered, resulting in a 95% confidence lower bound score of 97.48. This finding highlights the importance of cardiovascular variables in the clinical RL recommendation system. Conclusions: We established an efficient RL model for sepsis treatment in the ICU and demonstrated that cardiorespiratory variables provides critical information in devising optimal policies. Given the potentially continuous availability of cardiorespiratory features extracted from bedside physiological waveform monitoring, the proposed framework paves the way for a real time recommendation system for sepsis treatment.
Collapse
Affiliation(s)
- Cristian Drudi
- Department of Electronics, Informatics and EngineeringPolitecnico di Milano20133MilanoItaly
| | - Maximiliano Mollura
- Department of Electronics, Informatics and EngineeringPolitecnico di Milano20133MilanoItaly
| | - Li-wei H. Lehman
- Institute for Medical Engineering and ScienceMassachusetts Institute of TechnologyCambridgeMA02139USA
| | - Riccardo Barbieri
- Department of Electronics, Informatics and EngineeringPolitecnico di Milano20133MilanoItaly
| |
Collapse
|
24
|
Boussen S, Benard-Tertrais M, Ogéa M, Malet A, Simeone P, Antonini F, Bruder N, Velly L. Heart rate complexity helps mortality prediction in the intensive care unit: A pilot study using artificial intelligence. Comput Biol Med 2024; 169:107934. [PMID: 38183707 DOI: 10.1016/j.compbiomed.2024.107934] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Revised: 12/10/2023] [Accepted: 01/01/2024] [Indexed: 01/08/2024]
Abstract
BACKGROUND In intensive care units (ICUs), accurate mortality prediction is crucial for effective patient management and resource allocation. The Simplified Acute Physiology Score II (SAPS-2), though commonly used, relies heavily on comprehensive clinical data and blood samples. This study sought to develop an artificial intelligence (AI) model utilizing key hemodynamic parameters to predict ICU mortality within the first 24 h and assess its performance relative to SAPS-2. METHODS We conducted an analysis of select hemodynamic parameters and the structure of heart rate curves to identify potential predictors of ICU mortality. A machine-learning model was subsequently trained and validated on distinct patient cohorts. The AI algorithm's performance was then compared to the SAPS-2, focusing on classification accuracy, calibration, and generalizability. MEASUREMENTS AND MAIN RESULTS The study included 1298 ICU admissions from March 27th, 2015, to March 27th, 2017. An additional cohort from 2022 to 2023 comprised 590 patients, resulting in a total dataset of 1888 patients. The observed mortality rate stood at 24.0%. Key determinants of mortality were the Glasgow Coma Scale score, heart rate complexity, patient age, duration of diastolic blood pressure below 50 mmHg, heart rate variability, and specific mean and systolic blood pressure thresholds. The AI model, informed by these determinants, exhibited a performance profile in predicting mortality that was comparable, if not superior, to the SAPS-2. CONCLUSIONS The AI model, which integrates heart rate and blood pressure curve analyses with basic clinical parameters, provides a methodological approach to predict in-hospital mortality in ICU patients. This model offers an alternative to existing tools that depend on extensive clinical data and laboratory inputs. Its potential integration into ICU monitoring systems may facilitate more streamlined mortality prediction processes.
Collapse
Affiliation(s)
- Salah Boussen
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France; Laboratoire de Biomécanique Appliquée-Université Gustave-Eiffel, Aix-Marseille Université, UMR T24, 51 boulevard Pierre Dramard, 13015, Marseille, France.
| | - Manuela Benard-Tertrais
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Mathilde Ogéa
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Arthur Malet
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Pierre Simeone
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France; Aix Marseille University, CNRS, Inst Neurosci Timone, UMR7289, Marseille, France
| | - François Antonini
- Intensive Care and Anesthesiology Department, Hôpital Nord Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Nicolas Bruder
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Lionel Velly
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France; Aix Marseille University, CNRS, Inst Neurosci Timone, UMR7289, Marseille, France
| |
Collapse
|
25
|
Miyazawa Y, Katsuta N, Nara T, Nojiri S, Naito T, Hiki M, Ichikawa M, Takeshita Y, Kato T, Okumura M, Tobita M. Identification of risk factors for the onset of delirium associated with COVID-19 by mining nursing records. PLoS One 2024; 19:e0296760. [PMID: 38241284 PMCID: PMC10798448 DOI: 10.1371/journal.pone.0296760] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 12/18/2023] [Indexed: 01/21/2024] Open
Abstract
COVID-19 has a range of complications, from no symptoms to severe pneumonia. It can also affect multiple organs including the nervous system. COVID-19 affects the brain, leading to neurological symptoms such as delirium. Delirium, a sudden change in consciousness, can increase the risk of death and prolong the hospital stay. However, research on delirium prediction in patients with COVID-19 is insufficient. This study aimed to identify new risk factors that could predict the onset of delirium in patients with COVID-19 using machine learning (ML) applied to nursing records. This retrospective cohort study used natural language processing and ML to develop a model for classifying the nursing records of patients with delirium. We extracted the features of each word from the model and grouped similar words. To evaluate the usefulness of word groups in predicting the occurrence of delirium in patients with COVID-19, we analyzed the temporal changes in the frequency of occurrence of these word groups before and after the onset of delirium. Moreover, the sensitivity, specificity, and odds ratios were calculated. We identified (1) elimination-related behaviors and conditions and (2) abnormal patient behavior and conditions as risk factors for delirium. Group 1 had the highest sensitivity (0.603), whereas group 2 had the highest specificity and odds ratio (0.938 and 6.903, respectively). These results suggest that these parameters may be useful in predicting delirium in these patients. The risk factors for COVID-19-associated delirium identified in this study were more specific but less sensitive than the ICDSC (Intensive Care Delirium Screening Checklist) and CAM-ICU (Confusion Assessment Method for the Intensive Care Unit). However, they are superior to the ICDSC and CAM-ICU because they can predict delirium without medical staff and at no cost.
Collapse
Affiliation(s)
- Yusuke Miyazawa
- Department of Healthcare Innovation, Juntendo University Graduate School of Medicine, Tokyo, Japan
| | - Narimasa Katsuta
- Department of Psychiatry, Juntendo University Faculty of Medicine, Tokyo, Japan
| | - Tamaki Nara
- Department of Healthcare Innovation, Juntendo University Graduate School of Medicine, Tokyo, Japan
- Medical Technology Innovation Center, Juntendo University, Tokyo, Japan
- Clinical Research and Trial Center, Juntendo University, Tokyo, Japan
| | - Shuko Nojiri
- Department of Healthcare Innovation, Juntendo University Graduate School of Medicine, Tokyo, Japan
- Medical Technology Innovation Center, Juntendo University, Tokyo, Japan
- Clinical Research and Trial Center, Juntendo University, Tokyo, Japan
| | - Toshio Naito
- Department of General Medicine, Juntendo University Faculty of Medicine, Tokyo, Japan
| | - Makoto Hiki
- Department of Emergency and Disaster Medicine, Juntendo University Faculty of Medicine, Tokyo, Japan
- Department of Cardiovascular Biology and Medicine, Juntendo University Faculty of Medicine, Tokyo, Japan
| | - Masako Ichikawa
- Department of Emergency and Disaster Medicine, Juntendo University Faculty of Medicine, Tokyo, Japan
- Department of Respiratory Medicine, Juntendo University Graduate School of Medicine, Tokyo, Japan
| | - Yoshihide Takeshita
- Department of Psychiatry, Juntendo University Faculty of Medicine, Tokyo, Japan
| | - Tadafumi Kato
- Department of Psychiatry, Juntendo University Faculty of Medicine, Tokyo, Japan
| | | | - Morikuni Tobita
- Department of Healthcare Innovation, Juntendo University Graduate School of Medicine, Tokyo, Japan
- Medical Technology Innovation Center, Juntendo University, Tokyo, Japan
- Clinical Research and Trial Center, Juntendo University, Tokyo, Japan
| |
Collapse
|
26
|
Chen X, Gao J, Chen L, Khanna M, Gong B, Auffhammer M. The spatiotemporal pattern of surface ozone and its impact on agricultural productivity in China. PNAS NEXUS 2024; 3:pgad435. [PMID: 38152458 PMCID: PMC10752353 DOI: 10.1093/pnasnexus/pgad435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 12/04/2023] [Indexed: 12/29/2023]
Abstract
The slowing of agricultural productivity growth globally over the past two decades has brought a new urgency to detect its drivers and potential solutions. We show that air pollution, particularly surface ozone (O3), is strongly associated with declining agricultural total factor productivity (TFP) in China. We employ machine learning algorithms to generate estimates of high-resolution surface O3 concentrations from 2002 to 2019. Results indicate that China's O3 pollution has intensified over this 18-year period. We coupled these O3 estimates with a statistical model to show that rising O3 pollution during nonwinter seasons has reduced agricultural TFP by 18% over the 2002-2015 period. Agricultural TFP is projected to increase by 60% if surface O3 concentrations were reduced to meet the WHO air quality standards. This productivity gain has the potential to counter expected productivity losses from 2°C warming.
Collapse
Affiliation(s)
- Xiaoguang Chen
- Research Institute of Economics and Management, Southwestern University of Finance and Economics, 610074 Chengdu, China
| | - Jing Gao
- Research Institute of Economics and Management, Southwestern University of Finance and Economics, 610074 Chengdu, China
| | - Luoye Chen
- Carbon Neutrality and Climate Change Thrust, Society Hub, Hong Kong University of Science and Technology (Guangzhou), 511453 Guangzhou, China
| | - Madhu Khanna
- Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA
| | - Binlei Gong
- China Academy for Rural Development (CARD) and School of Public Affairs, Zhejiang University, 310025 Hangzhou, China
| | - Maximilian Auffhammer
- Department of Agricultural and Resource Economics, University of California, Berkeley, CA 94720, USA
- National Bureau of Economic Research, Cambridge, MA 02138, USA
| |
Collapse
|
27
|
Zhang Y, Xu W, Yang P, Zhang A. Machine learning for the prediction of sepsis-related death: a systematic review and meta-analysis. BMC Med Inform Decis Mak 2023; 23:283. [PMID: 38082381 PMCID: PMC10712076 DOI: 10.1186/s12911-023-02383-1] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 11/28/2023] [Indexed: 12/18/2023] Open
Abstract
BACKGROUND AND OBJECTIVES Sepsis is accompanied by a considerably high risk of mortality in the short term, despite the availability of recommended mortality risk assessment tools. However, these risk assessment tools seem to have limited predictive value. With the gradual integration of machine learning into clinical practice, some researchers have attempted to employ machine learning for early mortality risk prediction in sepsis patients. Nevertheless, there is a lack of comprehensive understanding regarding the construction of predictive variables using machine learning and the value of various machine learning methods. Thus, we carried out this systematic review and meta-analysis to explore the predictive value of machine learning for sepsis-related death at different time points. METHODS PubMed, Embase, Cochrane, and Web of Science databases were searched until August 9th, 2022. The risk of bias in predictive models was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). We also performed subgroup analysis according to time of death and type of model and summarized current predictive variables used to construct models for sepsis death prediction. RESULTS Fifty original studies were included, covering 104 models. The combined Concordance index (C-index), sensitivity, and specificity of machine learning models were 0.799, 0.81, and 0.80 in the training set, and 0.774, 0.71, and 0.68 in the validation set, respectively. Machine learning outperformed conventional clinical scoring tools and showed excellent C-index, sensitivity, and specificity in different subgroups. Random Forest (RF) and eXtreme Gradient Boosting (XGBoost) are the preferred machine learning models because they showed more favorable accuracy with similar modeling variables. This study found that lactate was the most frequent predictor but was seriously ignored by current clinical scoring tools. CONCLUSION Machine learning methods demonstrate relatively favorable accuracy in predicting the mortality risk in sepsis patients. Given the limitations in accuracy and applicability of existing prediction scoring systems, there is an opportunity to explore updates based on existing machine learning approaches. Specifically, it is essential to develop or update more suitable mortality risk assessment tools based on the specific contexts of use, such as emergency departments, general wards, and intensive care units.
Collapse
Affiliation(s)
- Yan Zhang
- Department of Critical Care Medicine, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, 400010, China
| | - Weiwei Xu
- Department of Endocrine and Metabolic Diseases, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, 400010, China
| | - Ping Yang
- Department of Critical Care Medicine, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, 400010, China.
| | - An Zhang
- Department of Critical Care Medicine, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, 400010, China.
| |
Collapse
|
28
|
Villar J, González-Martín JM, Hernández-González J, Armengol MA, Fernández C, Martín-Rodríguez C, Mosteiro F, Martínez D, Sánchez-Ballesteros J, Ferrando C, Domínguez-Berrot AM, Añón JM, Parra L, Montiel R, Solano R, Robaglia D, Rodríguez-Suárez P, Gómez-Bentolila E, Fernández RL, Szakmany T, Steyerberg EW, Slutsky AS. Predicting ICU Mortality in Acute Respiratory Distress Syndrome Patients Using Machine Learning: The Predicting Outcome and STratifiCation of severity in ARDS (POSTCARDS) Study. Crit Care Med 2023; 51:1638-1649. [PMID: 37651262 DOI: 10.1097/ccm.0000000000006030] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
Abstract
OBJECTIVES To assess the value of machine learning approaches in the development of a multivariable model for early prediction of ICU death in patients with acute respiratory distress syndrome (ARDS). DESIGN A development, testing, and external validation study using clinical data from four prospective, multicenter, observational cohorts. SETTING A network of multidisciplinary ICUs. PATIENTS A total of 1,303 patients with moderate-to-severe ARDS managed with lung-protective ventilation. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS We developed and tested prediction models in 1,000 ARDS patients. We performed logistic regression analysis following variable selection by a genetic algorithm, random forest and extreme gradient boosting machine learning techniques. Potential predictors included demographics, comorbidities, ventilatory and oxygenation descriptors, and extrapulmonary organ failures. Risk modeling identified some major prognostic factors for ICU mortality, including age, cancer, immunosuppression, Pa o2 /F io2 , inspiratory plateau pressure, and number of extrapulmonary organ failures. Together, these characteristics contained most of the prognostic information in the first 24 hours to predict ICU mortality. Performance with machine learning methods was similar to logistic regression (area under the receiver operating characteristic curve [AUC], 0.87; 95% CI, 0.82-0.91). External validation in an independent cohort of 303 ARDS patients confirmed that the performance of the model was similar to a logistic regression model (AUC, 0.91; 95% CI, 0.87-0.94). CONCLUSIONS Both machine learning and traditional methods lead to promising models to predict ICU death in moderate/severe ARDS patients. More research is needed to identify markers for severity beyond clinical determinants, such as demographics, comorbidities, lung mechanics, oxygenation, and extrapulmonary organ failure to guide patient management.
Collapse
Affiliation(s)
- Jesús Villar
- CIBER de Enfermedades Respiratorias, Instituto de Salud Carlos III, Madrid, Spain
- Research Unit, Hospital Universitario Dr. Negrín, Las Palmas de Gran Canaria, Spain
- Li Ka Shing Knowledge Institute at St. Michael's Hospital, Toronto, ON, Canada
| | - Jesús M González-Martín
- CIBER de Enfermedades Respiratorias, Instituto de Salud Carlos III, Madrid, Spain
- Research Unit, Hospital Universitario Dr. Negrín, Las Palmas de Gran Canaria, Spain
| | | | - Miguel A Armengol
- Big Data Department, PMC-FPS, Regional Ministry of Health and Consumer Affairs, Sevilla, Spain
| | - Cristina Fernández
- Research Unit, Hospital Universitario Dr. Negrín, Las Palmas de Gran Canaria, Spain
| | | | - Fernando Mosteiro
- Intensive Care Unit, Hospital Universitario de A Coruña, La Coruña, Spain
| | - Domingo Martínez
- Intensive Care Unit, Hospital Universitario Virgen de Arrixaca, Murcia, Spain
| | | | - Carlos Ferrando
- Surgical Intensive Care Unit, Department of Anesthesia, Hospital Clinic, IDIBAPS, Barcelona, Spain
| | | | - José M Añón
- Intensive Care Unit, Hospital Universitario La Paz, IdiPaz, Madrid, Spain
| | - Laura Parra
- Intensive Care Unit, Hospital Clínico Universitario de Valladolid, Valladolid, Spain
| | - Raquel Montiel
- Intensive Care Unit, Hospital Universitario NS de Candelaria, Santa Cruz de Tenerife, Spain
| | - Rosario Solano
- Intensive Care Unit, Hospital Virgen de La Luz, Cuenca, Spain
| | - Denis Robaglia
- Intensive Care Unit, Hospital Universitario Fundación Jiménez Díaz, Madrid, Spain
| | - Pedro Rodríguez-Suárez
- CIBER de Enfermedades Respiratorias, Instituto de Salud Carlos III, Madrid, Spain
- Thoracic Surgery, Hospital Universitario Dr. Negrín, Las Palmas de Gran Canaria, Spain
| | | | - Rosa L Fernández
- CIBER de Enfermedades Respiratorias, Instituto de Salud Carlos III, Madrid, Spain
- Research Unit, Hospital Universitario Dr. Negrín, Las Palmas de Gran Canaria, Spain
| | - Tamas Szakmany
- Department of Intensive Care Medicine & Anesthesia, Aneurin Bevan University Health Board, Newport, United Kingdom
- Cardiff University, Cardiff, United Kingdom
| | - Ewout W Steyerberg
- Department Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands
| | - Arthur S Slutsky
- Li Ka Shing Knowledge Institute at St. Michael's Hospital, Toronto, ON, Canada
- Division of Critical Care Medicine, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
29
|
Samad M, Angel M, Rinehart J, Kanomata Y, Baldi P, Cannesson M. Medical Informatics Operating Room Vitals and Events Repository (MOVER): a public-access operating room database. JAMIA Open 2023; 6:ooad084. [PMID: 37860605 PMCID: PMC10582520 DOI: 10.1093/jamiaopen/ooad084] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Revised: 08/18/2023] [Accepted: 10/06/2023] [Indexed: 10/21/2023] Open
Abstract
Objectives Artificial intelligence (AI) holds great promise for transforming the healthcare industry. However, despite its potential, AI is yet to see widespread deployment in clinical settings in significant part due to the lack of publicly available clinical data and the lack of transparency in the published AI algorithms. There are few clinical data repositories publicly accessible to researchers to train and test AI algorithms, and even fewer that contain specialized data from the perioperative setting. To address this gap, we present and release the Medical Informatics Operating Room Vitals and Events Repository (MOVER). Materials and Methods This first release of MOVER includes adult patients who underwent surgery at the University of California, Irvine Medical Center from 2015 to 2022. Data for patients who underwent surgery were captured from 2 different sources: High-fidelity physiological waveforms from all of the operating rooms were captured in real time and matched with electronic medical record data. Results MOVER includes data from 58 799 unique patients and 83 468 surgeries. MOVER is available for download at https://doi.org/10.24432/C5VS5G, it can be downloaded by anyone who signs a data usage agreement (DUA), to restrict traffic to legitimate researchers. Discussion To the best of our knowledge MOVER is the only freely available public data repository that contains electronic health record and high-fidelity physiological waveforms data for patients undergoing surgery. Conclusion MOVER is freely available to all researchers who sign a DUA, and we hope that it will accelerate the integration of AI into healthcare settings, ultimately leading to improved patient outcomes.
Collapse
Affiliation(s)
- Muntaha Samad
- Department of Computer Science, University of California, Irvine, Irvine, CA 92697, United States
- Institute for Genomics and Bioinformatics, University of California, Irvine, Irvine, CA 92697, United States
| | - Mirana Angel
- Department of Computer Science, University of California, Irvine, Irvine, CA 92697, United States
- Institute for Genomics and Bioinformatics, University of California, Irvine, Irvine, CA 92697, United States
| | - Joseph Rinehart
- Department of Anesthesiology & Perioperative Care, University of California, Irvine, Irvine, CA 92697, United States
| | - Yuzo Kanomata
- Department of Computer Science, University of California, Irvine, Irvine, CA 92697, United States
- Institute for Genomics and Bioinformatics, University of California, Irvine, Irvine, CA 92697, United States
| | - Pierre Baldi
- Department of Computer Science, University of California, Irvine, Irvine, CA 92697, United States
- Institute for Genomics and Bioinformatics, University of California, Irvine, Irvine, CA 92697, United States
| | - Maxime Cannesson
- Department of Anesthesiology & Perioperative Medicine, University of California, Los Angeles, Los Angeles, CA 90095, United States
| |
Collapse
|
30
|
Hydoub YM, Walker AP, Kirchoff RW, Alzu'bi HM, Chipi PY, Gerberi DJ, Burton MC, Murad MH, Dugani SB. Risk Prediction Models for Hospital Mortality in General Medical Patients: A Systematic Review. AMERICAN JOURNAL OF MEDICINE OPEN 2023; 10:100044. [PMID: 38090393 PMCID: PMC10715621 DOI: 10.1016/j.ajmo.2023.100044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/13/2022] [Revised: 03/20/2023] [Accepted: 05/27/2023] [Indexed: 07/20/2024]
Abstract
Objective To systematically review contemporary prediction models for hospital mortality developed or validated in general medical patients. Methods We screened articles in five databases, from January 1, 2010, through April 7, 2022, and the bibliography of articles selected for final inclusion. We assessed the quality for risk of bias and applicability using the Prediction Model Risk of Bias Assessment Tool (PROBAST) and extracted data using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist. Two investigators independently screened each article, assessed quality, and extracted data. Results From 20,424 unique articles, we identified 15 models in 8 studies across 10 countries. The studies included 280,793 general medical patients and 19,923 hospital deaths. Models included 7 early warning scores, 2 comorbidities indices, and 6 combination models. Ten models were studied in all general medical patients (general models) and 7 in general medical patients with infection (infection models). Of the 15 models, 13 were developed using logistic or Poisson regression and 2 using machine learning methods. Also, 4 of 15 models reported on handling of missing values. None of the infection models had high discrimination, whereas 4 of 10 general models had high discrimination (area under curve >0.8). Only 1 model appropriately assessed calibration. All models had high risk of bias; 4 of 10 general models and 5 of 7 infection models had low concern for applicability for general medical patients. Conclusion Mortality prediction models for general medical patients were sparse and differed in quality, applicability, and discrimination. These models require hospital-level validation and/or recalibration in general medical patients to guide mortality reduction interventions.
Collapse
Affiliation(s)
- Yousif M. Hydoub
- Division of Cardiology, Sheikh Shakhbout Medical City, Abu Dhabi, United Arab Emirates
| | - Andrew P. Walker
- Division of Hospital Internal Medicine, Mayo Clinic, Phoenix, Ariz
- Department of Critical Care Medicine, Mayo Clinic, Phoenix, Ariz
| | - Robert W. Kirchoff
- Division of Hospital Internal Medicine, Mayo Clinic, Phoenix, Ariz
- Division of Hospital Internal Medicine, Mayo Clinic, Rochester, Minn
| | | | - Patricia Y. Chipi
- Division of Hospital Internal Medicine, Mayo Clinic, Jacksonville, Fla
| | | | | | - M. Hassan Murad
- Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn
| | - Sagar B. Dugani
- Division of Hospital Internal Medicine, Mayo Clinic, Rochester, Minn
- Division of Health Care Delivery Research, Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minn
| |
Collapse
|
31
|
Hu W, Jin T, Pan Z, Xu H, Yu L, Chen T, Zhang W, Jiang H, Yang W, Xu J, Zhu F, Dai H. An interpretable ensemble learning model facilitates early risk stratification of ischemic stroke in intensive care unit: Development and external validation of ICU-ISPM. Comput Biol Med 2023; 166:107577. [PMID: 37852108 DOI: 10.1016/j.compbiomed.2023.107577] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 09/13/2023] [Accepted: 10/11/2023] [Indexed: 10/20/2023]
Abstract
Ischemic stroke (IS) is a common and severe condition that requires intensive care unit (ICU) admission, with high mortality and variable prognosis. Accurate and reliable predictive tools that enable early risk stratification can facilitate interventions to improve patient outcomes; however, such tools are currently lacking. In this study, we developed and validated novel ensemble learning models based on soft voting and stacking methods to predict in-hospital mortality from IS in the ICU using two public databases: MIMIC-IV and eICU-CRD. Additionally, we identified the key predictors of mortality and developed a user-friendly online prediction tool for clinical use. The soft voting ensemble model, named ICU-ISPM, achieved an AUROC of 0.861 (95% CI: 0.829-0.892) and 0.844 (95% CI: 0.819-0.869) in the internal and external test cohorts, respectively. It significantly outperformed the APACHE scoring system and was more robust than individual models. ICU-ISPM obtained the highest performance compared to other models in similar studies. Using the SHAP method, the model was interpretable, revealing that GCS score, age, and intubation were the most important predictors of mortality. This model also provided a risk stratification system that can effectively distinguish between low-, medium-, and high-risk patients. Therefore, the ICU-ISPM is an accurate, reliable, interpretable, and clinically applicable tool, which is expected to assist clinicians in stratifying IS patients by the risk of mortality and rationally allocating medical resources. Based on ICU-ISPM, an online risk prediction tool was further developed, which was freely available at: http://ispm.idrblab.cn/.
Collapse
Affiliation(s)
- Wei Hu
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China
| | - Tingting Jin
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China
| | - Ziqi Pan
- College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, 310058, China
| | - Huimin Xu
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China
| | - Lingyan Yu
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China
| | - Tingting Chen
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China
| | - Wei Zhang
- College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, 310058, China
| | - Huifang Jiang
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China
| | - Wenjun Yang
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China
| | - Junjun Xu
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China
| | - Feng Zhu
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China; College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, 310058, China.
| | - Haibin Dai
- Department of Pharmacy, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310009, China; Clinical Pharmacy Research Center, Zhejiang University School of Medicine, Hangzhou, 310009, China.
| |
Collapse
|
32
|
Lim SH, Kim MJ, Choi WH, Cheong JC, Kim JW, Lee KJ, Park JH. Explainable machine learning using perioperative serial laboratory results to predict postoperative mortality in patients with peritonitis-induced sepsis. Ann Surg Treat Res 2023; 105:237-244. [PMID: 37908377 PMCID: PMC10613826 DOI: 10.4174/astr.2023.105.4.237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 08/07/2023] [Accepted: 08/28/2023] [Indexed: 11/02/2023] Open
Abstract
Purpose Sepsis is one of the most common causes of death after surgery. Several conventional scoring systems have been developed to predict the outcome of sepsis; however, their predictive power is insufficient. The present study applies explainable machine-learning algorithms to improve the accuracy of predicting postoperative mortality in patients with sepsis caused by peritonitis. Methods We performed a retrospective analysis of data from demographic, clinical, and laboratory analyses, including the delta neutrophil index (DNI), WBC and neutrophil counts, and CRP level. Laboratory data were measured before surgery, 12-36 hours after surgery, and 60-84 hours after surgery. The primary study output was the probability of mortality. The areas under the receiver operating characteristic curves (AUCs) of several machine-learning algorithms using the Sequential Organ Failure Assessment (SOFA) and Simplified Acute Physiology Score (SAPS) 3 models were compared. 'SHapley Additive exPlanations' values were used to indicate the direction of the relationship between a variable and mortality. Results The CatBoost model yielded the highest AUC (0.933) for mortality compared to SAPS3 and SOFA (0.860 and 0.867, respectively). Increased DNI on day 3, septic shock, use of norepinephrine therapy, and increased international normalized ratio on day 3 had the greatest impact on the model's prediction of mortality. Conclusion Machine-learning algorithms increase the accuracy of predicting postoperative mortality in patients with sepsis caused by peritonitis.
Collapse
Affiliation(s)
- Seung Hee Lim
- Department of Surgery, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea
| | - Min Jeong Kim
- Department of Surgery, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea
| | - Won Hyuk Choi
- Department of Surgery, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea
| | - Jin Cheol Cheong
- Department of Surgery, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea
| | - Jong Wan Kim
- Department of Surgery, Dongtan Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea
| | - Kyung Joo Lee
- Department of Medical Informatics & Statistics, Kangdong Sacred Heart Hospital, Seoul, Korea
| | - Jun Ho Park
- Department of Surgery, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea
- Department of Medical Informatics & Statistics, Kangdong Sacred Heart Hospital, Seoul, Korea
| |
Collapse
|
33
|
Shikha A, Kasem A. The Development and Validation of Artificial Intelligence Pediatric Appendicitis Decision-Tree for Children 0 to 12 Years Old. Eur J Pediatr Surg 2023; 33:395-402. [PMID: 36113502 DOI: 10.1055/a-1946-0157] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
INTRODUCTION Diagnosing appendicitis in young children (0-12 years) still poses a special difficulty despite the advent of radiological investigations. Few scoring models have evolved and been applied worldwide, but with significant fluctuations in accuracy upon validation. AIM To utilize artificial intelligence (AI) techniques to develop and validate a diagnostic model based on clinical and laboratory parameters only (without imaging), in addition to prospective validation to confirm the findings. METHODS In Stage-I, observational data of children (0-12 years), referred for acute appendicitis (March 1, 2016-February 28, 2019, n = 166), was used for model development and evaluation using 10-fold cross-validation (XV) technique to simulate a prospective validation. In Stage-II, prospective validation of the model and the XV estimates were performed (March 1, 2019-November 30, 2021, n = 139). RESULTS The developed model, AI Pediatric Appendicitis Decision-tree (AiPAD), is both accurate and explainable, with an XV estimation of average accuracy to be 93.5% ± 5.8 (91.4% positive predictive value [PPV] and 94.8% negative predictive value [NPV]). Prospective validation revealed that the model was indeed accurate and close to the XV evaluations, with an overall accuracy of 97.1% (96.7% PPV and 97.4% NPV). CONCLUSION The AiPAD is validated, highly accurate, easy to comprehend, and offers an invaluable tool to use in diagnosing appendicitis in children without the need for imaging. Ultimately, this would lead to significant practical benefits, improved outcomes, and reduced costs.
Collapse
Affiliation(s)
- Anas Shikha
- Department of Pediatric Surgery, Raja Isteri Pengiran Anak Saleha (RIPAS) Hospital, Jalan Putera Al-Muhtadee Billah, Bandar Seri Begawan, Brunei
| | - Asem Kasem
- Computer Information Science, Higher Colleges of Technology, UAE
| |
Collapse
|
34
|
Khan SH, Perkins AJ, Fuchita M, Holler E, Ortiz D, Boustani M, Khan BA, Gao S. Development of a population-level prediction model for intensive care unit (ICU) survivorship and mortality in older adults: A population-based cohort study. Health Sci Rep 2023; 6:e1634. [PMID: 37867787 PMCID: PMC10587446 DOI: 10.1002/hsr2.1634] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2023] [Revised: 09/21/2023] [Accepted: 10/05/2023] [Indexed: 10/24/2023] Open
Abstract
Background and Aims Given the growing utilization of critical care services by an aging population, development of population-level risk models which predict intensive care unit (ICU) survivorship and mortality may offer advantages for researchers and health systems. Our objective was to develop a risk model for ICU survivorship and mortality among community dwelling older adults. Methods This was a population-based cohort study of 48,127 patients who were 50 years and older with at least one primary care visit between January 1, 2017, and December 31, 2017. We used electronic health record (EHR) data to identify variables predictive of ICU survivorship. Results ICU admission and mortality within 2 years after index primary care visit date were used to divide patients into three groups of "alive without ICU admission", "ICU survivors," and "death." Multinomial logistic regression was used to identify EHR predictive variables for the three patient outcomes. Cross-validation by randomly splitting the data into derivation and validation data sets (60:40 split) was used to identify predictor variables and validate model performance using area under the receiver operating characteristics (AUC) curve. In our overall sample, 92.2% of patients were alive without ICU admission, 6.2% were admitted to the ICU at least once and survived, and 1.6% died. Greater deciles of age over 50 years, diagnoses of chronic obstructive pulmonary disorder or chronic heart failure, and laboratory abnormalities in alkaline phosphatase, hematocrit, and albumin contributed highest risk score weights for mortality. Risk scores derived from the model discriminated between patients that died versus remained alive without ICU admission (AUC = 0.858), and between ICU survivors versus alive without ICU admission (AUC = 0.765). Conclusion Our risk scores provide a feasible and scalable tool for researchers and health systems to identify patient cohorts at increased risk for ICU admission and survivorship. Further studies are needed to prospectively validate the risk scores in other patient populations.
Collapse
Affiliation(s)
- Sikandar H. Khan
- Division of Pulmonary, Critical CareSleep and Occupational MedicineIndianapolisIndianaUSA
- Regenstrief InstituteIndiana University Center for Aging ResearchIndianapolisIndianaUSA
- Department of MedicineIndiana University School of MedicineIndianapolisIndianaUSA
| | - Anthony J. Perkins
- Department of Biostatistics and Health Data ScienceIndiana University School of MedicineIndianapolisIndianaUSA
| | - Mikita Fuchita
- Department of AnesthesiologyUniversity of Colorado Anschutz Medical CampusAuroraColoradoUSA
| | - Emma Holler
- Department of Epidemiology and BiostatisticsIndiana University School of Public HealthBloomingtonIndianaUSA
| | - Damaris Ortiz
- Department of SurgeryIndiana University School of MedicineIndianapolisIndianaUSA
| | - Malaz Boustani
- Center for Health Innovation and Implementation ScienceIndiana University School of MedicineIndianapolisIndianaUSA
| | - Babar A. Khan
- Division of Pulmonary, Critical CareSleep and Occupational MedicineIndianapolisIndianaUSA
- Regenstrief InstituteIndiana University Center for Aging ResearchIndianapolisIndianaUSA
- Department of MedicineIndiana University School of MedicineIndianapolisIndianaUSA
| | - Sujuan Gao
- Department of Biostatistics and Health Data ScienceIndiana University School of MedicineIndianapolisIndianaUSA
| |
Collapse
|
35
|
Atallah L, Nabian M, Brochini L, Amelung PJ. Machine Learning for Benchmarking Critical Care Outcomes. Healthc Inform Res 2023; 29:301-314. [PMID: 37964452 PMCID: PMC10651403 DOI: 10.4258/hir.2023.29.4.301] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 08/23/2023] [Accepted: 09/25/2023] [Indexed: 11/16/2023] Open
Abstract
OBJECTIVES Enhancing critical care efficacy involves evaluating and improving system functioning. Benchmarking, a retrospective comparison of results against standards, aids risk-adjusted assessment and helps healthcare providers identify areas for improvement based on observed and predicted outcomes. The last two decades have seen the development of several models using machine learning (ML) for clinical outcome prediction. ML is a field of artificial intelligence focused on creating algorithms that enable computers to learn from and make predictions or decisions based on data. This narrative review centers on key discoveries and outcomes to aid clinicians and researchers in selecting the optimal methodology for critical care benchmarking using ML. METHODS We used PubMed to search the literature from 2003 to 2023 regarding predictive models utilizing ML for mortality (592 articles), length of stay (143 articles), or mechanical ventilation (195 articles). We supplemented the PubMed search with Google Scholar, making sure relevant articles were included. Given the narrative style, papers in the cohort were manually curated for a comprehensive reader perspective. RESULTS Our report presents comparative results for benchmarked outcomes and emphasizes advancements in feature types, preprocessing, model selection, and validation. It showcases instances where ML effectively tackled critical care outcome-prediction challenges, including nonlinear relationships, class imbalances, missing data, and documentation variability, leading to enhanced results. CONCLUSIONS Although ML has provided novel tools to improve the benchmarking of critical care outcomes, areas that require further research include class imbalance, fairness, improved calibration, generalizability, and long-term validation of published models.
Collapse
Affiliation(s)
- Louis Atallah
- Clinical Integration and Insights, Philips, Cambridge, MA,
USA
| | - Mohsen Nabian
- Clinical Integration and Insights, Philips, Cambridge, MA,
USA
| | - Ludmila Brochini
- Clinical Integration and Insights, Philips, Eindhoven, The
Netherlands
| | | |
Collapse
|
36
|
Cheng TY, Yu-Chieh Ho S, Chien TW, Chou W. Global research trends in artificial intelligence for critical care with a focus on chord network charts: Bibliometric analysis. Medicine (Baltimore) 2023; 102:e35082. [PMID: 37746962 PMCID: PMC10519472 DOI: 10.1097/md.0000000000035082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/05/2023] [Revised: 08/14/2023] [Accepted: 08/15/2023] [Indexed: 09/26/2023] Open
Abstract
BACKGROUND The field of critical care-related artificial intelligence (AI) research is rapidly gaining interest. However, there is still a lack of comprehensive bibliometric studies that measure and analyze scientific publications on a global scale. Network charts have traditionally been used to highlight author collaborations and coword phenomena (ACCP). It is necessary to determine whether chord network charts (CNCs) can provide a better understanding of ACCP, thus requiring clarification. This study aimed to achieve 2 objectives: evaluate global research trends in AI in intensive care medicine on publication outputs, coauthorships between nations, citations, and co-occurrences of keywords; and demonstrate the use of CNCs for ACCP in bibliometric analysis. METHODS The web of science database was searched for a total of 1992 documents published between 2013 and 2022. The document type was limited to articles and article reviews, and titles and abstracts were screened for eligibility. The characteristics of the publications, including preferred journals, leading research countries, international collaborations, top institutions, and major keywords, were analyzed using the category-journal rank-authorship-L-index score and trend analysis. The 100 most highly cited articles are also listed in detail. RESULTS Between 2018 and 2022, there was a sharp increase in publications, which accounted for 92.8% (1849/1992) of all papers included in the study. The United States and China were responsible for nearly 50% (936/1992) of the total publications. The leading countries, institutes, departments, authors, and journals in terms of publications were the US, Massachusetts Gen Hosp (US), Medical School, Zhongheng Zhang (China), and Science Reports. The top 3 primary keywords denoting research hotspots for AI in critically ill patients were mortality, model, and intensive care unit, with mortality having the highest burst strength (4.49). The keywords risk and system showed the highest growth trend (0.98) in counts over the past 4 years. CONCLUSIONS This study provides valuable insights into the potential for ACCP and future research opportunities. For AI-based clinical research to become widely accepted in critical care practice, collaborative research efforts are necessary to strengthen the maturity and robustness of AI-driven models using CNCs for display.
Collapse
Affiliation(s)
- Teng-Yun Cheng
- Department of Emergency Medicine, Chi-Mei Medical Center, Liouying, Tainan, Taiwan
| | - Sam Yu-Chieh Ho
- Department of Emergency Medicine, Chi-Mei Medical Center, Tainan, Taiwan
| | - Tsair-Wei Chien
- Department of Medical Research, Chi-Mei Medical Center, Tainan, Taiwan
| | - Willy Chou
- Department of Physical Medicine and Rehabilitation, Chi Mei Medical Center, Jiali, Tainan, Taiwan
- Department of Physical Medicine and Rehabilitation, Chung San Medical University Hospital, Taichung, Taiwan
| |
Collapse
|
37
|
Phillips RV, van der Laan MJ, Lee H, Gruber S. Practical considerations for specifying a super learner. Int J Epidemiol 2023; 52:1276-1285. [PMID: 36905602 DOI: 10.1093/ije/dyad023] [Citation(s) in RCA: 34] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Accepted: 02/23/2023] [Indexed: 03/12/2023] Open
Abstract
Common tasks encountered in epidemiology, including disease incidence estimation and causal inference, rely on predictive modelling. Constructing a predictive model can be thought of as learning a prediction function (a function that takes as input covariate data and outputs a predicted value). Many strategies for learning prediction functions from data (learners) are available, from parametric regressions to machine learning algorithms. It can be challenging to choose a learner, as it is impossible to know in advance which one is the most suitable for a particular dataset and prediction task. The super learner (SL) is an algorithm that alleviates concerns over selecting the one 'right' learner by providing the freedom to consider many, such as those recommended by collaborators, used in related research or specified by subject-matter experts. Also known as stacking, SL is an entirely prespecified and flexible approach for predictive modelling. To ensure the SL is well specified for learning the desired prediction function, the analyst does need to make a few important choices. In this educational article, we provide step-by-step guidelines for making these decisions, walking the reader through each of them and providing intuition along the way. In doing so, we aim to empower the analyst to tailor the SL specification to their prediction task, thereby ensuring their SL performs as well as possible. A flowchart provides a concise, easy-to-follow summary of key suggestions and heuristics, based on our accumulated experience and guided by SL optimality theory.
Collapse
Affiliation(s)
- Rachael V Phillips
- Division of Biostatistics, School of Public Health, University of California at Berkeley, Berkeley, California, United States
| | - Mark J van der Laan
- Division of Biostatistics, School of Public Health, University of California at Berkeley, Berkeley, California, United States
| | - Hana Lee
- Office of Biostatistics, Center for Drug Evaluation and Research, United States Food and Drug Administration, Silver Spring, Maryland, United States
| | - Susan Gruber
- Putnam Data Sciences, LLC, Cambridge, Massachusetts, United States
| |
Collapse
|
38
|
Jeon ET, Lee HJ, Park TY, Jin KN, Ryu B, Lee HW, Kim DH. Machine learning-based prediction of in-ICU mortality in pneumonia patients. Sci Rep 2023; 13:11527. [PMID: 37460837 DOI: 10.1038/s41598-023-38765-8] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Accepted: 07/14/2023] [Indexed: 07/20/2023] Open
Abstract
Conventional severity-of-illness scoring systems have shown suboptimal performance for predicting in-intensive care unit (ICU) mortality in patients with severe pneumonia. This study aimed to develop and validate machine learning (ML) models for mortality prediction in patients with severe pneumonia. This retrospective study evaluated patients admitted to the ICU for severe pneumonia between January 2016 and December 2021. The predictive performance was analyzed by comparing the area under the receiver operating characteristic curve (AU-ROC) of ML models to that of conventional severity-of-illness scoring systems. Three ML models were evaluated: (1) logistic regression with L2 regularization, (2) gradient-boosted decision tree (LightGBM), and (3) multilayer perceptron (MLP). Among the 816 pneumonia patients included, 223 (27.3%) patients died. All ML models significantly outperformed the Simplified Acute Physiology Score II (AU-ROC: 0.650 [0.584-0.716] vs 0.820 [0.771-0.869] for logistic regression vs 0.827 [0.777-0.876] for LightGBM 0.838 [0.791-0.884] for MLP; P < 0.001). In the analysis for NRI, the LightGBM and MLP models showed superior reclassification compared with the logistic regression model in predicting in-ICU mortality in all length of stay in the ICU subgroups; all age subgroups; all subgroups with any APACHE II score, PaO2/FiO2 ratio < 200; all subgroups with or without history of respiratory disease; with or without history of CVA or dementia; treatment with mechanical ventilation, and use of inotropic agents. In conclusion, the ML models have excellent performance in predicting in-ICU mortality in patients with severe pneumonia. Moreover, this study highlights the potential advantages of selecting individual ML models for predicting in-ICU mortality in different subgroups.
Collapse
Affiliation(s)
- Eun-Tae Jeon
- Department of Radiology, Seoul National University College of Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, 5 Gil 20, Boramae-Road, Dongjak-gu, Seoul, South Korea
| | - Hyo Jin Lee
- Division of Respiratory and Critical Care, Department of Internal Medicine, Seoul National University College of Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, 5 Gil 20, Boramae-Road, Dongjak-gu, Seoul, South Korea
| | - Tae Yun Park
- Division of Respiratory and Critical Care, Department of Internal Medicine, Seoul National University College of Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, 5 Gil 20, Boramae-Road, Dongjak-gu, Seoul, South Korea
| | - Kwang Nam Jin
- Department of Radiology, Seoul National University College of Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, 5 Gil 20, Boramae-Road, Dongjak-gu, Seoul, South Korea
| | - Borim Ryu
- Center for Data Science, Biomedical Research Institute, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, South Korea
| | - Hyun Woo Lee
- Division of Respiratory and Critical Care, Department of Internal Medicine, Seoul National University College of Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, 5 Gil 20, Boramae-Road, Dongjak-gu, Seoul, South Korea.
| | - Dong Hyun Kim
- Department of Radiology, Seoul National University College of Medicine, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, 5 Gil 20, Boramae-Road, Dongjak-gu, Seoul, South Korea.
| |
Collapse
|
39
|
Khader F, Kather JN, Müller-Franzes G, Wang T, Han T, Tayebi Arasteh S, Hamesch K, Bressem K, Haarburger C, Stegmaier J, Kuhl C, Nebelung S, Truhn D. Medical transformer for multimodal survival prediction in intensive care: integration of imaging and non-imaging data. Sci Rep 2023; 13:10666. [PMID: 37393383 PMCID: PMC10314902 DOI: 10.1038/s41598-023-37835-1] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Accepted: 06/28/2023] [Indexed: 07/03/2023] Open
Abstract
When clinicians assess the prognosis of patients in intensive care, they take imaging and non-imaging data into account. In contrast, many traditional machine learning models rely on only one of these modalities, limiting their potential in medical applications. This work proposes and evaluates a transformer-based neural network as a novel AI architecture that integrates multimodal patient data, i.e., imaging data (chest radiographs) and non-imaging data (clinical data). We evaluate the performance of our model in a retrospective study with 6,125 patients in intensive care. We show that the combined model (area under the receiver operating characteristic curve [AUROC] of 0.863) is superior to the radiographs-only model (AUROC = 0.811, p < 0.001) and the clinical data-only model (AUROC = 0.785, p < 0.001) when tasked with predicting in-hospital survival per patient. Furthermore, we demonstrate that our proposed model is robust in cases where not all (clinical) data points are available.
Collapse
Affiliation(s)
- Firas Khader
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany.
| | - Jakob Nikolas Kather
- Department of Medicine III, University Hospital Aachen, Aachen, Germany
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
- Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| | - Gustav Müller-Franzes
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
| | - Tianci Wang
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
| | - Tianyu Han
- Physics of Molecular Imaging Systems, Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany
| | - Soroosh Tayebi Arasteh
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
| | - Karim Hamesch
- Department of Medicine III, University Hospital Aachen, Aachen, Germany
| | - Keno Bressem
- Department of Radiology, Charité-University Medicine Berlin, Berlin, Germany
| | | | - Johannes Stegmaier
- Institute of Imaging and Computer Vision, RWTH Aachen University, Aachen, Germany
| | - Christiane Kuhl
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
| | - Sven Nebelung
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
| | - Daniel Truhn
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
| |
Collapse
|
40
|
Park H, Yang J, Chun BC. Assessment of severity scoring systems for predicting mortality in critically ill patients receiving continuous renal replacement therapy. PLoS One 2023; 18:e0286246. [PMID: 37228073 DOI: 10.1371/journal.pone.0286246] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Accepted: 05/12/2023] [Indexed: 05/27/2023] Open
Abstract
The incidence of acute kidney injury (AKI) is increasing every year and many patients with AKI admitted to the intensive care unit (ICU) require continuous renal replacement therapy (CRRT). This study compared and analyzed severity scoring systems to assess their suitability in predicting mortality in critically ill patients receiving CRRT. Data from 612 patients receiving CRRT in four ICUs of the Korea University Medical Center between January 2016 and November 2018 were retrospectively collected. The mean age of all patients was 67.6 ± 14.8 years, and the proportion of males was 59.6%. The endpoints were in-hospital mortality and 7-day mortality from the day of CRRT initiation to the date of death. The Program to Improve Care in Acute Renal Disease (PICARD), Demirjian's, Acute Physiology and Chronic Health Evaluation (APACHE) II, Simplified Acute Physiology Score (SAPS) 3, Sequential Organ Failure Assessment (SOFA), Multiple Organ Dysfunction Score (MODS), and Liano's scores were used to predict mortality. The in-hospital and 7-day mortality rates in the study population were 72.7% and 45.1%, respectively. The area under the receiver operator characteristic curve (AUROC) revealed the highest discrimination ability for Demirjian's score (0.770), followed by Liano's score (0.728) and APACHE II (0.710). The AUROC curves for the SAPS 3, MODS, and PICARD were 0.671, 0.665, and 0.658, respectively. The AUROC of Demirjian's score was significantly higher than that of the other scores, except for Liano's score. The Hosmer-Lemeshow test on Demirjian's score showed a poor fit in our analysis; however, it was more acceptable than general severity scores. Kidney-specific severity scoring systems showed better performance in predicting mortality in critically ill patients receiving CRRT than general severity scoring systems.
Collapse
Affiliation(s)
- Hyunmyung Park
- Department of Epidemiology and Health Informatics, Graduate School of Public Health, Korea University, Seoul, Korea
| | - Jihyun Yang
- Department of Internal Medicine, Kangbuk Samsung Medical Center, Seoul, Korea
| | - Byung Chul Chun
- Department of Preventive Medicine, Korea University College of Medicine, Seoul, Korea
| |
Collapse
|
41
|
Montoya LM, van der Laan MJ, Luedtke AR, Skeem JL, Coyle JR, Petersen ML. The optimal dynamic treatment rule superlearner: considerations, performance, and application to criminal justice interventions. Int J Biostat 2023; 19:217-238. [PMID: 35708222 PMCID: PMC10238854 DOI: 10.1515/ijb-2020-0127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Accepted: 05/06/2022] [Indexed: 11/15/2022]
Abstract
The optimal dynamic treatment rule (ODTR) framework offers an approach for understanding which kinds of patients respond best to specific treatments - in other words, treatment effect heterogeneity. Recently, there has been a proliferation of methods for estimating the ODTR. One such method is an extension of the SuperLearner algorithm - an ensemble method to optimally combine candidate algorithms extensively used in prediction problems - to ODTRs. Following the ``causal roadmap," we causally and statistically define the ODTR and provide an introduction to estimating it using the ODTR SuperLearner. Additionally, we highlight practical choices when implementing the algorithm, including choice of candidate algorithms, metalearners to combine the candidates, and risk functions to select the best combination of algorithms. Using simulations, we illustrate how estimating the ODTR using this SuperLearner approach can uncover treatment effect heterogeneity more effectively than traditional approaches based on fitting a parametric regression of the outcome on the treatment, covariates and treatment-covariate interactions. We investigate the implications of choices in implementing an ODTR SuperLearner at various sample sizes. Our results show the advantages of: (1) including a combination of both flexible machine learning algorithms and simple parametric estimators in the library of candidate algorithms; (2) using an ensemble metalearner to combine candidates rather than selecting only the best-performing candidate; (3) using the mean outcome under the rule as a risk function. Finally, we apply the ODTR SuperLearner to the ``Interventions" study, an ongoing randomized controlled trial, to identify which justice-involved adults with mental illness benefit most from cognitive behavioral therapy to reduce criminal re-offending.
Collapse
Affiliation(s)
- Lina M. Montoya
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | | | | | - Jennifer L. Skeem
- School of Social Work and Goldman School of Public Policy, University of California Berkeley, Berkeley, USA
| | - Jeremy R. Coyle
- Division of Biostatistics, University of California Berkeley, Berkeley, USA
| | - Maya L. Petersen
- Divisions of Biostatistics and Epidemiology, University of California Berkeley, Berkeley, USA
| |
Collapse
|
42
|
Boeck L, Pargger H, Schellongowski P, Luyt CE, Maggiorini M, Jahn K, Muller G, Lötscher R, Bucher E, Cueni N, Staudinger T, Chastre J, Siegemund M, Tamm M, Stolz D. Subjective and objective survival prediction in mechanically ventilated critically ill patients: a prospective cohort study. Crit Care 2023; 27:150. [PMID: 37076881 PMCID: PMC10114386 DOI: 10.1186/s13054-023-04381-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2022] [Accepted: 02/21/2023] [Indexed: 04/21/2023] Open
Abstract
BACKGROUND ICU risk assessment tools, routinely used for predicting population outcomes, are not recommended for evaluating individual risk. The state of health of single patients is mostly subjectively assessed to inform relatives and presumably to decide on treatment decisions. However, little is known how subjective and objective survival estimates compare. METHODS We performed a prospective cohort study in mechanically ventilated critically ill patients across five European centres, assessed 62 objective markers and asked the clinical staff to subjectively estimate the probability of surviving 28 days. RESULTS Within the 961 included patients, we identified 27 single objective predictors for 28-day survival (73.8%) and pooled them into predictive groups. While patient characteristics and treatment models performed poorly, the disease and biomarker models had a moderate discriminative performance for predicting 28-day survival, which improved for predicting 1-year survival. Subjective estimates of nurses (c-statistic [95% CI] 0.74 [0.70-0.78]), junior physicians (0.78 [0.74-0.81]) and attending physicians (0.75 [0.72-0.79]) discriminated survivors from non-survivors at least as good as the combination of all objective predictors (c-statistic: 0.67-0.72). Unexpectedly, subjective estimates were insufficiently calibrated, overestimating death in high-risk patients by about 20% in absolute terms. Combining subjective and objective measures refined discrimination and reduced the overestimation of death. CONCLUSIONS Subjective survival estimates are simple, cheap and similarly discriminative as objective models; however, they overestimate death risking that live-saving therapies are withheld. Therefore, subjective survival estimates of individual patients should be compared with objective tools and interpreted with caution if not agreeing. Trial registration ISRCTN ISRCTN59376582 , retrospectively registered October 31st 2013.
Collapse
Affiliation(s)
- Lucas Boeck
- Department of Clinical Research, University Hospital Basel, Basel, Switzerland.
- Department of Biomedicine, University Basel, Basel, Switzerland.
| | - Hans Pargger
- Intensive Care Unit, Department of Acute Medicine, University Hospital Basel, Basel, Switzerland
| | | | - Charles-Edouard Luyt
- Médecine Intensive Réanimation, Institut de Cardiologie, Groupe Hospitalier Pitié-Salpêtrière, Assistance Publique-Hôpitaux de Paris, Sorbonne-Université, Paris, France
- UMRS 1166, INSERM, ICAN Institute of Cardiometabolism and Nutrition, Sorbonne Université, Paris, France
| | - Marco Maggiorini
- Department of Internal Medicine, Intensive Care Unit, University Hospital Zürich, Zürich, Switzerland
| | - Kathleen Jahn
- Department of Clinical Research, University Hospital Basel, Basel, Switzerland
| | - Grégoire Muller
- Médecine Intensive Réanimation, Institut de Cardiologie, Groupe Hospitalier Pitié-Salpêtrière, Assistance Publique-Hôpitaux de Paris, Sorbonne-Université, Paris, France
| | - Rene Lötscher
- Surgical and Medical Intensive Care Medicine, Kantonsspital Baselland, Liestal, Switzerland
| | - Evelyne Bucher
- Intensive Care Unit, Department of Acute Medicine, University Hospital Basel, Basel, Switzerland
| | - Nadine Cueni
- Intensive Care Unit, Department of Acute Medicine, University Hospital Basel, Basel, Switzerland
| | - Thomas Staudinger
- Department of Internal Medicine I, University Hospital Vienna, Vienna, Austria
| | - Jean Chastre
- Médecine Intensive Réanimation, Institut de Cardiologie, Groupe Hospitalier Pitié-Salpêtrière, Assistance Publique-Hôpitaux de Paris, Sorbonne-Université, Paris, France
| | - Martin Siegemund
- Department of Clinical Research, University Hospital Basel, Basel, Switzerland
- Intensive Care Unit, Department of Acute Medicine, University Hospital Basel, Basel, Switzerland
| | - Michael Tamm
- Department of Clinical Research, University Hospital Basel, Basel, Switzerland
| | - Daiana Stolz
- Department of Clinical Research, University Hospital Basel, Basel, Switzerland.
- Department of Pneumology, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany.
| |
Collapse
|
43
|
Huber M, Schober P, Petersen S, Luedi MM. Decision curve analysis confirms higher clinical utility of multi-domain versus single-domain prediction models in patients with open abdomen treatment for peritonitis. BMC Med Inform Decis Mak 2023; 23:63. [PMID: 37024840 PMCID: PMC10078078 DOI: 10.1186/s12911-023-02156-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Accepted: 03/17/2023] [Indexed: 04/08/2023] Open
Abstract
BACKGROUND Prediction modelling increasingly becomes an important risk assessment tool in perioperative systems approaches, e.g. in complex patients with open abdomen treatment for peritonitis. In this population, combining predictors from multiple medical domains (i.e. demographical, physiological and surgical variables) outperforms the prediction capabilities of single-domain prediction models. However, the benefit of these prediction models for clinical decision-making remains to be investigated. We therefore examined the clinical utility of mortality prediction models in patients suffering from peritonitis with a decision curve analysis. METHODS In this secondary analysis of a large dataset, a traditional logistic regression approach, three machine learning methods and a stacked ensemble were employed to examine the predictive capability of demographic, physiological and surgical variables in predicting mortality under open abdomen treatment for peritonitis. Calibration was examined with calibration belts and predictive performance was assessed with the area both under the receiver operating characteristic curve (AUROC) and under the precision recall curve (AUPRC) and with the Brier Score. Clinical utility of the prediction models was examined by means of a decision curve analysis (DCA) within a treatment threshold range of interest of 0-30%, where threshold probabilities are traditionally defined as the minimum probability of disease at which further intervention would be warranted. RESULTS Machine learning methods supported available evidence of a higher prediction performance of a multi- versus single-domain prediction models. Interestingly, their prediction performance was similar to a logistic regression model. The DCA demonstrated that the overall net benefit is largest for a multi-domain prediction model and that this benefit is larger compared to the default "treat all" strategy only for treatment threshold probabilities above about 10%. Importantly, the net benefit for low threshold probabilities is dominated by physiological predictors: surgical and demographics predictors provide only secondary decision-analytic benefit. CONCLUSIONS DCA provides a valuable tool to compare single-domain and multi-domain prediction models and demonstrates overall higher decision-analytic value of the latter. Importantly, DCA provides a means to clinically differentiate the risks associated with each of these domains in more depth than with traditional performance metrics and highlighted the importance of physiological predictors for conservative intervention strategies for low treatment thresholds. Further, machine learning methods did not add significant benefit either in prediction performance or decision-analytic utility compared to logistic regression in these data.
Collapse
Affiliation(s)
- Markus Huber
- Department of Anaesthesiology and Pain Medicine, Bern University Hospital, Inselspital, University of Bern, Freiburgstrasse 10, Bern, 3010, Switzerland.
| | - Patrick Schober
- Department of Anaesthesiology, Amsterdam University Medical Centres, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Sven Petersen
- Department of General and Visceral Surgery, Asklepios Hospital Altona, Hamburg, Germany
| | - Markus M Luedi
- Department of Anaesthesiology and Pain Medicine, Bern University Hospital, Inselspital, University of Bern, Freiburgstrasse 10, Bern, 3010, Switzerland
| |
Collapse
|
44
|
Ishii E, Nawa N, Hashimoto S, Shigemitsu H, Fujiwara T. Development, validation, and feature extraction of a deep learning model predicting in-hospital mortality using Japan's largest national ICU database: a validation framework for transparent clinical Artificial Intelligence (cAI) development. Anaesth Crit Care Pain Med 2023; 42:101167. [PMID: 36302489 DOI: 10.1016/j.accpm.2022.101167] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Revised: 09/01/2022] [Accepted: 09/28/2022] [Indexed: 02/01/2023]
Abstract
OBJECTIVE While clinical Artificial Intelligence (cAI) mortality prediction models and relevant studies have increased, limitations including the lack of external validation studies and inadequate model calibration leading to decreased overall accuracy have been observed. To combat this, we developed and evaluated a novel deep neural network (DNN) and a validation framework to promote transparent cAI development. METHODS Data from Japan's largest ICU database was used to develop the DNN model, predicting in-hospital mortality including ICU and post-ICU mortality by days since ICU discharge. The most important variables to the model were extracted with SHapley Additive exPlanations (SHAP) to examine the DNN's efficacy as well as develop models that were also externally validated. MAIN RESULTS The area under the receiver operating characteristic curve (AUC) for predicting ICU mortality was 0.94 [0.93-0.95], and 0.91 [0.90-0.92] for in-hospital mortality, ranging between 0.91-0.95 throughout one year since ICU discharge. An external validation using only the top 20 variables resulted with higher AUCs than traditional severity scores. CONCLUSIONS Our DNN model consistently generated AUCs between 0.91-0.95 regardless of days since ICU discharge. The 20 most important variables to our DNN, also generated higher AUCs than traditional severity scores regardless of days since ICU discharge. To our knowledge, this is the first study that predicts ICU and in-hospital mortality using cAI by post-ICU discharge days up to over a year. This finding could contribute to increased transparency on cAI applications.
Collapse
Affiliation(s)
- Euma Ishii
- Department of Global Health Promotion, Tokyo Medical and Dental University, Tokyo, Japan
| | - Nobutoshi Nawa
- Department of Medical Education Research and Development, Tokyo Medical and Dental University, Tokyo, Japan
| | - Satoru Hashimoto
- Department of Anesthesiology and Intensive Care Medicine, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Hidenobu Shigemitsu
- Institute of Global Affairs, Tokyo Medical and Dental University, Tokyo, Japan
| | - Takeo Fujiwara
- Department of Global Health Promotion, Tokyo Medical and Dental University, Tokyo, Japan.
| |
Collapse
|
45
|
Potty AG, Potty ASR, Maffulli N, Blumenschein LA, Ganta D, Mistovich RJ, Fuentes M, Denard PJ, Sethi PM, Shah AA, Gupta A. Approaching Artificial Intelligence in Orthopaedics: Predictive Analytics and Machine Learning to Prognosticate Arthroscopic Rotator Cuff Surgical Outcomes. J Clin Med 2023; 12:2369. [PMID: 36983368 PMCID: PMC10056706 DOI: 10.3390/jcm12062369] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/09/2023] [Accepted: 03/17/2023] [Indexed: 03/30/2023] Open
Abstract
Machine learning (ML) has not yet been used to identify factors predictive for post-operative functional outcomes following arthroscopic rotator cuff repair (ARCR). We propose a novel algorithm to predict ARCR outcomes using machine learning. This is a retrospective cohort study from a prospectively collected database. Data were collected from the Surgical Outcome System Global Registry (Arthrex, Naples, FL, USA). Pre-operative and 3-month, 6-month, and 12-month post-operative American Shoulder and Elbow Surgeons (ASES) scores were collected and used to develop a ML model. Pre-operative factors including demography, comorbidities, cuff tear, tissue quality, and fixation implants were fed to the ML model. The algorithm then produced an expected post-operative ASES score for each patient. The ML-produced scores were compared to actual scores using standard test-train machine learning principles. Overall, 631 patients who underwent shoulder arthroscopy from January 2011 to March 2020 met inclusion criteria for final analysis. A substantial number of the test dataset predictions using the XGBoost algorithm were within the minimal clinically important difference (MCID) and substantial clinical benefit (SCB) thresholds: 67% of the 12-month post-operative predictions were within MCID, while 84% were within SCB. Pre-operative ASES score, pre-operative pain score, body mass index (BMI), age, and tendon quality were the most important features in predicting patient recovery as identified using Shapley additive explanations (SHAP). In conclusion, the proposed novel machine learning algorithm can use pre-operative factors to predict post-operative ASES scores accurately. This can further supplement pre-operative counselling, planning, and resource allocation. Level of Evidence: III.
Collapse
Affiliation(s)
- Anish G. Potty
- South Texas Orthopedic Research Institute (STORI Inc.), Laredo, TX 78045, USA
- The Institute of Musculoskeletal Excellence (TIME Orthopaedics), Laredo, TX 78041, USA
- School of Osteopathic Medicine, The University of the Incarnate Word, San Antonio, TX 78209, USA
| | - Ajish S. R. Potty
- South Texas Orthopedic Research Institute (STORI Inc.), Laredo, TX 78045, USA
| | - Nicola Maffulli
- Department of Musculoskeletal Disorders, School of Medicine and Surgery, University of Salerno, 84084 Fisciano, Italy
- San Giovanni di Dio e Ruggi D’Aragona Hospital “Clinica Ortopedica” Department, Hospital of Salerno, 84124 Salerno, Italy
- Centre for Sports and Exercise Medicine, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London E1 4DG, UK
- School of Pharmacy and Bioengineering, Keele University School of Medicine, Stoke on Trent ST5 5BG, UK
| | - Lucas A. Blumenschein
- Department of Orthopaedics, School of Medicine, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Deepak Ganta
- School of Engineering, Texas A&M International University, Laredo, TX 78041, USA
| | - R. Justin Mistovich
- Department of Orthopaedics, School of Medicine, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Mario Fuentes
- School of Engineering, Texas A&M International University, Laredo, TX 78041, USA
| | | | - Paul M. Sethi
- Orthopaedic & Neurosurgery Specialists, Greenwich, CT 06905, USA
| | | | - Ashim Gupta
- South Texas Orthopedic Research Institute (STORI Inc.), Laredo, TX 78045, USA
- Future Biologics, Lawrenceville, GA 30043, USA
- BioIntegrate, Lawrenceville, GA 30043, USA
- Regenerative Orthopaedics, Noida 201301, Uttar Pradesh, India
| |
Collapse
|
46
|
Highly adaptive regression trees. EVOLUTIONARY INTELLIGENCE 2023. [DOI: 10.1007/s12065-023-00836-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/15/2023]
|
47
|
Viderman D, Abdildin YG, Batkuldinova K, Badenes R, Bilotta F. Artificial Intelligence in Resuscitation: A Scoping Review. J Clin Med 2023; 12:2254. [PMID: 36983255 PMCID: PMC10054374 DOI: 10.3390/jcm12062254] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 02/17/2023] [Accepted: 03/09/2023] [Indexed: 03/17/2023] Open
Abstract
INTRODUCTION Cardiac arrest is a significant cause of premature mortality and severe disability. Despite the death rate steadily decreasing over the previous decade, only 22% of survivors achieve good clinical status and only 25% of patients survive until their discharge from the hospital. The objective of this scoping review was to review relevant AI modalities and the main potential applications of AI in resuscitation. METHODS We conducted the literature search for related studies in PubMed, EMBASE, and Google Scholar. We included peer-reviewed publications and articles in the press, pooling and characterizing the data by their model types, goals, and benefits. RESULTS After identifying 268 original studies, we chose 59 original studies (reporting 1,817,419 patients) to include in the qualitative synthesis. AI-based methods appear to be superior to traditional methods in achieving high-level performance. CONCLUSION AI might be useful in predicting cardiac arrest, heart rhythm disorders, and post-cardiac arrest outcomes, as well as in the delivery of drone-delivered defibrillators and notification of dispatchers. AI-powered technologies could be valuable assistants to continuously track patient conditions. Healthcare professionals should assist in the research and development of AI-powered technologies as well as their implementation into clinical practice.
Collapse
Affiliation(s)
- Dmitriy Viderman
- Department of Surgery, Nazarbayev University School of Medicine (NUSOM), Kerei, Zhanibek khandar Str. 5/1, Astana 010000, Kazakhstan;
| | - Yerkin G. Abdildin
- Department of Mechanical and Aerospace Engineering, School of Engineering and Digital Sciences, Nazarbayev University, 53 Kabanbay Batyr Ave., Astana 010000, Kazakhstan
| | - Kamila Batkuldinova
- Department of Mechanical and Aerospace Engineering, School of Engineering and Digital Sciences, Nazarbayev University, 53 Kabanbay Batyr Ave., Astana 010000, Kazakhstan
| | - Rafael Badenes
- Department of Anaesthesiology and Intensive Care, Hospital Clìnico Universitario de Valencia, University of Valencia, 46001 Valencia, Spain
| | - Federico Bilotta
- Department of Anesthesia and Intensive Care, University La Sapienza, 00185 Rome, Italy
| |
Collapse
|
48
|
Targeted learning: Towards a future informed by real-world evidence. Stat Biopharm Res 2023. [DOI: 10.1080/19466315.2023.2182356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
|
49
|
Lu M, Zhang Y, Zhang S, Shi H, Huang Z. Knowledge-aware patient representation learning for multiple disease subtypes. J Biomed Inform 2023; 138:104292. [PMID: 36641030 DOI: 10.1016/j.jbi.2023.104292] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Accepted: 01/10/2023] [Indexed: 01/13/2023]
Abstract
Learning latent representations of patients with a target disease is a core problem in a broad range of downstream applications, such as clinical endpoint prediction. The suffering of patients may have multiple subtypes with certain similarities and differences, which need to be addressed for learning effective patient representation to facilitate the downstream tasks. However, existing studies either ignore the distinction of disease subtypes to learn disease-level representations, or neglect the correlations between subtypes and only learn disease subtype-level representations, which affects the performance of patient representation learning. To alleviate this problem, we studied how to effectively integrate data from all disease subtypes to improve the representation of each subtype. Specifically, we proposed a knowledge-aware shared-private neural network model to explicitly use disease-oriented knowledge and learn shared and specific representations from the disease and its subtype perspectives. To evaluate the feasibility of the proposed model, we conducted a particular downstream task, i.e., clinical endpoint prediction, on the basis of the learned patient presentations. The results on the real-world clinical datasets demonstrated that our model could yield a significant improvement over state-of-the-art models.
Collapse
Affiliation(s)
- Menglin Lu
- College of Computer Science and Technology, Zhejiang University, 866 Yuhangtang Road, 310058 Hangzhou, People's Republic of China.
| | - Yujie Zhang
- College of Computer Science and Technology, Zhejiang University, 866 Yuhangtang Road, 310058 Hangzhou, People's Republic of China.
| | - Suixia Zhang
- College of Computer Science and Technology, Zhejiang University, 866 Yuhangtang Road, 310058 Hangzhou, People's Republic of China.
| | - Hanrui Shi
- College of Computer Science and Technology, Zhejiang University, 866 Yuhangtang Road, 310058 Hangzhou, People's Republic of China.
| | - Zhengxing Huang
- College of Computer Science and Technology, Zhejiang University, 866 Yuhangtang Road, 310058 Hangzhou, People's Republic of China.
| |
Collapse
|
50
|
Yang W, Zou H, Wang M, Zhang Q, Li S, Liang H. Mortality prediction among ICU inpatients based on MIMIC-III database results from the conditional medical generative adversarial network. Heliyon 2023; 9:e13200. [PMID: 36798767 PMCID: PMC9925961 DOI: 10.1016/j.heliyon.2023.e13200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Revised: 01/18/2023] [Accepted: 01/19/2023] [Indexed: 01/26/2023] Open
Abstract
Background and aims Improved mortality prediction among intensive care unit (ICU) inpatients is a valuable and challenging task. Limited clinical data, especially with appropriate labels, are an important element restricting accurate predictions. Generative adversarial networks (GANs) are excellent generative models and have shown great potential for data simulation. However, there have been no relevant studies using GANs to predict mortality among ICU inpatients. In this study, we aim to evaluate the predictive performance of a variant of GAN called conditional medical GAN (c-med GAN) compared with some baseline models, including simplified acute physiology score II (SAPS II), support vector machine (SVM), and multilayer perceptron (MLP). Methods Data from a publicly available intensive care database, the Medical Information Mart for Intensive Care III (MIMIC-III) database (v1.4), were included in this study. The area under the precision-recall curve (PR-AUC), area under the receiver operating characteristic curve (ROC-AUC), and F1 score were used to evaluate the predictive performance. In addition, the size of the dataset was artificially reduced, and the performance of the c-med GAN was compared in different size datasets. Results The results showed that c-med GAN achieves the best PR-AUC, ROC-AUC, and F1 score compared with SAPS II, SVM, and MLP when training in the full MIMIC-III dataset. When the size of the dataset was reduced, the prediction performances of both MLP and c-med GAN were affected. However, the c-med GAN still outperformed MLP on smaller datasets and had less degradation. Conclusion The prediction of in-hospital mortality based on the c-med GAN for ICU patients showed better performance than the baseline models. Despite some inadequacies, this model may have a promising future in clinical applications which will be explored by further research.
Collapse
Affiliation(s)
- Wei Yang
- Department of Urology, The General Hospital of Western Theater Command (Chengdu Military General Hospital), Chengdu, 610083, China
| | - Hong Zou
- Department of General Surgery, The General Hospital of Western Theater Command (Chengdu Military General Hospital), Chengdu, 610083, China,Department of Liver Surgery & Liver Transplantation, State Key Laboratory of Biotherapy and Cancer Center, West China Hospital, Sichuan University and Collaborative Innovation Center of Biotherapy, Chengdu 610044, Sichuan Province, China
| | - Meng Wang
- Department of Traditional Chinese Medicine, The General Hospital of Western Theater Command (Chengdu Military General Hospital), Chengdu, 610083, China
| | - Qin Zhang
- Department of Gastroenterology, The 77th Army Hospital, Jiajiang, 614100, China
| | - Shadan Li
- Department of Urology, The General Hospital of Western Theater Command (Chengdu Military General Hospital), Chengdu, 610083, China
| | - Hongyin Liang
- Department of General Surgery, The General Hospital of Western Theater Command (Chengdu Military General Hospital), Chengdu, 610083, China,Corresponding author.
| |
Collapse
|