1
|
Jiang H, Ren S, Zhang S, Luo X, He R, Wang SF, Yan JD, Zhou S, Yin C, Xiao Y, Li Z. Analyzing factors influencing hospitalization costs for five common cancers in China using neural network models. J Med Econ 2025; 28:615-624. [PMID: 40241623 DOI: 10.1080/13696998.2025.2494459] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/11/2025] [Accepted: 04/14/2025] [Indexed: 04/18/2025]
Abstract
BACKGROUND Malignant tumors are a major global health crisis, causing 25% of deaths in China, with lung, liver, thyroid, breast, and colon cancers being the most common. Understanding the factors influencing hospitalization costs for these cancers is crucial for public health and economics. This study aimed to identify key cost factors and develop a neural network model for predicting hospitalization costs, thereby providing tools to ease the financial burden on patients and healthcare systems. METHODS Data on hospitalization costs for 30,893 cancer patients from secondary or higher-level hospitals in Zhuhai, Guangdong Province, between 2017 and 2022, were analyzed. Neural network classification and feature importance analysis were used to determine the main factors influencing costs and to develop predictive models. Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC), with a 95% confidence interval (CI) calculated for the AUROC value. RESULTS The key factors influencing hospitalization costs for lung cancer are metastasis and malignant solid tumor (MST), with correlation coefficients of 0.126 and 0.086, respectively, both showing statistical significance (p < 0.05). For colon cancer, the key factors influencing hospitalization costs are mortality and coronary disease (CD), with correlation coefficients of 0.092 and 0.090, respectively, both demonstrating statistical significance (p < 0.05). The AUROC value for the lung cancer model is 0.9078 (95% CI = 0.8975-0.9186), and the AUROC value for the colon cancer model is 0.9017 (95% CI = 0.8848-0.9196). CONCLUSION This study confirmed the strong clinical applicability of the neural network predictive model in analyzing hospitalization costs for lung and colon cancer and revealed the factors that influence hospitalization costs for these cancers.
Collapse
Affiliation(s)
- Hong Jiang
- Faculty of Medicine, Macau University of Science and Technology, Macau, China
- Statistical office, Zhuhai People's Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University), Zhuhai, China
| | - Sinuo Ren
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China
| | - Shengbo Zhang
- Department of General Surgery, Zhuhai People's Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University), Zhuhai, China
| | - Xudan Luo
- Department of Industrial Engineering and Operations Research, University of California, Berkeley, CA, USA
| | - Rui He
- Grammar and Cognition Lab, Department of Translation & Language Sciences, Universitat Pompeu Fabra, Barcelona, Spain
| | - Shuai Fei Wang
- Statistical office, Zhuhai People's Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University), Zhuhai, China
| | - Jian Dong Yan
- Statistical office, Zhuhai People's Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University), Zhuhai, China
| | - Shan Zhou
- Florida Research and Innovation Center, Cleveland Clinic, Port St. Lucie, FL, USA
| | - Chengliang Yin
- Faculty of Medicine, Macau University of Science and Technology, Macau, China
| | - Ying Xiao
- Faculty of Medicine, Macau University of Science and Technology, Macau, China
| | - Zhihuan Li
- Statistical office, Zhuhai People's Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University), Zhuhai, China
- China Resources Power Intelligent Security Laboratory, Research Institute of Tsinghua University in Shenzhen, Shenzhen, China
| |
Collapse
|
2
|
Cao F, Yang Y, Guo C, Zhang H, Yu Q, Guo J. Advancements in artificial intelligence for atopic dermatitis: diagnosis, treatment, and patient management. Ann Med 2025; 57:2484665. [PMID: 40200717 PMCID: PMC11983576 DOI: 10.1080/07853890.2025.2484665] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/09/2024] [Revised: 03/05/2025] [Accepted: 03/16/2025] [Indexed: 04/10/2025] Open
Abstract
Atopic dermatitis (AD) is a common and complex skin disease that significantly affects the quality of life of patients. The latest advances in artificial intelligence (AI) technology have introduced new methods for diagnosing, treating, and managing AD. AI has various innovative applications in the diagnosis and treatment of atopic dermatitis, with particular emphasis on its significant benefits in medical diagnosis, treatment monitoring, and patient care. AI algorithms, especially those that use deep learning techniques, demonstrate strong performance in recognizing skin images and effectively distinguishing different types of skin lesions, including common AD manifestations. In addition, artificial intelligence has also shown promise in creating personalized treatment plans, simplifying drug development processes, and managing clinical trials. Despite challenges in data privacy and model transparency, the potential of artificial intelligence in advancing AD care is enormous, bringing the future to precision medicine and improving patient outcomes. This manuscript provides a comprehensive review of the application of AI in the process of AD disease for the first time, aiming to play a key role in the advancement of AI in skin health care and further enhance the clinical diagnosis and treatment of AD.
Collapse
Affiliation(s)
- Fang Cao
- Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Yujie Yang
- Sinopharm Chongqing Southwest Aluminum Hospital, Beijing, China
| | - Cui Guo
- Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Hui Zhang
- Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Qianying Yu
- Hospital of Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Jing Guo
- Hospital of Chengdu University of Traditional Chinese Medicine, Chengdu, China
| |
Collapse
|
3
|
Drăgoi AL, Nemeș RM. “Electronic Pediatrician”, a non-machine learning prototype artificial intelligence software for pediatric computer-assisted pathophysiologic diagnosis ― general presentation. World J Methodol 2025; 15:100903. [DOI: 10.5662/wjm.v15.i3.100903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/05/2024] [Revised: 10/24/2024] [Accepted: 11/25/2024] [Indexed: 03/06/2025] Open
Abstract
BACKGROUND Knowledge-based systems (KBS) are software applications based on a knowledge database and an inference engine. Various experimental KBS for computer-assisted medical diagnosis and treatment were started to be used since 70s (VisualDx, GIDEON, DXPlain, CADUCEUS, Internist-I, Mycin etc.).
AIM To present in detail the “Electronic Pediatrician (EPed)”, a medical non-machine learning artificial intelligence (nml-AI) KBS in its prototype version created by the corresponding author (with database written in Romanian) that offers a physiopathology-based differential and positive diagnosis and treatment of ill children.
METHODS EPed specifically focuses on the physiopathological reasoning of pediatric clinical cases. EPed has currently reached its prototype version 2.0, being able to diagnose 302 physiopathological macro-links (briefly named “clusters”) and 269 pediatric diseases: Some examples of diagnosis and a previous testing of EPed on a group of 34 patients are also presented in this paper.
RESULTS The prototype EPed can currently diagnose 269 pediatric infectious and non-infectious diseases (based on 302 clusters), including the most frequent respiratory/digestive/renal/central nervous system infections, but also many other non-infectious pediatric diseases like autoimmune, oncological, genetical diseases and even intoxications, plus some important surgical pathologies.
CONCLUSION EPed is the first and only physiopathology-based nml-AI KBS focused on general pediatrics and is the first and only pediatric Romanian KBS addressed to medical professionals. Furthermore, EPed is the first and only nml-AI KBS that offers not only both a physiopathology-based differential and positive disease diagnosis, but also identifies possible physiopathological “clusters” that may explain the signs and symptoms of any child-patient and may help treating that patient physiopathologically (until a final diagnosis is found), thus encouraging and developing the physiopathological reasoning of any clinician.
Collapse
Affiliation(s)
- Andrei-Lucian Drăgoi
- Medical Doctoral School of University "Titu Maiorescu", Bucharest 040051, Romania
- The Emergency County Hospital Târgoviște (SJUT), Dambovita 130095, Târgoviște, Romania
| | - Roxana-Maria Nemeș
- Medical Doctoral School of University "Titu Maiorescu", Bucharest 040051, Romania
| |
Collapse
|
4
|
Wu J, Chen J, Zhang H, Luan Z, Zhao Y, Sun M, Wang S, Li C, Zhao Z, Zhang W, Chen Y, Zhang J, Li Y, Liu K, Niu J, Sun G. Guideline-driven clinical decision support for colonoscopy patients using the hierarchical multi-label deep learning method. Chin Med J (Engl) 2025:00029330-990000000-01560. [PMID: 40405345 DOI: 10.1097/cm9.0000000000003469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2024] [Indexed: 05/24/2025] Open
Abstract
BACKGROUND Over 20 million colonoscopies are performed in China annually. An automatic clinical decision support system (CDSS) with accurate semantic recognition of colonoscopy reports and guideline-based is helpful to relieve the increasing medical burden and standardize the healthcare. In this study, the CDSS was built under a hierarchical-label interpretable classification framework, trained by a state-of-the-art transformer-based model, and validated in a multi-center style. METHODS We conducted stratified sampling on a previously established dataset containing 302,965 electronic colonoscopy reports with pathology, identified 2041 records representative of overall features, and randomly divided into the training and testing sets (7:3). A total of 5 main labels and 22 sublabels were applied to annotate each record on a network platform, and the data were trained respectively by three pre-training models on Chinese corpus website, including BERT-base-Chinese (BC), the BERT-wwm-ext-Chinese (BWEC), and ernie-3.0-base-zh (E3BZ). The performance of trained models was subsequently compared with a randomly initialized model, and the preferred model was selected. Model fine-tuning was applied to further enhance the capacity. The system was validated in five other hospitals with 3177 consecutive colonoscopy cases. RESULTS The E3BZ pre-trained model exhibited the best performance, with a 90.18% accuracy and a 69.14% Macro-F1 score overall. The model achieved 100% accuracy in identifying cancer cases and 99.16% for normal cases. In external validation, the model exhibited favorable consistency and good performance among five hospitals. CONCLUSIONS The novel CDSS possesses high-level semantic recognition of colonoscopy reports, provides appropriate recommendations, and holds the potential to be a powerful tool for physicians and patients. The hierarchical multi-label strategy and pre-training method should be amendable to manage more medical text in the future.
Collapse
Affiliation(s)
- Junling Wu
- Medical School of Chinese PLA, Beijing 100853, China
- Department of Gastroenterology and Hepatology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Jun Chen
- Medical School of Chinese PLA, Beijing 100853, China
- Department of Gastroenterology and Hepatology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Hanwen Zhang
- Medical School of Chinese PLA, Beijing 100853, China
- Department of Gastroenterology and Hepatology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Zhe Luan
- Medical School of Chinese PLA, Beijing 100853, China
- Department of Gastroenterology and Hepatology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Yiming Zhao
- Department of Gastroenterology and Hepatology, Hainan Hospital of PLA General Hospital, Sanya, Hainan 572013, China
| | - Mengxuan Sun
- University of Chinese Academy of Sciences, Beijing 101408, China
- Institute of Automation Chinese Academy of Sciences, Beijing 100190, China
| | - Shufang Wang
- Department of Gastroenterology and Hepatology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Congyong Li
- Sixth Health Care Department, Second Medical Center of PLA General Hospital, Beijing 100853, China
| | - Zhizhuang Zhao
- Department of Geriatrics, Hainan Hospital of PLA General Hospital, Sanya, Hainan 572013, China
| | - Wei Zhang
- Medical School of Chinese PLA, Beijing 100853, China
- Department of Gastroenterology and Hepatology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Yi Chen
- Medical School of Chinese PLA, Beijing 100853, China
- Department of Gastroenterology and Hepatology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Jiaqi Zhang
- Medical School of Chinese PLA, Beijing 100853, China
- Department of Gastroenterology and Hepatology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Yansheng Li
- DHC Mediway Technology Co., Ltd, Beijing 100089, China
| | - Kejia Liu
- DHC Mediway Technology Co., Ltd, Beijing 100089, China
| | - Jinghao Niu
- Institute of Automation Chinese Academy of Sciences, Beijing 100190, China
| | - Gang Sun
- Department of Gastroenterology and Hepatology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| |
Collapse
|
5
|
Saparov A, Zech M. Big data and transformative bioinformatics in genomic diagnostics and beyond. Parkinsonism Relat Disord 2025; 134:107311. [PMID: 39924354 DOI: 10.1016/j.parkreldis.2025.107311] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/11/2024] [Revised: 01/23/2025] [Accepted: 01/25/2025] [Indexed: 02/11/2025]
Abstract
The current era of high-throughput analysis-driven research offers invaluable insights into disease etiologies, accurate diagnostics, pathogenesis, and personalized therapy. In the field of movement disorders, investigators are facing an increasing growth in the volume of produced patient-derived datasets, providing substantial opportunities for precision medicine approaches based on extensive information accessibility and advanced annotation practices. Integrating data from multiple sources, including phenomics, genomics, and multi-omics, is crucial for comprehensively understanding different types of movement disorders. Here, we explore formats and analytics of big data generated for patients with movement disorders, including strategies to meaningfully share the data for optimized patient benefit. We review computational methods that are essential to accelerate the process of evaluating the increasing amounts of specialized data collected. Based on concrete examples, we highlight how bioinformatic approaches facilitate the translation of multidimensional biological information into clinically relevant knowledge. Moreover, we outline the feasibility of computer-aided therapeutic target evaluation, and we discuss the importance of expanding the focus of big data research to understudied phenotypes such as dystonia.
Collapse
Affiliation(s)
- Alice Saparov
- Institute of Human Genetics, Technical University of Munich, School of Medicine and Health, Munich, Germany; Institute of Neurogenomics, Helmholtz Munich, Neuherberg, Germany; Institute for Advanced Study, Technical University of Munich, Garching, Germany
| | - Michael Zech
- Institute of Human Genetics, Technical University of Munich, School of Medicine and Health, Munich, Germany; Institute of Neurogenomics, Helmholtz Munich, Neuherberg, Germany; Institute for Advanced Study, Technical University of Munich, Garching, Germany.
| |
Collapse
|
6
|
Wang X, Yao K, Huang Z, Zhao W, Fu J, Lou P, Liu Y, Hu J, Li Y, Fang A, Chen W. An artificial intelligence malnutrition screening tool based on electronic medical records. Clin Nutr ESPEN 2025; 68:153-159. [PMID: 40311925 DOI: 10.1016/j.clnesp.2025.03.178] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2024] [Revised: 03/17/2025] [Accepted: 03/31/2025] [Indexed: 05/03/2025]
Abstract
BACKGROUND & AIMS Nutrition screening is a fundamental step to ensure appropriate intervention in patients with malnutrition. An automatic tool of nutritional risk screening based on electronic health records will improve efficiency and elevate the malnutrition diagnosis rate. We aimed to develop an artificial intelligence (AI) malnutrition screening tool based on electronic medical records and compare it with the patient interview-based tool. METHODS We conducted a cross-sectional study at a comprehensive tertiary hospital in China. Data of malnutrition information were extracted from electronic health records (EHR) and were used to train and test an AI tool for the malnutritional risk screening. We adopted the GLIM framework as a reference standard for assessing malnutrition. Six widely used machine learning algorithms for auxiliary diagnosis prediction, including Support Vector Machine, Random Forest, extreme gradient boosting (XGBoost), Logistic Regression, AdaBoost, and Gradient Boosting were compared and visualized using SHapley Additive exPlanations (SHAP). After feature screening, simplified algoritms were cross validated at an independent data set. RESULTS 495 inpatients enrolled were randomly divided into training and validation groups for algorithm development. 10 features annotated manually from free texts and 32 features selected from structured EHRs entered the models. XGBoost had the highest area under the receiver operating characteristic curve (AUC) and the top six features were weight loss, decreased food intake, prealbmine, white cell, BMI group, and percent of neutrophils. In simplified models, Random Forest acquired the highest AUC of 0.97 based on first sources data from interviews and 0.87 based on EHR data. CONCLUSIONS Inpatients' EHR data could be integrated by AI to detect the risk of malnutrition. This AI-enabled tool may hold promise for timely and efficient nutrition screening in newly admitted inpatients.
Collapse
Affiliation(s)
- Xue Wang
- Department of Clinical Nutrition, Department of Health Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100730, China; Department of Neurology, Zhongshan Hospital, Fudan University, Shanghai, 200032, China
| | - Kuanda Yao
- Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100020, China
| | - Zhicheng Huang
- Department of Thoracic Surgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100730, China
| | - Wanqing Zhao
- Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100020, China
| | - Jin Fu
- Department of Clinical Nutrition, Department of Health Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100730, China
| | - Pei Lou
- Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100020, China
| | - Yan Liu
- Department of Clinical Nutrition, Department of Health Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100730, China
| | - Jiahui Hu
- Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100020, China
| | - Yansheng Li
- iMEDWAY Technology Co., Ltd, Zhongguancun, Beijing, China
| | - An Fang
- Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100020, China.
| | - Wei Chen
- Department of Clinical Nutrition, Department of Health Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100730, China.
| |
Collapse
|
7
|
Ito H, Wada T, Ichinose G, Tanimoto J, Yoshimura J, Yamamoto T, Morita S. Barriers to the widespread adoption of diagnostic artificial intelligence for preventing antimicrobial resistance. Sci Rep 2025; 15:13113. [PMID: 40240443 PMCID: PMC12003763 DOI: 10.1038/s41598-025-95110-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2024] [Accepted: 03/19/2025] [Indexed: 04/18/2025] Open
Abstract
Currently, antimicrobial resistance (AMR) poses a major public health challenge. The emergence of AMR, which significantly threatens public health, is primarily due to the overuse of antimicrobial agents. This study explored the possibility that the ethical dilemmas inherent in the context of AMR may hinder the adoption of diagnostic artificial intelligence (AI). We conducted a web survey across eight countries/areas to assess public preference between two hypothetical AI types: one prioritizing individual health and the other considering the global AMR threat. Our results revealed a societal preference for the utilization of both AI types, reflecting a conflict between recognizing the significance of AMR and the desire for individualized treatment. Interestingly, the survey indicated significant gender and age differences in AI preferences, and the majority of respondents opposed the idea of AI standardization in treatment. These findings highlight the challenges of incorporating AI into public health and the necessity of considering public sentiment in addressing global health issues such as AMR.
Collapse
Affiliation(s)
- Hiromu Ito
- Department of International Health and Medical Anthropology, Institute of Tropical Medicine, Nagasaki University, Nagasaki, Japan.
| | - Takayuki Wada
- Graduate School of Human Life and Ecology, Osaka Metropolitan University, Osaka, Japan
- Osaka International Research Center for Infectious Diseases, Osaka Metropolitan University, Osaka, Japan
| | - Genki Ichinose
- Department of Mathematical and Systems Engineering, Shizuoka University, Shizuoka, Japan
| | - Jun Tanimoto
- Department of Energy and Environmental Engineering, Interdisciplinary Graduate School of Engineering Sciences, Kyushu University, Fukuoka, Japan
- Department of Advanced Environmental Science and Engineering, Faculty of Engineering Sciences, Kyushu University, Fukuoka, Japan
| | - Jin Yoshimura
- Department of International Health and Medical Anthropology, Institute of Tropical Medicine, Nagasaki University, Nagasaki, Japan
- Marine Biosystems Research Center, Chiba University, Chiba, Japan
- Department of Biological Science, Tokyo Metropolitan University, Tokyo, Japan
| | - Taro Yamamoto
- Department of International Health and Medical Anthropology, Institute of Tropical Medicine, Nagasaki University, Nagasaki, Japan
| | - Satoru Morita
- Department of Mathematical and Systems Engineering, Shizuoka University, Shizuoka, Japan
| |
Collapse
|
8
|
Liu F, Zhou H, Wang K, Yu Y, Gao Y, Sun Z, Liu S, Sun S, Zou Z, Li Z, Li B, Miao H, Liu Y, Hou T, Fok M, Patil NG, Xue K, Li T, Oermann E, Yin Y, Duan L, Qu J, Huang X, Jin S, Zhang K. MetaGP: A generative foundation model integrating electronic health records and multimodal imaging for addressing unmet clinical needs. Cell Rep Med 2025; 6:102056. [PMID: 40187356 PMCID: PMC12047458 DOI: 10.1016/j.xcrm.2025.102056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2024] [Revised: 01/17/2025] [Accepted: 03/11/2025] [Indexed: 04/07/2025]
Abstract
Artificial intelligence makes strides in specialized diagnostics but faces challenges in complex clinical scenarios, such as rare disease diagnosis and emergency condition identification. To address these limitations, we develop Meta General Practitioner (MetaGP), a 32-billion-parameter generative foundation model trained on extensive datasets, including over 8 million electronic health records, biomedical literature, and medical textbooks. MetaGP demonstrates robust diagnostic capabilities, achieving accuracy comparable to experienced clinicians. In rare disease cases, it achieves an average diagnostic score of 1.57, surpassing GPT-4's 0.93. For emergency conditions, it improves diagnostic accuracy for junior and mid-level clinicians by 53% and 46%, respectively. MetaGP also excels in generating medical imaging reports, producing high-quality outputs for chest X-rays and computed tomography, often rated comparable to or superior to physician-authored reports. These findings highlight MetaGP's potential to transform clinical decision-making across diverse medical contexts.
Collapse
Affiliation(s)
- Fei Liu
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China; State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China; State Key Laboratory of Quality Research in Chinese Medicine/Macau Institute for Applied Research in Medicine and Health, Macau University of Science and Technology, Macau SAR 999078, China
| | - Hongyu Zhou
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Kai Wang
- State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China; Department of Big Data and Biomedical AI, College of Future Technology, Peking University and Peking-Tsinghua Center for Life Sciences, Beijing, China
| | - Yunfang Yu
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China; Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Yuanxu Gao
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China; State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China; Department of Big Data and Biomedical AI, College of Future Technology, Peking University and Peking-Tsinghua Center for Life Sciences, Beijing, China
| | - Zhuo Sun
- State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; Department of Ophthalmology, The Third People's Hospital of Changzhou, Changzhou, China
| | - Sian Liu
- State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Shanshan Sun
- State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; Department of Ophthalmology, The Third People's Hospital of Changzhou, Changzhou, China
| | - Zixing Zou
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China; State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China; Guangzhou National Laboratory, Guangzhou, China
| | - Zhuomin Li
- State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Bingzhou Li
- State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Hanpei Miao
- Dongguan Hospital, Southern Medical University, Dongguan, China
| | - Yang Liu
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China
| | - Taiwa Hou
- Conde S. Januário Hospital, Macau, China
| | - Manson Fok
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China
| | - Nivritti Gajanan Patil
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China
| | - Kanmin Xue
- Nuffield Department of Neuroscience, Oxford University, Oxford, UK
| | - Ting Li
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China; State Key Laboratory of Quality Research in Chinese Medicine/Macau Institute for Applied Research in Medicine and Health, Macau University of Science and Technology, Macau SAR 999078, China
| | - Eric Oermann
- NYU Langone Medical Center, New York University, New York, NY, USA
| | - Yun Yin
- Faculty of Health and Wellness, Faculty of Business, City University of Macau, Macau, China
| | - Lian Duan
- Faculty of Pediatrics and Department of Pediatric Surgery of the Seventh Medical Center, the Chinese PLA General Hospital, Beijing, China.
| | - Jia Qu
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China.
| | - Xiaoying Huang
- Division of Pulmonary Medicine, the First Affiliated Hospital, Wenzhou Medical University, Wenzhou Key Laboratory of Interdisciplinary and Translational Medicine, Wenzhou Key Laboratory of Heart and Lung, Wenzhou, Zhejiang, China.
| | - Shengwei Jin
- Department of Anesthesia and Critical Care, The Second Affiliated Hospital and Yuying Children's Hospital of Wenzhou Medical University, Key Laboratory of Pediatric Anesthesiology, Ministry of Education, Wenzhou Medical University, Wenzhou, Zhejiang, China.
| | - Kang Zhang
- Institute for AI in Medicine and Faculty of Medicine, Macau University of Science and Technology, Macau, China; State Key Laboratory of Eye Health, Eye Hospital and Institute for Advanced Study on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China; Guangzhou National Laboratory, Guangzhou, China.
| |
Collapse
|
9
|
Borsetto D, Sia E, Axon P, Donnelly N, Tysome JR, Anschuetz L, Bernardeschi D, Capriotti V, Caye-Thomasen P, West NC, Erbele ID, Franchella S, Gatto A, Hess-Erga J, Kunst HPM, Marinelli JP, Mannion R, Panizza B, Trabalzini F, Obholzer R, Vaira LA, Polesel J, Giudici F, Carlson ML, Tirelli G, Boscolo-Rizzo P. Quality of Information Provided by Artificial Intelligence Chatbots Surrounding the Management of Vestibular Schwannomas: A Comparative Analysis Between ChatGPT-4 and Claude 2. Otol Neurotol 2025; 46:432-436. [PMID: 39965220 DOI: 10.1097/mao.0000000000004410] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/20/2025]
Abstract
OBJECTIVE To examine the quality of information provided by artificial intelligence platforms ChatGPT-4 and Claude 2 surrounding the management of vestibular schwannomas. STUDY DESIGN Cross-sectional. SETTING Skull base surgeons were involved from different centers and countries. INTERVENTION Thirty-six questions regarding vestibular schwannoma management were tested. Artificial intelligence responses were subsequently evaluated by 19 lateral skull base surgeons using the Quality Assessment of Medical Artificial Intelligence (QAMAI) questionnaire, assessing "Accuracy," "Clarity," "Relevance," "Completeness," "Sources," and "Usefulness." MAIN OUTCOME MEASURE The scores of the answers from both chatbots were collected and analyzed using the Student t test. Analysis of responses grouped by stakeholders was performed with McNemar test. Stuart-Maxwell test was used to compare reading level among chatbots. Intraclass correlation coefficient was calculated. RESULTS ChatGPT-4 demonstrated significantly improved quality over Claude 2 in 14 of 36 (38.9%) questions, whereas higher-quality scores for Claude 2 were only observed in 2 (5.6%) answers. Chatbots exhibited variation across the dimensions of "Accuracy," "Clarity," "Completeness," "Relevance," and "Usefulness," with ChatGPT-4 demonstrating a statistically significant superior performance. However, no statistically significant difference was found in the assessment of "Sources." Additionally, ChatGPT-4 provided information at a significant lower reading grade level. CONCLUSIONS Artificial intelligence platforms failed to consistently provide accurate information surrounding the management of vestibular schwannoma, although ChatGPT-4 achieved significantly higher scores in most analyzed parameters. These findings demonstrate the potential for significant misinformation for patients seeking information through these platforms.
Collapse
Affiliation(s)
- Daniele Borsetto
- Department of Otolaryngology-Head & Neck Surgery, Cambridge University Hospitals NHS Trust, Cambridge, UK
| | - Egidio Sia
- Department of Medical, Surgical and Health Sciences, Section of Otolaryngology, University of Trieste, 34149, Trieste, Italy
| | - Patrick Axon
- Department of Otolaryngology-Head & Neck Surgery, Cambridge University Hospitals NHS Trust, Cambridge, UK
| | - Neil Donnelly
- Department of Otolaryngology-Head & Neck Surgery, Cambridge University Hospitals NHS Trust, Cambridge, UK
| | - James R Tysome
- Department of Otolaryngology-Head & Neck Surgery, Cambridge University Hospitals NHS Trust, Cambridge, UK
| | - Lukas Anschuetz
- Department of Otorhinolaryngology, Head and Neck Surgery, Inselspital, University Hospital and University of Bern, Bern, Switzerland
| | - Daniele Bernardeschi
- AP-HP6, GHU Pitié-Salpêtrière, Service ORL, Unité Fonctionnelle Implants Auditifs et explorations fonctionnelles, Paris, France
| | - Vincenzo Capriotti
- Department of Otorhinolaryngology, Portogruaro Hospital, UlSS4 Veneto Orientale, Portogruaro, VE, Italy
| | - Per Caye-Thomasen
- Department of Otorhinolaryngology, Head and Neck Surgery, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark
| | - Niels Cramer West
- Department of Otorhinolaryngology, Head and Neck Surgery, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark
| | - Isaac D Erbele
- Department of Otolaryngology-Head and Neck Surgery, Brooke Army Medical Center, Fort Sam Houston, Texas, USA
| | - Sebastiano Franchella
- Section of Otorhinolaryngology-Head and Neck Surgery, Department of Neurosciences, University of Padova, 35128 Padova, Italy
| | - Annalisa Gatto
- Department of Medical, Surgical and Health Sciences, Section of Otolaryngology, University of Trieste, 34149, Trieste, Italy
| | - Jeanette Hess-Erga
- Department of Otorhinolaryngology, Head and Neck Surgery, Haukeland University Hospital, 5021 Bergen, Norway
| | | | - John P Marinelli
- Department of Otolaryngology-Head and Neck Surgery, Mayo Clinic, Rochester, Minnesota, USA
| | - Richard Mannion
- Department of Neurosurgery, Addenbrooke's Hospital, Cambridge University Hospitals, Cambridge, UK
| | - Benedict Panizza
- Department of Otolaryngology-Head & Neck surgery, Princess Alexandra Hospital, Brisbane, QLD, Australia
| | - Franco Trabalzini
- Department of Otolaryngology, Ospedale Pediatrico Meyer, Firenze, Italy
| | - Rupert Obholzer
- Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, UK
| | - Luigi Angelo Vaira
- Maxillofacial Surgery Operative Unit, Department of Medicine, Surgery and Pharmacy, University of Sassari, Sassari, Italy
| | - Jerry Polesel
- Unit of Cancer Epidemiology, Centro di Riferimento Oncologico di Aviano (CRO) IRCCS, Aviano, Italy
| | - Fabiola Giudici
- Unit of Cancer Epidemiology, Centro di Riferimento Oncologico di Aviano (CRO) IRCCS, Aviano, Italy
| | - Matthew L Carlson
- Department of Otolaryngology-Head and Neck Surgery, Mayo Clinic, Rochester, Minnesota, USA
| | - Giancarlo Tirelli
- Department of Medical, Surgical and Health Sciences, Section of Otolaryngology, University of Trieste, 34149, Trieste, Italy
| | - Paolo Boscolo-Rizzo
- Department of Medical, Surgical and Health Sciences, Section of Otolaryngology, University of Trieste, 34149, Trieste, Italy
| |
Collapse
|
10
|
Ren Y, Kang YN, Cao SY, Meng F, Zhang J, Liao R, Li X, Chen Y, Wen Y, Wu J, Xia W, Xu L, Wen S, Liu H, Li Y, Gu J, Lv Q. Evaluating the performance of large language models in health education for patients with ankylosing spondylitis/spondyloarthritis: a cross-sectional, single-blind study in China. BMJ Open 2025; 15:e097528. [PMID: 40118477 PMCID: PMC11931893 DOI: 10.1136/bmjopen-2024-097528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/04/2024] [Accepted: 02/28/2025] [Indexed: 03/23/2025] Open
Abstract
OBJECTIVES To evaluate the potential of large language models (LLMs) in health education for patients with ankylosing spondylitis (AS)/spondyloarthritis (SpA), focusing on the accuracy of information transmission, patient acceptance and performance differences between different models. DESIGN Cross-sectional, single-blind study. SETTING Multiple centres in China. PARTICIPANTS 182 volunteers, including 4 rheumatologists and 178 patients with AS/SpA. PRIMARY AND SECONDARY OUTCOME MEASURES Scientificity, precision and accessibility of the content of the answers provided by LLMs; patient acceptance of the answers. RESULTS LLMs performed well in terms of scientificity, precision and accessibility, with ChatGPT-4o and Kimi models outperforming traditional guidelines. Most patients with AS/SpA showed a higher level of understanding and acceptance of the responses from LLMs. CONCLUSIONS LLMs have significant potential in medical knowledge transmission and patient education, making them promising tools for future medical practice.
Collapse
Affiliation(s)
- Yong Ren
- Pazhou Lab, Guangzhou, Guangdong, China
- The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Yue-Ning Kang
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Shuang-Yan Cao
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Fanxuan Meng
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Jingyu Zhang
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Ruyi Liao
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Xiaomin Li
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Yuling Chen
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Ya Wen
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Jiayun Wu
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Wenqi Xia
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Liling Xu
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Shenghui Wen
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Huifen Liu
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Yuanqing Li
- School of Automation Science and Engineering, South China University of Technology, Guangzhou, China
- Research Center for Brain-Computer Interface, Pazhou Lab, Guangzhou, Guangdong, China
| | - Jieruo Gu
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
- Department of Rheumatology, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Qing Lv
- Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China
| |
Collapse
|
11
|
Du Z, Liu Z, Fu L, Wang C, Sun Z, Zhu L, Deng K. Interpretable personalized surgical recommendation with joint consideration of multiple decisional dimensions. NPJ Digit Med 2025; 8:168. [PMID: 40108342 PMCID: PMC11923295 DOI: 10.1038/s41746-025-01509-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2024] [Accepted: 02/09/2025] [Indexed: 03/22/2025] Open
Abstract
Surgical planning can be highly complicated and personalized, where a surgeon needs to balance multiple decisional dimensions including surgical effectiveness, risk, cost, and patient's conditions and preferences. Turning to artificial intelligence is a great appeal. This study filled in this gap with Multi-Dimensional Recommendation (MUDI), an interpretable data-driven intelligent system that supported personalized surgical recommendations on both the patient's and the surgeon's side with joint consideration of multiple decisional dimensions. Applied to Pelvic Organ Prolapse, a common female disease with significant impacts on life quality, MUDI stood out from a crowd of competing methods and achieved excellent performance that was comparable to top urogynecologists, with a transparent process that made communications between surgeons and patients easier. Users showed a willingness to accept the recommendations and achieved higher accuracy with the aid of MUDI. Such a success indicated that MUDI had the potential to solve similar challenges in other situations.
Collapse
Affiliation(s)
- Zhe Du
- Department of Obstetrics and Gynecology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, National Clinical Research Center for Obstetric & Gynecologic Diseases, Beijing, 100730, China
| | - Zhaoyang Liu
- Department of Statistics & Data Science, Tsinghua University, Beijing, 100084, China
| | - Linru Fu
- Department of Obstetrics and Gynecology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, National Clinical Research Center for Obstetric & Gynecologic Diseases, Beijing, 100730, China
| | - Che Wang
- Department of Statistics & Data Science, Tsinghua University, Beijing, 100084, China
| | - Zhijing Sun
- Department of Obstetrics and Gynecology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, National Clinical Research Center for Obstetric & Gynecologic Diseases, Beijing, 100730, China.
| | - Lan Zhu
- Department of Obstetrics and Gynecology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, National Clinical Research Center for Obstetric & Gynecologic Diseases, Beijing, 100730, China.
| | - Ke Deng
- Department of Statistics & Data Science, Tsinghua University, Beijing, 100084, China.
| |
Collapse
|
12
|
Ren G, Wang P, Wang Z, Xie Z, Liu L, Wang Y, Wu X. Automated detection of cervical spondylotic myelopathy: harnessing the power of natural language processing. Front Neurosci 2025; 19:1421792. [PMID: 40177375 PMCID: PMC11962790 DOI: 10.3389/fnins.2025.1421792] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2024] [Accepted: 03/03/2025] [Indexed: 04/05/2025] Open
Abstract
Background The objective of this study was to develop machine learning (ML) algorithms utilizing natural language processing (NLP) techniques for the automated detection of cervical spondylotic myelopathy (CSM) through the analysis of positive symptoms in free-text admission notes. This approach enables the timely identification and management of CSM, leading to optimal outcomes. Methods The dataset consisted of 1,214 patients diagnosed with cervical diseases as their primary condition between June 2013 and June 2020. A random ratio of 7:3 was employed to partition the dataset into training and testing subsets. Two machine learning models, Extreme Gradient Boosting (XGBoost) and Bidirectional Long Short Term Memory Network (LSTM), were developed. The performance of these models was assessed using various metrics, including the Receiver Operating Characteristic (ROC) curve, Area Under the Curve (AUC), accuracy, precision, recall, and F1 score. Results In the testing set, the LSTM achieved an AUC of 0.9025, an accuracy of 0.8740, a recall of 0.9560, an F1 score of 0.9122, and a precision of 0.8723. The LSTM model demonstrated superior clinical applicability compared to the XGBoost model, as evidenced by calibration curves and decision curve analysis. Conclusions The timely identification of suspected CSM allows for prompt confirmation of diagnosis and treatment. The utilization of NLP algorithm demonstrated excellent discriminatory capabilities in identifying CSM based on positive symptoms in free-text admission notes complaint data. This study showcases the potential of a pre-diagnosis system in the field of spine.
Collapse
Affiliation(s)
- GuanRui Ren
- Department of Orthopedics, Zhongda Hospital, Medical College, Southeast University, Nanjing, Jiangsu, China
| | - PeiYang Wang
- Department of Spine Surgery, Zhongda Hospital, Medical College, Southeast University, Nanjing, Jiangsu, China
| | - ZhiWei Wang
- Department of Orthopedics, Zhongda Hospital, Medical College, Southeast University, Nanjing, Jiangsu, China
| | - ZhiYang Xie
- Department of Spine Surgery, Zhongda Hospital, Medical College, Southeast University, Nanjing, Jiangsu, China
| | - Lei Liu
- Department of Spine Surgery, Zhongda Hospital, Medical College, Southeast University, Nanjing, Jiangsu, China
| | - YunTao Wang
- Department of Spine Surgery, Zhongda Hospital, Medical College, Southeast University, Nanjing, Jiangsu, China
- Xuyi County People's Hospital, Huai'an, Jiangsu, China
| | - XiaoTao Wu
- Department of Spine Surgery, Zhongda Hospital, Medical College, Southeast University, Nanjing, Jiangsu, China
| |
Collapse
|
13
|
Liu X, Liu H, Yang G, Jiang Z, Cui S, Zhang Z, Wang H, Tao L, Sun Y, Song Z, Hong T, Yang J, Gao T, Zhang J, Li X, Zhang J, Sang Y, Yang Z, Xue K, Wu S, Zhang P, Yang J, Song C, Wang G. A generalist medical language model for disease diagnosis assistance. Nat Med 2025; 31:932-942. [PMID: 39779927 DOI: 10.1038/s41591-024-03416-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Accepted: 11/12/2024] [Indexed: 01/11/2025]
Abstract
The delivery of accurate diagnoses is crucial in healthcare and represents the gateway to appropriate and timely treatment. Although recent large language models (LLMs) have demonstrated impressive capabilities in few-shot or zero-shot learning, their effectiveness in clinical diagnosis remains unproven. Here we present MedFound, a generalist medical language model with 176 billion parameters, pre-trained on a large-scale corpus derived from diverse medical text and real-world clinical records. We further fine-tuned MedFound to learn physicians' inferential diagnosis with a self-bootstrapping strategy-based chain-of-thought approach and introduced a unified preference alignment framework to align it with standard clinical practice. Extensive experiments demonstrate that our medical LLM outperforms other baseline LLMs and specialized models in in-distribution (common diseases), out-of-distribution (external validation) and long-tailed distribution (rare diseases) scenarios across eight specialties. Further ablation studies indicate the effectiveness of key components in our medical LLM training approach. We conducted a comprehensive evaluation of the clinical applicability of LLMs for diagnosis involving artificial intelligence (AI) versus physician comparison, AI-assistance study and human evaluation framework. Our proposed framework incorporates eight clinical evaluation metrics, covering capabilities such as medical record summarization, diagnostic reasoning and risk management. Our findings demonstrate the model's feasibility in assisting physicians with disease diagnosis as part of the clinical workflow.
Collapse
Affiliation(s)
- Xiaohong Liu
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Hao Liu
- Department of Orthopedics, Peking University Third Hospital & Beijing Key Laboratory of Spinal Disease & Engineering Research Center of Bone and Joint Precision Medicine, Beijing, China
| | - Guoxing Yang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Zeyu Jiang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Shuguang Cui
- School of Science and Engineering (SSE), Future Network of Intelligence Institute (FNii) and Guangdong Provincial Key Laboratory of Future Networks of Intelligence, Chinese University of Hong Kong, Shenzhen, China
| | - Zhaoze Zhang
- Department of Orthopedics, Peking University Third Hospital & Beijing Key Laboratory of Spinal Disease & Engineering Research Center of Bone and Joint Precision Medicine, Beijing, China
| | - Huan Wang
- Department of Orthopedics, Peking University Third Hospital & Beijing Key Laboratory of Spinal Disease & Engineering Research Center of Bone and Joint Precision Medicine, Beijing, China
| | - Liyuan Tao
- Research Center of Clinical Epidemiology, Peking University Third Hospital, Beijing, China
| | - Yongchang Sun
- Department of Respiratory and Critical Care Medicine, Peking University Third Hospital and Research Center for Chronic Airway Diseases, Peking University Health Science Center, Beijing, China
| | - Zhu Song
- Department of Respiratory and Critical Care Medicine, Peking University Third Hospital and Research Center for Chronic Airway Diseases, Peking University Health Science Center, Beijing, China
| | - Tianpei Hong
- Department of Endocrinology and Metabolism, Peking University Third Hospital, Beijing, China
| | - Jin Yang
- Department of Endocrinology and Metabolism, Peking University Third Hospital, Beijing, China
| | - Tianrun Gao
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Jiangjiang Zhang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Xiaohu Li
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Jing Zhang
- Department of Cardiology, The First College of Clinical Medical Science, China Three Gorges University and Yichang Central People's Hospital, Yichang, China
| | - Ye Sang
- Department of Cardiology, The First College of Clinical Medical Science, China Three Gorges University and Yichang Central People's Hospital, Yichang, China
| | - Zhao Yang
- Peking University First Hospital and Research Center of Public Policy, Peking University, Beijing, China
| | - Kanmin Xue
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Song Wu
- South China Hospital, Medical School, Shenzhen University, Shenzhen, China
| | - Ping Zhang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Jian Yang
- Department of Cardiology, The First College of Clinical Medical Science, China Three Gorges University and Yichang Central People's Hospital, Yichang, China.
| | - Chunli Song
- Department of Orthopedics, Peking University Third Hospital & Beijing Key Laboratory of Spinal Disease & Engineering Research Center of Bone and Joint Precision Medicine, Beijing, China.
| | - Guangyu Wang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China.
| |
Collapse
|
14
|
Medela A, Sabater A, Montilla IH, MacCarthy T, Aguilar A, Chiesa-Estomba CM. The utility and reliability of a deep learning algorithm as a diagnosis support tool in head & neck non-melanoma skin malignancies. Eur Arch Otorhinolaryngol 2025; 282:1585-1592. [PMID: 39242415 DOI: 10.1007/s00405-024-08951-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2024] [Accepted: 08/27/2024] [Indexed: 09/09/2024]
Abstract
OBJECTIVE The incidence of non-melanoma skin cancers, encompassing basal cell carcinoma (BCC) and cutaneous squamous cell carcinoma (cSCC), is on the rise globally and new methods to improve skin malignancy diagnosis are necessary. This study aims to assess the performance of a CE-certified medical device as a diagnosis support tool in a head & neck (H&N) outpatient clinic, specifically focusing on the classification of three key diagnostics: BCC, cSCC, and non-malignant lesions (such as Actinic Cheilitis, Actinic Keratosis, and Seborrheic Keratosis). METHODS a prospective, longitudinal, non-randomized study was designed to evaluate the performance of a deep learning-based method as a diagnosis tool in a group of patients referred to the head & neck clinic for suspicious skin lesions. RESULTS 135 patients were included, 92 (68.1%) were male and 43 (31.9%) were female. The median age was 71 years +/- 9 (Min: 56/Max: 91). Of those, 108 were malignant pathologies (54 basal cell carcinoma and 54 squamous cell carcinoma) and 27 benign pathologies (14 seborrheic keratoses, 2 actinic keratoses, and 11 actinic cheilitis). Of special significance is the remarkable performance of the medical device in identifying malignant lesions (basal cell carcinoma and squamous cell carcinoma) within the top-5 most likely diagnoses in above 90% of cases, underscoring its potential utility for early diagnosis and treatment. CONCLUSION In this study, the effectiveness of deep learning methods, with a particular focus on vision transformers, as a diagnostic aid for H&N cutaneous non-melanoma skin cancers was demonstrated, highlighting its potential value for early detection and treatment of non-melanoma skin cancers. In this vein, further research is needed in the future to elucidate the role of this technology, because of its potential in the primary care clinic, dermatology, and head & neck surgery clinic as well as in patients with suspicious lesions, as a self-exploration tool.
Collapse
Affiliation(s)
| | | | | | - Taig MacCarthy
- Clinical Endpoint Innovation, Legit.Health, Bilbao, Spain
| | - Andy Aguilar
- Clinical Endpoint Innovation, Legit.Health, Bilbao, Spain
| | - Carlos Miguel Chiesa-Estomba
- Department of Otorhinolaryngology-Head & Neck Surgery, Osakidetza, Donostia University Hospital, San Sebastian, 20014, Spain.
- Bioguipuzkoa Health Research Institute, San Sebastian, 20014, Spain.
- Faculty of Medicine, Deusto University, Bilbao, Spain.
- Servicio de Otorrinolaringología - Cirugía de Cabeza y Cuello, Hospital Universitario Donostia, Paseo Dr. Begiristain #1. CP, San Sebastian- Donosti, 20014, España.
| |
Collapse
|
15
|
Ma Y, Yang Y, Du Y, Jin L, Liang B, Zhang Y, Wang Y, Liu L, Zhang Z, Jin Z, Qiu Z, Ye M, Wang Z, Tong C. Development of an artificial intelligence-based multimodal diagnostic system for early detection of biliary atresia. BMC Med 2025; 23:127. [PMID: 40016769 PMCID: PMC11866655 DOI: 10.1186/s12916-025-03962-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/28/2024] [Accepted: 02/20/2025] [Indexed: 03/01/2025] Open
Abstract
BACKGROUND Early diagnosis of biliary atresia (BA) is crucial for improving patient outcomes, yet remains a significant global challenge. This challenge may be ameliorated through the application of artificial intelligence (AI). Despite the promise of AI in medical diagnostics, its application to multimodal BA data has not yet achieved substantial breakthroughs. This study aims to leverage diverse data sources and formats to develop an intelligent diagnostic system for BA. METHODS We constructed the largest known multimodal BA dataset, comprising ultrasound images, clinical data, and laboratory results. Using this dataset, we developed a novel deep learning model and simplified it using easily obtainable data, eliminating the need for blood samples. The models were externally validated in a prospective study. We compared the performance of our model with human experts of varying expertise levels and evaluated the AI system's potential to enhance its diagnostic accuracy. RESULTS The retrospective study included 1579 participants. The multimodal model achieved an AUC of 0.9870 on the internal test set, outperforming human experts. The simplified model yielded an AUC of 0.9799. In the prospective study's external test set of 171 cases, the multimodal model achieved an AUC of 0.9740, comparable to that of a radiologist with over 10 years of experience (AUC = 0.9766). For less experienced radiologists, the AI-assisted diagnostic AUC improved from 0.6667 to 0.9006. CONCLUSIONS This AI-based screening application effectively facilitates early diagnosis of BA and serves as a valuable reference for addressing common challenges in rare diseases. The model's high accuracy and its ability to enhance the diagnostic performance of human experts underscore its potential for significant clinical impact.
Collapse
Affiliation(s)
- Ya Ma
- Department of Ultrasound, Capital Institute of Pediatrics, Beijing, China
| | - Yuancheng Yang
- School of Computer Science and Engineering, Beihang University, Beijing, China
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
| | - Yuxin Du
- School of Computer Science and Engineering, Beihang University, Beijing, China
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
| | - Luyang Jin
- School of Computer Science and Engineering, Beihang University, Beijing, China
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
| | - Baoyu Liang
- School of Computer Science and Engineering, Beihang University, Beijing, China
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
| | - Yuqi Zhang
- School of Computer Science and Engineering, Beihang University, Beijing, China
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
| | - Yedi Wang
- Department of Ultrasound, Capital Institute of Pediatrics, Beijing, China
| | - Luyu Liu
- Department of Ultrasound, Capital Institute of Pediatrics, Beijing, China
| | - Zijian Zhang
- Department of Ultrasound, Capital Institute of Pediatrics, Beijing, China
| | - Zelong Jin
- Department of Ultrasound, Capital Institute of Pediatrics, Beijing, China
| | - Zhimin Qiu
- Department of Ultrasound, Capital Institute of Pediatrics, Beijing, China
| | - Mao Ye
- Department of General Surgery, Capital Institute of Pediatrics, Beijing, China
| | - Zhengrong Wang
- Department of Ultrasound, Capital Institute of Pediatrics, Beijing, China.
| | - Chao Tong
- School of Computer Science and Engineering, Beihang University, Beijing, China.
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China.
| |
Collapse
|
16
|
Chundi R, G S, Basivi PK, Tippana A, Hulipalled VR, N P, Simha JB, Kim CW, Kakani V, Pasupuleti VR. Exploring diabetes through the lens of AI and computer vision: Methods and future prospects. Comput Biol Med 2025; 185:109537. [PMID: 39672014 DOI: 10.1016/j.compbiomed.2024.109537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2024] [Revised: 08/03/2024] [Accepted: 12/04/2024] [Indexed: 12/15/2024]
Abstract
Early diagnosis and timely initiation of treatment plans for diabetes are crucial for ensuring individuals' well-being. Emerging technologies like artificial intelligence (AI) and computer vision are highly regarded for their ability to enhance the accessibility of large datasets for dynamic training and deliver efficient real-time intelligent technologies and predictable models. The application of AI and computer vision techniques to enhance the analysis of clinical data is referred to as eHealth solutions that employ advanced approaches to aid medical applications. This study examines several advancements and applications of machine learning, deep learning, and machine vision in global perception, with a focus on sustainability. This article discusses the significance of utilizing artificial intelligence and computer vision to detect diabetes, as it has the potential to significantly mitigate harm to human life. This paper provides several comments addressing challenges and recommendations for the use of this technology in the field of diabetes. This study explores the potential of employing Industry 4.0 technologies, including machine learning, deep learning, and computer vision robotics, as effective tools for effectively dealing with diabetes related aspects.
Collapse
Affiliation(s)
- Ramesh Chundi
- School of Computer Applications, Dayananda Sagar University, Bangalore, India
| | - Sasikala G
- School of Computer Science and Applications, REVA University, Rukmini Knowledge Park, Bangalore 560064, India
| | - Praveen Kumar Basivi
- Pukyong National University Industry-University Cooperation Foundation, Pukyong National University, Busan 48513, Republic of Korea
| | - Anitha Tippana
- Department of Biotechnology, School of Applied Sciences, REVA University, Rukmini Knowledge Park, Bangalore 560064, India
| | - Vishwanath R Hulipalled
- School of Computing and Information Technology, REVA University, Rukmini Knowledge Park, Bangalore 560064, India
| | - Prabakaran N
- School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, 632014, Tamilnadu, India
| | - Jay B Simha
- Abiba Systems, CTO, and RACE Labs, REVA University, Rukmini Knowledge Park, Bangalore 560064, India
| | - Chang Woo Kim
- Department of Nanotechnology Engineering, College of Engineering, Pukyong National University, Busan 48513, Republic of Korea
| | - Vijay Kakani
- Integrated System Engineering, Inha University, 100 Inha-ro, Nam-gu, 22212, Incheon, Republic of Korea.
| | - Visweswara Rao Pasupuleti
- Department of Biotechnology, School of Applied Sciences, REVA University, Rukmini Knowledge Park, Bangalore 560064, India; School of Biosciences, Taylor's University, Lakeside Campus, 47500, Subang Jaya, Selangor, Malaysia; Faculty of Earth Sciences, Universiti Malaysia Kelantan, Campus Jeli, Kelantan, 17600 Jeli, Malaysia.
| |
Collapse
|
17
|
Aljawarneh SA, Al-Quraan R. Pneumonia Detection Using Enhanced Convolutional Neural Network Model on Chest X-Ray Images. BIG DATA 2025; 13:16-29. [PMID: 37074075 DOI: 10.1089/big.2022.0261] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
Pneumonia, caused by microorganisms, is a severely contagious disease that damages one or both the lungs of the patients. Early detection and treatment are typically favored to recover infected patients since untreated pneumonia can lead to major complications in the elderly (>65 years) and children (<5 years). The objectives of this work are to develop several models to evaluate big X-ray images (XRIs) of the chest, to determine whether the images show/do not show signs of pneumonia, and to compare the models based on their accuracy, precision, recall, loss, and receiver operating characteristic area under the ROC curve scores. Enhanced convolutional neural network (CNN), VGG-19, ResNet-50, and ResNet-50 with fine-tuning are some of the deep learning (DL) algorithms employed in this study. By training the transfer learning model and enhanced CNN model using a big data set, these techniques are used to identify pneumonia. The data set for the study was obtained from Kaggle. It should be noted that the data set has been expanded to include further records. This data set included 5863 chest XRIs, which were categorized into 3 different folders (i.e., train, val, test). These data are produced every day from personnel records and Internet of Medical Things devices. According to the experimental findings, the ResNet-50 model showed the lowest accuracy, that is, 82.8%, while the enhanced CNN model showed the highest accuracy of 92.4%. Owing to its high accuracy, enhanced CNN was regarded as the best model in this study. The techniques developed in this study outperformed the popular ensemble techniques, and the models showed better results than those generated by cutting-edge methods. Our study implication is that a DL models can detect the progression of pneumonia, which improves the general diagnostic accuracy and gives patients new hope for speedy treatment. Since enhanced CNN and ResNet-50 showed the highest accuracy compared with other algorithms, it was concluded that these techniques could be effectively used to identify pneumonia after performing fine-tuning.
Collapse
Affiliation(s)
| | - Romesaa Al-Quraan
- CIS, CIT, Jordan University of Science and Technology, Irbid, Jordan
| |
Collapse
|
18
|
Tian H, Zhang K, Zhang J, Shi J, Qiu H, Hou N, Han F, Kan C, Sun X. Revolutionizing public health through digital health technology. PSYCHOL HEALTH MED 2025:1-16. [PMID: 39864819 DOI: 10.1080/13548506.2025.2458254] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/25/2024] [Accepted: 01/20/2025] [Indexed: 01/28/2025]
Abstract
The aging population and increasing chronic diseases strain public health systems. Advancements in digital health promise to tackle these challenges and enhance public health outcomes. Digital health integrates digital health technology (DHT) across healthcare, including smart consumer devices. This article examines the application of DHT in public health and its significant impact on revolutionizing the field. Historically, DHT has not only enhanced the efficiency of disease prevention, diagnosis, and treatment but also facilitated the equitable distribution of global health resources. Looking ahead, DHT holds vast potential in areas such as personalized medicine, telemedicine, and intelligent health management. However, it also encounters challenges such as ethics, privacy, and data security. To further advance DHT, concerted efforts are essential, including policy support, investment in research and development, involvement of medical institutions, and improvement of public digital health literacy.
Collapse
Affiliation(s)
- Hongzhan Tian
- Department of Endocrinology and Metabolism, Clinical Research Center, Affiliated Hospital of Shandong Second Medical University, Weifang, China
| | - Kexin Zhang
- Department of Endocrinology and Metabolism, Clinical Research Center, Affiliated Hospital of Shandong Second Medical University, Weifang, China
| | - Jingwen Zhang
- Department of Endocrinology and Metabolism, Clinical Research Center, Affiliated Hospital of Shandong Second Medical University, Weifang, China
| | - Junfeng Shi
- Department of Endocrinology and Metabolism, Clinical Research Center, Affiliated Hospital of Shandong Second Medical University, Weifang, China
| | - Hongyan Qiu
- Department of Endocrinology and Metabolism, Clinical Research Center, Affiliated Hospital of Shandong Second Medical University, Weifang, China
| | - Ningning Hou
- Department of Endocrinology and Metabolism, Clinical Research Center, Affiliated Hospital of Shandong Second Medical University, Weifang, China
| | - Fang Han
- Department of Pathology, Affiliated Hospital of Shandong Second Medical University, Weifang, China
| | - Chengxia Kan
- Department of Endocrinology and Metabolism, Clinical Research Center, Affiliated Hospital of Shandong Second Medical University, Weifang, China
| | - Xiaodong Sun
- Department of Endocrinology and Metabolism, Clinical Research Center, Affiliated Hospital of Shandong Second Medical University, Weifang, China
| |
Collapse
|
19
|
Indrio F, Masciari E, Marchese F, Rinaldi M, Maffei G, Gangai I, Grillo A, De Benedetto R, Napolitano EV, Beghetti I, Corvaglia L, Di Mauro A, Aceti A. Functional gastrointestinal disorders predictors in neonates and toddlers: A machine learning approach to risk assessment. Heliyon 2025; 11:e41516. [PMID: 39834435 PMCID: PMC11743318 DOI: 10.1016/j.heliyon.2024.e41516] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2024] [Revised: 12/16/2024] [Accepted: 12/25/2024] [Indexed: 01/22/2025] Open
Abstract
Background Functional Gastrointestinal Disorders (FGIDs) can pose a great burden on affected children, their families, and the healthcare system. Due to the lack of knowledge about the precise pathophysiology of FGIDs, a proper identification of children at risk to develop FGIDs has never been attempted. The research aims to identify early-life risk factors for FGIDs such as infantile colic, regurgitation, and functional constipation, within the first year of life. Methods This prospective observational cohort study enrolled both term and preterm infants from a tertiary care university hospital between January 1, 2020, and December 31, 2022. The study employed both traditional statistical methods and artificial intelligence (AI) techniques, specifically a random forest classification model, to identify key risk factors associated with the development of FGIDs. Based on these findings, an AI-based predictive model will be developed, along with a user-friendly, web-based interface designed for practical risk assessment. Results 6060 infants were enrolled. 8.1 % were born preterm. According to random forest classification model by AI, birth weight (BW), cord blood pH, and maternal age were the most relevant variables linked to development of FGIDs in the first year of life. Some discrepancies between potential risk factors identified through conventional statistics and AI were detected. Conclusion For the first time machine learning allowed to identify BW, cord blood pH and maternal age as important variable for risk prediction of FGIDs in the first year of life. This practical risk assessment tool would help clinicians to identify infants at risk of FGIDs who would benefit from a tailored preventive approach.
Collapse
Affiliation(s)
- Flavia Indrio
- Department of Experimental Medicine School of Medicine University of Salento, Lecce, Italy
| | - Elio Masciari
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione, University Federico II, Naples, Italy
| | - Flavia Marchese
- Department of Medical and Surgical Science Pediatric Section, University of Foggia, 71100, Foggia, Italy
| | - Matteo Rinaldi
- Department of Neonatology and NICU, Ospedali Riuniti Foggia, Viale Pinto 1, 71122, Foggia, Italy
| | - Gianfranco Maffei
- Department of Neonatology and NICU, Ospedali Riuniti Foggia, Viale Pinto 1, 71122, Foggia, Italy
| | - Ilaria Gangai
- Department of Medical and Surgical Science Pediatric Section, University of Foggia, 71100, Foggia, Italy
| | - Assunta Grillo
- Department of Medical and Surgical Science Pediatric Section, University of Foggia, 71100, Foggia, Italy
| | - Roberta De Benedetto
- Department of Medical and Surgical Science Pediatric Section, University of Foggia, 71100, Foggia, Italy
| | - Enea Vincenzo Napolitano
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione, University Federico II, Naples, Italy
| | - Isadora Beghetti
- Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy
| | - Luigi Corvaglia
- Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy
- Neonatal Intensive Care Unit, IRCCS AOUBO, Bologna, Italy
| | - Antonio Di Mauro
- Pediatric Primary Care, National Pediatric Health Care System, ASL BA, BARI, Italy
| | - Arianna Aceti
- Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy
| |
Collapse
|
20
|
Yu H, Cao Z, Pang G, Wu F, Zhu H, Zhu F. A deep-learning system for diagnosing ectopic eruption. J Dent 2025; 152:105399. [PMID: 39424256 DOI: 10.1016/j.jdent.2024.105399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 10/01/2024] [Accepted: 10/06/2024] [Indexed: 10/21/2024] Open
Abstract
OBJECTIVES To construct a diagnostic model for mixed dentition using a multistage deep-learning network to predict potential ectopic eruption in permanent teeth by integrating dentition segmentation into the process of automatic classification of dental development stages. METHODS A database was established by reviewing 1576 anonymous panoramic radiographs of children aged 6-12 years, collected at the Stomatology Hospital, Zhejiang University School of Medicine. These radiographs were categorised as normal or ectopic eruption, with expert diagnoses serving as a benchmark for training and evaluating artificial intelligence (AI) models. Furthermore, tooth boundaries and dental development stages were manually annotated by three pediatric dentistry experts. The dataset was split into training, validation, and test sets at an 8:1:1 ratio. RESULTS The diagnostic performance of the deep-learning model was rigorously evaluated. The model demonstrated accuracy in tooth segmentation, with Intersection over Union, precision, sensitivity, and F1 scores of 0.959, 0.993, 0.966, and 0.979, respectively. Furthermore, its ability to identify tooth ectopic eruptions on panoramic radiographs, when compared to evaluations by three dentists. Based on McNemar's test, the model's specificity and accuracy in identifying ectopic tooth eruptions on the test dataset surpassed that of Dentist 1 (P < 0.05), while no significant difference was observed compared to the other two dentists. Besides, the deep learning model also showed its potential in classifying dental development stages, as tested against three different standards. CONCLUSIONS The adaptability of the AI-enabled model in this study was demonstrated across multiple scenarios, with clinical validation highlighting its efficacy in diagnosing ectopic eruptions using a multistage deep-learning approach. CLINICAL SIGNIFICANCE Our findings provide new insights and technical support for the prevention and treatment of abnormal tooth eruption, laying the groundwork for predictive models for other prevalent pediatric dentistry conditions.
Collapse
Affiliation(s)
- Haojie Yu
- Stomatology Hospital, School of Stomatology, Zhejiang University School of Medicine, Zhejiang Provincial Clinical Research Center for Oral Diseases, Key Laboratory of Oral Biomedical Research of Zhejiang Province, Cancer Center of Zhejiang University, Engineering Research Center of Oral Biomaterials and Devices of Zhejiang Province, Hangzhou, Zhejiang, China
| | - Zheng Cao
- School of Artificial Intelligence, School of Future Technology, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, 210044, China
| | - Gaozhi Pang
- College of Computer Science and Technology, Zhejiang University of Technology.Hangzhou, Zhejiang, 310023, China
| | - Fuli Wu
- College of Computer Science and Technology, Zhejiang University of Technology.Hangzhou, Zhejiang, 310023, China
| | - Haihua Zhu
- Stomatology Hospital, School of Stomatology, Zhejiang University School of Medicine, Zhejiang Provincial Clinical Research Center for Oral Diseases, Key Laboratory of Oral Biomedical Research of Zhejiang Province, Cancer Center of Zhejiang University, Engineering Research Center of Oral Biomaterials and Devices of Zhejiang Province, Hangzhou, Zhejiang, China
| | - Fudong Zhu
- Stomatology Hospital, School of Stomatology, Zhejiang University School of Medicine, Zhejiang Provincial Clinical Research Center for Oral Diseases, Key Laboratory of Oral Biomedical Research of Zhejiang Province, Cancer Center of Zhejiang University, Engineering Research Center of Oral Biomaterials and Devices of Zhejiang Province, Hangzhou, Zhejiang, China.
| |
Collapse
|
21
|
Tekin A, Herasevich S, Minteer SA, Gajic O, Barwise AK. Exploring Stakeholder Perceptions about Using Artificial Intelligence for the Diagnosis of Rare and Atypical Infections. Appl Clin Inform 2025; 16:223-233. [PMID: 39454642 PMCID: PMC11882315 DOI: 10.1055/a-2451-9046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2024] [Accepted: 10/24/2024] [Indexed: 10/28/2024] Open
Abstract
OBJECTIVES This study aimed to evaluate critical care provider perspectives about diagnostic practices for rare and atypical infections and the potential for using artificial intelligence (AI) as a decision support system (DSS). METHODS We conducted an anonymous web-based survey among critical care providers at Mayo Clinic Rochester between November 25, 2023, and January 15, 2024, to evaluate their experience with rare and atypical infection diagnostic processes and AI-based DSSs. We also assessed the perceived usefulness of AI-based DSSs, their potential impact on improving diagnostic practices for rare and atypical infections, and the perceived risks and benefits of their use. RESULTS A total of 47/143 providers completed the survey. Thirty-eight out of 47 agreed that there was a delay in diagnosing rare and atypical infections. Among those who agreed, limited assessment of specific patient factors and failure to consider them were the most frequently cited important contributing factors (33/38). Thirty-eight out of 47 reported familiarity with the AI-based DSS applications available to critical care providers. Less than half (18/38) thought AI-based DSSs often provided valuable insights into patient care, but almost three-quarters (34/47) thought AI-based DDSs often provided valuable insight when specifically asked about their ability to improve the diagnosis of rare and atypical infections. All respondents rated reliability as important in enhancing the perceived utility of AI-based DSSs (47/47) and almost all rated interpretability and integration into the workflow as important (45/47). The primary concern about implementing an AI-based DSS in this context was alert fatigue (44/47). CONCLUSION Most critical care providers perceive that there are delays in diagnosing rare infections, indicating inadequate assessment and consideration of the diagnosis as the major contributors. Reliability, interpretability, workflow integration, and alert fatigue emerged as key factors impacting the usability of AI-based DSS. These findings will inform the development and implementation of an AI-based diagnostic algorithm to aid in identifying rare and atypical infections.
Collapse
Affiliation(s)
- Aysun Tekin
- Division of Nephrology and Hypertension, Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota, United States
| | - Svetlana Herasevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, Minnesota, United States
| | - Sarah A. Minteer
- Department of Physical Medicine and Rehabilitation Research, Mayo Clinic, Rochester, Minnesota, United States
| | - Ognjen Gajic
- Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota, United States
| | - Amelia K. Barwise
- Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota, United States
| |
Collapse
|
22
|
Ji XL, Xu S, Li XY, Xu JH, Han RS, Guo YJ, Duan LP, Tian ZB. Prognostic prediction models for postoperative patients with stage I to III colorectal cancer based on machine learning. World J Gastrointest Oncol 2024; 16:4597-4613. [PMID: 39678810 PMCID: PMC11577370 DOI: 10.4251/wjgo.v16.i12.4597] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/19/2024] [Revised: 09/07/2024] [Accepted: 09/14/2024] [Indexed: 11/12/2024] Open
Abstract
BACKGROUND Colorectal cancer (CRC) is characterized by high heterogeneity, aggressiveness, and high morbidity and mortality rates. With machine learning (ML) algorithms, patient, tumor, and treatment features can be used to develop and validate models for predicting survival. In addition, important variables can be screened and different applications can be provided that could serve as vital references when making clinical decisions and potentially improving patient outcomes in clinical settings. AIM To construct prognostic prediction models and screen important variables for patients with stage I to III CRC. METHODS More than 1000 postoperative CRC patients were grouped according to survival time (with cutoff values of 3 years and 5 years) and assigned to training and testing cohorts (7:3). For each 3-category survival time, predictions were made by 4 ML algorithms (all-variable and important variable-only datasets), each of which was validated via 5-fold cross-validation and bootstrap validation. Important variables were screened with multivariable regression methods. Model performance was evaluated and compared before and after variable screening with the area under the curve (AUC). SHapley Additive exPlanations (SHAP) further demonstrated the impact of important variables on model decision-making. Nomograms were constructed for practical model application. RESULTS Our ML models performed well; the model performance before and after important parameter identification was consistent, and variable screening was effective. The highest pre- and postscreening model AUCs 95% confidence intervals in the testing set were 0.87 (0.81-0.92) and 0.89 (0.84-0.93) for overall survival, 0.75 (0.69-0.82) and 0.73 (0.64-0.81) for disease-free survival, 0.95 (0.88-1.00) and 0.88 (0.75-0.97) for recurrence-free survival, and 0.76 (0.47-0.95) and 0.80 (0.53-0.94) for distant metastasis-free survival. Repeated cross-validation and bootstrap validation were performed in both the training and testing datasets. The SHAP values of the important variables were consistent with the clinicopathological characteristics of patients with tumors. The nomograms were created. CONCLUSION We constructed a comprehensive, high-accuracy, important variable-based ML architecture for predicting the 3-category survival times. This architecture could serve as a vital reference for managing CRC patients.
Collapse
Affiliation(s)
- Xiao-Lin Ji
- Department of Gastroenterology, Beijing Key Laboratory for Helicobacter Pylori Infection and Upper Gastrointestinal Diseases, Peking University Third Hospital, Beijing 100191, China
| | - Shuo Xu
- Beijing Aerospace Wanyuan Science Technology Co., Ltd., China Academy of Launch Vehicle Technology, Beijing 100176, China
| | - Xiao-Yu Li
- Department of Gastroenterology, The Affiliated Hospital of Qingdao University, Qingdao 266003, Shandong Province, China
| | - Jin-Huan Xu
- Institute of Automation, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250014, Shandong Province, China
| | - Rong-Shuang Han
- Department of Gastroenterology, The Affiliated Hospital of Qingdao University, Qingdao 266003, Shandong Province, China
| | - Ying-Jie Guo
- Department of Gastroenterology, The Affiliated Hospital of Qingdao University, Qingdao 266003, Shandong Province, China
| | - Li-Ping Duan
- Department of Gastroenterology, Beijing Key Laboratory for Helicobacter Pylori Infection and Upper Gastrointestinal Diseases, Peking University Third Hospital, Beijing 100191, China
| | - Zi-Bin Tian
- Department of Gastroenterology, The Affiliated Hospital of Qingdao University, Qingdao 266003, Shandong Province, China
| |
Collapse
|
23
|
Lee H, Chung JW, Kim KO, Kwon KA, Kim JH, Yun SC, Jung SW, Sheeraz A, Yoon YJ, Kim JH, Kayasseh MA. Validation of Artificial Intelligence Computer-Aided Detection of Colonic Neoplasm in Colonoscopy. Diagnostics (Basel) 2024; 14:2762. [PMID: 39682670 DOI: 10.3390/diagnostics14232762] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2024] [Revised: 12/02/2024] [Accepted: 12/04/2024] [Indexed: 12/18/2024] Open
Abstract
BACKGROUND/OBJECTIVES Controlling colonoscopic quality is important in the detection of colon polyps during colonoscopy as it reduces the overall long-term colorectal cancer risk. Artificial intelligence has recently been introduced in various medical fields. In this study, we aimed to validate a previously developed artificial intelligence (AI) computer-aided detection (CADe) algorithm called ALPHAON® and compare outcomes with previous studies that showed that AI outperformed and assisted endoscopists of diverse levels of expertise in detecting colon polyps. METHODS We used the retrospective data of 500 still images, including 100 polyp images and 400 healthy colon images. In addition, we validated the CADe algorithm and compared its diagnostic performance with that of two expert endoscopists and six trainees from Gachon University Gil Medical Center. After a washing-out period of over 2 weeks, endoscopists performed polyp detection on the same dataset with the assistance of ALPHAON®. RESULTS The CADe algorithm presented a high capability in detecting colon polyps, with an accuracy of 0.97 (95% CI: 0.96 to 0.99), sensitivity of 0.91 (95% CI: 0.85 to 0.97), specificity of 0.99 (95% CI: 0.97 to 0.99), and AUC of 0.967. When evaluating and comparing the polyp detection ability of ALPHAON® with that of endoscopists with different levels of expertise (regarding years of endoscopic experience), it was found that ALPHAON® outperformed the experts in accuracy (0.97, 95% CI: 0.96 to 0.99), sensitivity (0.91, 95% CI: 0.85 to 0.97), and specificity (0.99, 95% CI: 0.97 to 0.99). After a washing-out period of over 2 weeks, the overall capability significantly improved for both experts and trainees with the assistance of ALPHAON®. CONCLUSIONS The high performance of the CADe algorithm system in colon polyp detection during colonoscopy was verified. The sensitivity of ALPHAON® led to it outperforming the experts, and it demonstrated the ability to enhance the polyp detection ability of both experts and trainees, which suggests a significant possibility of ALPHAON® being able to provide endoscopic assistance.
Collapse
Affiliation(s)
- Hannah Lee
- Division of Gastroenterology, Department of Internal Medicine, Gachon University Gil Medical Center, Incheon 21565, Republic of Korea
| | - Jun-Won Chung
- Division of Gastroenterology, Department of Internal Medicine, Gachon University Gil Medical Center, Incheon 21565, Republic of Korea
| | - Kyoung Oh Kim
- Division of Gastroenterology, Department of Internal Medicine, Gachon University Gil Medical Center, Incheon 21565, Republic of Korea
| | - Kwang An Kwon
- Division of Gastroenterology, Department of Internal Medicine, Gachon University Gil Medical Center, Incheon 21565, Republic of Korea
| | - Jung Ho Kim
- Division of Gastroenterology, Department of Internal Medicine, Gachon University Gil Medical Center, Incheon 21565, Republic of Korea
| | - Sung-Cheol Yun
- Division of Biostatistics, Center for Medical Research and Information, University of Ulsan College of Medicine, Seoul 05505, Republic of Korea
| | - Sung Woo Jung
- Division of Gastroenterology, Department of Internal Medicine, Korea University College of Medicine, Ansan 15355, Republic of Korea
| | | | | | - Ji Hee Kim
- CAIMI Co., Ltd., Incheon 22004, Republic of Korea
| | - Mohd Azzam Kayasseh
- Division of Gastroenterology, Dr. Sulaiman AI Habib Medical Group, Dubai Healthcare City, Dubai 51431, United Arab Emirates
| |
Collapse
|
24
|
Choi A, Lee K, Hyun H, Kim KJ, Ahn B, Lee KH, Hahn S, Choi SY, Kim JH. A novel deep learning algorithm for real-time prediction of clinical deterioration in the emergency department for a multimodal clinical decision support system. Sci Rep 2024; 14:30116. [PMID: 39627310 PMCID: PMC11615388 DOI: 10.1038/s41598-024-80268-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2024] [Accepted: 11/18/2024] [Indexed: 12/06/2024] Open
Abstract
The array of complex and evolving patient data has limited clinical decision making in the emergency department (ED). This study introduces an advanced deep learning algorithm designed to enhance real-time prediction accuracy for integration into a novel Clinical Decision Support System (CDSS). A retrospective study was conducted using data from a level 1 tertiary hospital. The algorithm's predictive performance was evaluated based on in-hospital cardiac arrest, inotropic circulatory support, advanced airway, and intensive care unit admission. We developed an artificial intelligence (AI) algorithm for CDSS that integrates multiple data modalities, including vitals, laboratory, and imaging results from electronic health records. The AI model was trained and tested on a dataset of 237,059 ED visits. The algorithm's predictions, based solely on triage information, significantly outperformed traditional logistic regression models, with notable improvements in the area under the precision-recall curve (AUPRC). Additionally, predictive accuracy improved with the inclusion of continuous data input at shorter intervals. This study suggests the feasibility of using AI algorithms in diverse clinical scenarios, particularly for earlier detection of clinical deterioration. Future work should focus on expanding the dataset and enhancing real-time data integration across multiple centers to further optimize its application within the novel CDSS.
Collapse
Affiliation(s)
- Arom Choi
- Department of Emergency medicine, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, Republic of Korea.
- Institute for Innovation in Digital Healthcare, Yonsei University, Seodaemun-gu, 50 Yonsei-ro, Seoul, Republic of Korea.
| | - Kwanhyung Lee
- AITRICS Corp., 218, Teheran-ro, Gangnam-gu, Seoul, Republic of Korea
| | - Heejung Hyun
- AITRICS Corp., 218, Teheran-ro, Gangnam-gu, Seoul, Republic of Korea
| | - Kwang Joon Kim
- AITRICS Corp., 218, Teheran-ro, Gangnam-gu, Seoul, Republic of Korea
- Division of Geriatrics, Department of Internal Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Byungeun Ahn
- AITRICS Corp., 218, Teheran-ro, Gangnam-gu, Seoul, Republic of Korea
| | - Kyung Hyun Lee
- AITRICS Corp., 218, Teheran-ro, Gangnam-gu, Seoul, Republic of Korea
| | - Sangchul Hahn
- AITRICS Corp., 218, Teheran-ro, Gangnam-gu, Seoul, Republic of Korea
| | - So Yeon Choi
- Department of Emergency medicine, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, Republic of Korea
| | - Ji Hoon Kim
- Department of Emergency medicine, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, Republic of Korea.
- Institute for Innovation in Digital Healthcare, Yonsei University, Seodaemun-gu, 50 Yonsei-ro, Seoul, Republic of Korea.
| |
Collapse
|
25
|
Leoste J, Strömberg-Järvis K, Robal T, Marmor K, Kangur K, Rebane AM. Testing scenarios for using telepresence robots in healthcare settings. Comput Struct Biotechnol J 2024; 24:105-114. [PMID: 38314026 PMCID: PMC10837455 DOI: 10.1016/j.csbj.2024.01.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 01/09/2024] [Accepted: 01/14/2024] [Indexed: 02/06/2024] Open
Abstract
The ageing global population puts heavy pressure on healthcare systems everywhere. Addressing ageing-related chronic conditions requires employment of novel innovative solutions. Telehealth technologies, including telepresence robots (TPRs), are being rapidly developed to provide healthcare services efficiently wherever needed. This article explores the role of TPRs in addressing the challenges of providing healthcare to an ageing population, emphasizing their potential advantages and drawbacks. Employing an exploratory research approach with qualitative data collection techniques, we tested three TPR usage scenarios in simulated healthcare settings: anamnesis, measurements, and falls and frailty. The study employed a non-random purposive sample comprising 25 participants, and was conducted at a medical facility in June 2023. The findings suggest that TPRs offer promising solutions for healthcare professionals and patients, especially in scenarios when physical presence is impossible or physical isolation is required to prevent contagion. However, the technology is not yet ready to substitute fully human medical workers, potentially causing patient reluctance and emphasizing the need for patient-centered approaches to technology adoption. In addition, more studies are needed to address ethical, privacy, and scalability concerns.
Collapse
Affiliation(s)
- Janika Leoste
- Tallinn University, Narva rd 25, 10120 Tallinn, Estonia
- Tallinn University of Technology, Ehitajate tee 5, 19086 Tallinn, Estonia
| | | | - Tarmo Robal
- Tallinn University of Technology, Ehitajate tee 5, 19086 Tallinn, Estonia
| | - Kristel Marmor
- Tallinn University of Technology, Ehitajate tee 5, 19086 Tallinn, Estonia
| | - Katrin Kangur
- Tallinn University of Technology, Ehitajate tee 5, 19086 Tallinn, Estonia
| | | |
Collapse
|
26
|
Lee H, Chung JW, Yun SC, Jung SW, Yoon YJ, Kim JH, Cha B, Kayasseh MA, Kim KO. Validation of Artificial Intelligence Computer-Aided Detection on Gastric Neoplasm in Upper Gastrointestinal Endoscopy. Diagnostics (Basel) 2024; 14:2706. [PMID: 39682614 DOI: 10.3390/diagnostics14232706] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2024] [Revised: 11/22/2024] [Accepted: 11/25/2024] [Indexed: 12/18/2024] Open
Abstract
BACKGROUND/OBJECTIVES Gastric cancer ranks fifth for incidence and fourth in the leading causes of mortality worldwide. In this study, we aimed to validate previously developed artificial intelligence (AI) computer-aided detection (CADe) algorithm, called ALPHAON® in detecting gastric neoplasm. METHODS We used the retrospective data of 500 still images, including 5 benign gastric ulcers, 95 with gastric cancer, and 400 normal images. Thereby we validated the CADe algorithm measuring accuracy, sensitivity, and specificity with the result of receiver operating characteristic curves (ROC) and area under curve (AUC) in addition to comparing the diagnostic performance status of four expert endoscopists, four trainees, and four beginners from two university-affiliated hospitals with CADe algorithm. After a washing-out period of over 2 weeks, endoscopists performed gastric detection on the same dataset of the 500 endoscopic images again marked by ALPHAON®. RESULTS The CADe algorithm presented high validity in detecting gastric neoplasm with accuracy (0.88, 95% CI: 0.85 to 0.91), sensitivity (0.93, 95% CI: 0.88 to 0.98), specificity (0.87, 95% CI: 0.84 to 0.90), and AUC (0.962). After a washing-out period of over 2 weeks, overall validity improved in the trainee and beginner groups with the assistance of ALPHAON®. Significant improvement was present, especially in the beginner group (accuracy 0.94 (0.93 to 0.96) p < 0.001, sensitivity 0.87 (0.82 to 0.92) p < 0.001, specificity 0.96 (0.95 to 0.97) p < 0.001). CONCLUSIONS The high validation performance state of the CADe algorithm system was verified. Also, ALPHAON® has demonstrated its potential to serve as an endoscopic educator for beginners improving and making progress in sensitivity and specificity.
Collapse
Affiliation(s)
- Hannah Lee
- Division of Gastroenterology, Department of Internal Medicine, Gachon University, Gil Medical Center, Incheon 21565, Republic of Korea
| | - Jun-Won Chung
- Division of Gastroenterology, Department of Internal Medicine, Gachon University, Gil Medical Center, Incheon 21565, Republic of Korea
| | - Sung-Cheol Yun
- Division of Biostatistics, Center for Medical Research and Information, University of Ulsan College of Medicine, Seoul 05505, Republic of Korea
| | - Sung Woo Jung
- Division of Gastroenterology, Department of Internal Medicine, Korea University College of Medicine, Ansan 15355, Republic of Korea
| | | | - Ji Hee Kim
- CAIMI Co., Ltd., Incheon 22004, Republic of Korea
| | - Boram Cha
- Division of Gastroenterology, Department of Internal Medicine, Inha University Hospital, Inha University School of Medicine, Incheon 22332, Republic of Korea
| | - Mohd Azzam Kayasseh
- Division of Gastroenterology, Dr. Sulaiman AI Habib Medical Group, Dubai Healthcare City, Dubai 51431, United Arab Emirates
| | - Kyoung Oh Kim
- Division of Gastroenterology, Department of Internal Medicine, Gachon University, Gil Medical Center, Incheon 21565, Republic of Korea
| |
Collapse
|
27
|
Park HJ, Huh JY, Chae G, Choi MG. Extraction of clinical data on major pulmonary diseases from unstructured radiologic reports using a large language model. PLoS One 2024; 19:e0314136. [PMID: 39585830 PMCID: PMC11588275 DOI: 10.1371/journal.pone.0314136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2024] [Accepted: 11/06/2024] [Indexed: 11/27/2024] Open
Abstract
Despite significant strides in big data technology, extracting information from unstructured clinical data remains a formidable challenge. This study investigated the utility of large language models (LLMs) for extracting clinical data from unstructured radiological reports without additional training. In this retrospective study, 1800 radiologic reports, 600 from each of the three university hospitals, were collected, with seven pulmonary outcomes defined. Three pulmonology-trained specialists discerned the presence or absence of diseases. Data extraction from the reports was executed using Google Gemini Pro 1.0, OpenAI's GPT-3.5, and GPT-4. The gold standard was predicated on agreement between at least two pulmonologists. This study evaluated the performance of the three LLMs in diagnosing seven pulmonary diseases (active tuberculosis, emphysema, interstitial lung disease, lung cancer, pleural effusion, pneumonia, and pulmonary edema) utilizing chest radiography and computed tomography scans. All models exhibited high accuracy (0.85-1.00) for most conditions. GPT-4 consistently outperformed its counterparts, demonstrating a sensitivity of 0.71-1.00; specificity of 0.89-1.00; and accuracy of 0.89 and 0.99 across both modalities, thus underscoring its superior capability in interpreting radiological reports. Notably, the accuracy of pleural effusion and emphysema on chest radiographs and pulmonary edema on chest computed tomography scans reached 0.99. The proficiency of LLMs, particularly GPT-4, in accurately classifying unstructured radiological data hints at their potential as alternatives to the traditional manual chart reviews conducted by clinicians.
Collapse
Affiliation(s)
- Hyung Jun Park
- Department of Internal Medicine, Division of Pulmonary and Critical Care Medicine, Shihwa Medical Center, Siheung, Korea
| | - Jin-Young Huh
- Department of Internal Medicine, Division of Pulmonary, Allergy and Critical Care Medicine, Chung-Ang University Gwangmyeong Hospital, Gwangmyeong, Korea
| | - Ganghee Chae
- Department of Internal Medicine, Division of Pulmonary and Critical Care Medicine, Ulsan University Hospital, University of Ulsan College of Medicine, Ulsan, Korea
| | - Myeong Geun Choi
- Department of Internal Medicine, Division of Pulmonary and Critical Care Medicine, Mokdong Hospital, College of Medicine, Ewha Womans University, Seoul, Korea
| |
Collapse
|
28
|
McOmber BG, Moreira AG, Kirkman K, Acosta S, Rusin C, Shivanna B. Predictive analytics in bronchopulmonary dysplasia: past, present, and future. Front Pediatr 2024; 12:1483940. [PMID: 39633818 PMCID: PMC11615574 DOI: 10.3389/fped.2024.1483940] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/20/2024] [Accepted: 10/29/2024] [Indexed: 12/07/2024] Open
Abstract
Bronchopulmonary dysplasia (BPD) remains a significant complication of prematurity, impacting approximately 18,000 infants annually in the United States. Advances in neonatal care have not reduced BPD, and its management is challenged by the rising survival of extremely premature infants and the variability in clinical practices. Leveraging statistical and machine learning techniques, predictive analytics can enhance BPD management by utilizing large clinical datasets to predict individual patient outcomes. This review explores the foundations and applications of predictive analytics in the context of BPD, examining commonly used data sources, modeling techniques, and metrics for model evaluation. We also highlight bioinformatics' potential role in understanding BPD's molecular basis and discuss case studies demonstrating the use of machine learning models for risk prediction and prognosis in neonates. Challenges such as data bias, model complexity, and ethical considerations are outlined, along with strategies to address these issues. Future directions for advancing the integration of predictive analytics into clinical practice include improving model interpretability, expanding data sharing and interoperability, and aligning predictive models with precision medicine goals. By overcoming current challenges, predictive analytics holds promise for transforming neonatal care and providing personalized interventions for infants at risk of BPD.
Collapse
Affiliation(s)
- Bryan G. McOmber
- Division of Neonatology, Department of Pediatrics, University Hospital, University of Texas Health Science Center at San Antonio, San Antonio, TX, United States
| | - Alvaro G. Moreira
- Division of Neonatology, Department of Pediatrics, University Hospital, University of Texas Health Science Center at San Antonio, San Antonio, TX, United States
| | - Kelsey Kirkman
- Division of Neonatology, Department of Pediatrics, Texas Children’s Hospital, Baylor College of Medicine, Houston, TX, United States
| | - Sebastian Acosta
- Division of Pediatric Cardiology, Department of Pediatrics, Texas Children’s Hospital, Baylor College of Medicine, Houston, TX, United States
| | - Craig Rusin
- Division of Pediatric Cardiology, Department of Pediatrics, Texas Children’s Hospital, Baylor College of Medicine, Houston, TX, United States
| | - Binoy Shivanna
- Division of Neonatology, Department of Pediatrics, Texas Children’s Hospital, Baylor College of Medicine, Houston, TX, United States
| |
Collapse
|
29
|
Liu J, Duan X, Duan M, Jiang Y, Mao W, Wang L, Liu G. Development and external validation of an interpretable machine learning model for the prediction of intubation in the intensive care unit. Sci Rep 2024; 14:27174. [PMID: 39511328 PMCID: PMC11544239 DOI: 10.1038/s41598-024-77798-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2024] [Accepted: 10/25/2024] [Indexed: 11/15/2024] Open
Abstract
Given the limited capacity to accurately determine the necessity for intubation in intensive care unit settings, this study aimed to develop and externally validate an interpretable machine learning model capable of predicting the need for intubation among ICU patients. Seven widely used machine learning (ML) algorithms were employed to construct the prediction models. Adult patients from the Medical Information Mart for Intensive Care IV database who stayed in the ICU for longer than 24 h were included in the development and internal validation. The model was subsequently externally validated using the eICU-CRD database. In addition, the SHapley Additive exPlanations method was employed to interpret the influence of individual parameters on the predictions made by the model. A total of 11,988 patients were included in the final cohort for this study. The CatBoost model demonstrated the best performance (AUC: 0.881). In the external validation set, the efficacy of our model was also confirmed (AUC: 0.750), which suggests robust generalization capabilities. The Glasgow Coma Scale (GCS), body mass index (BMI), arterial partial pressure of oxygen (PaO2), respiratory rate (RR) and length of stay (LOS) before ICU were the top 5 features of the CatBoost model with the greatest impact. We developed an externally validated CatBoost model that accurately predicts the need for intubation in ICU patients within 24 to 96 h of admission, facilitating clinical decision-making and has the potential to improve patient outcomes. The prediction model utilizes readily obtainable monitoring parameters and integrates the SHAP method to enhance interpretability, providing clinicians with clear insights into the factors influencing predictions.
Collapse
Affiliation(s)
- Jianyuan Liu
- Emergency Medicine Clinical Research Center, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Xiangjie Duan
- Department of Infectious Diseases, Department of Emergency Medicine, The First Affiliated Hospital of Jinan University, Guangzhou, China
| | - Minjie Duan
- Center for Artificial Intelligence in Medicine, Chinese PLA General Hospital, Beijing, China
| | - Yu Jiang
- Department of Respiratory and Critical Care Medicine, University-Town Hospital of Chongqing Medical University, Chongqing, China
| | - Wei Mao
- Department of Emergency and Critical Care Medicine, University-Town Hospital of Chongqing Medical University, Chongqing, China
| | - Lilin Wang
- Department of Emergency and Critical Care Medicine, University-Town Hospital of Chongqing Medical University, Chongqing, China
| | - Gang Liu
- Department of Emergency and Critical Care Medicine, University-Town Hospital of Chongqing Medical University, Chongqing, China.
| |
Collapse
|
30
|
Feng M, Meng F, Jia Y, Wang Y, Ji G, Gao C, Luo J. Exploration of Risk Factors for Cardiovascular Disease in Patients with Rheumatoid Arthritis: A Retrospective Study. Inflammation 2024:10.1007/s10753-024-02157-5. [PMID: 39414673 DOI: 10.1007/s10753-024-02157-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 08/21/2024] [Accepted: 09/27/2024] [Indexed: 10/18/2024]
Abstract
OBJECTIVE Patients with rheumatoid arthritis (RA) have increased mortality and morbidity rates owing to cardiovascular diseases (CVD). Timely detection of CVD in RA can greatly improve patient prognosis; however, this technique remains challenging. We aimed to investigate the risk factors for CVD incidence in patients with RA. METHODS This retrospective study included RA patients without CVD risk factors (n = 402), RA with CVD risk factors (n = 394), and RA with CVD (n = 201). Their data on routine examination indicators, vascular endothelial growth factor (VEGF), and immune cells were obtained from medical records. The characteristic variables between each group were screened using univariate analysis, least absolute shrinkage and selection operator (LASSO), random forest (RF), and logistic regression (LR) models, and individualized nomograms were further established to more conveniently observe the likelihood of CVD in RA. RESULTS Univariate analysis revealed significantly elevated levels of white blood cells (WBC), blood urea nitrogen (BUN), creatinine, creatine kinase (CK), lactate dehydrogenase (LDH), VEGF, serum total cholesterol (TC), triglycerides (TG), low-density lipoprotein (LDL), apolipoprotein B100 (ApoB100), and apolipoprotein E (ApoE) in RA patients with CVD, whereas apolipoprotein A1 (ApoA1) and high-density lipoprotein/cholesterol (HDL/TC) were decreased. Furthermore, the ratio of regulatory T (Treg) cells exhibiting excellent separation performance in RA patients with CVD was significantly lower than that in other groups, whereas the ratios of Th1/Th2/NK and Treg cells were significantly elevated. The LASSO, RF, and LR models were also used to identify the risk factors for CVD in patients with RA. Through the final selected indicators screened using the three machine learning models and univariate analysis, a convenient nomogram was established to observe the likelihood of CVD in patients with RA. CONCLUSIONS Serum lipids, lipoproteins, and reduction of Treg cells have been identified as risk factors for CVD in patients with RA. Three nomograms combining various risk factors were constructed to predict CVD occurring in patients with RA (RA with/without CVD risk factors).
Collapse
Affiliation(s)
- Min Feng
- Department of Rheumatology, the Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Fanxing Meng
- Shanxi Medical University, Taiyuan, Shanxi, China
| | - Yuhan Jia
- Shanxi Medical University, Taiyuan, Shanxi, China
| | - Yanlin Wang
- Department of Rheumatology, the Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Guozhen Ji
- Shanxi Medical University, Taiyuan, Shanxi, China
| | - Chong Gao
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Jing Luo
- Department of Rheumatology, the Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China.
| |
Collapse
|
31
|
Jia H, Tang S, Guo W, Pan P, Qian Y, Hu D, Dai Y, Yang Y, Geng C, Lv H. Differential diagnosis of congenital ventricular septal defect and atrial septal defect in children using deep learning-based analysis of chest radiographs. BMC Pediatr 2024; 24:661. [PMID: 39407181 PMCID: PMC11476512 DOI: 10.1186/s12887-024-05141-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 10/09/2024] [Indexed: 10/20/2024] Open
Abstract
BACKGROUND Children with atrial septal defect (ASD) and ventricular septal defect (VSD) are frequently examined for respiratory symptoms, even when the underlying disease is not found. Chest radiographs often serve as the primary imaging modality. It is crucial to differentiate between ASD and VSD due to their distinct treatment. PURPOSE To assess whether deep learning analysis of chest radiographs can more effectively differentiate between ASD and VSD in children. METHODS In this retrospective study, chest radiographs and corresponding radiology reports from 1,194 patients were analyzed. The cases were categorized into a training set and a validation set, comprising 480 cases of ASD and 480 cases of VSD, and a test set with 115 cases of ASD and 119 cases of VSD. Four deep learning network models-ResNet-CBAM, InceptionV3, EfficientNet, and ViT-were developed for training, and a fivefold cross-validation method was employed to optimize the models. Receiver operating characteristic (ROC) curve analyses were conducted to assess the performance of each model. The most effective algorithm was compared with the interpretations provided by two radiologists on 234 images from the test group. RESULTS The average accuracy, sensitivity, and specificity of the four deep learning models in the differential diagnosis of VSD and ASD were higher than 70%. The AUC values of ResNet-CBAM, IncepetionV3, EfficientNet, and ViT were 0.87, 0.91, 0.90, and 0.66, respectively. Statistical analysis showed that the differential diagnosis efficiency of InceptionV3 was the highest, reaching 87% classification accuracy. The accuracy of InceptionV3 in the differential diagnosis of VSD and ASD was higher than that of the radiologists. CONCLUSIONS Deep learning methods such as IncepetionV3 based on chest radiographs in the study showed good performance for differential diagnosis of congenital VSD and ASD, which may be able to assist radiologists in diagnosis, education, and training, and reduce missed diagnosis and misdiagnosis.
Collapse
Affiliation(s)
- Huihui Jia
- Department of Radiology, Children ' s Hospital of Soochow University, 215025, Suzhou, P. R. China
| | - Songqiao Tang
- School of Electronic & Information Engineering, Suzhou University of Science and Technology, 215009, Suzhou, China
| | - Wanliang Guo
- Department of Radiology, Children ' s Hospital of Soochow University, 215025, Suzhou, P. R. China
| | - Peng Pan
- Department of Radiology, Children ' s Hospital of Soochow University, 215025, Suzhou, P. R. China
| | - Yufeng Qian
- Department of Radiology, Children ' s Hospital of Soochow University, 215025, Suzhou, P. R. China
| | - Dongliang Hu
- Department of Radiology, Children ' s Hospital of Soochow University, 215025, Suzhou, P. R. China
| | - Yakang Dai
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, 215163, Suzhou, China
| | - Yang Yang
- Department of Radiology, Children ' s Hospital of Soochow University, 215025, Suzhou, P. R. China
| | - Chen Geng
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, 215163, Suzhou, China.
- Jinan Guoke Medical Technology Development Co., Ltd, 250102, Shandong, China.
| | - Haitao Lv
- Department of Pediatric Cardiology, Children ' s Hospital of Soochow University, 215025, Suzhou, P. R. China.
| |
Collapse
|
32
|
Pupic N, Gabison S, Evans G, Fernie G, Dolatabadi E, Dutta T. Detecting Patient Position Using Bed-Reaction Forces for Pressure Injury Prevention and Management. SENSORS (BASEL, SWITZERLAND) 2024; 24:6483. [PMID: 39409523 PMCID: PMC11479332 DOI: 10.3390/s24196483] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/24/2024] [Revised: 09/21/2024] [Accepted: 10/01/2024] [Indexed: 10/20/2024]
Abstract
A key best practice to prevent and treat pressure injuries (PIs) is to ensure at-risk individuals are repositioned regularly. Our team designed a non-contact position detection system that predicts an individual's position in bed using data from load cells under the bed legs. The system was originally designed to predict the individual's position as left-side lying, right-side lying, or supine. Our previous work suggested that a higher precision for detecting position (classifying more than three positions) may be needed to determine whether key bony prominences on the pelvis at high risk of PIs have been off-loaded. The objective of this study was to determine the impact of categorizing participant position with higher precision using the system prediction F1 score. Data from 18 participants was collected from four load cells placed under the bed legs and a pelvis-mounted inertial measurement unit while the participants assumed 21 positions. The data was used to train classifiers to predict the participants' transverse pelvic angle using three different position bin sizes (45°, ~30°, and 15°). A leave-one-participant-out cross validation approach was used to evaluate classifier performance for each bin size. Results indicated that our prediction F1 score dropped as the position category precision was increased.
Collapse
Affiliation(s)
- Nikola Pupic
- KITE Research Institute, Toronto Rehabilitation Institute—University Health Network, Toronto, ON M5G 2A2, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON M5S 3G9, Canada
| | - Sharon Gabison
- KITE Research Institute, Toronto Rehabilitation Institute—University Health Network, Toronto, ON M5G 2A2, Canada
- Department of Physical Therapy, University of Toronto, Toronto, ON M5G 1V7, Canada
- Rehabilitation Sciences Institute, University of Toronto, Toronto, ON M5G 1V7, Canada
| | - Gary Evans
- KITE Research Institute, Toronto Rehabilitation Institute—University Health Network, Toronto, ON M5G 2A2, Canada
| | - Geoff Fernie
- KITE Research Institute, Toronto Rehabilitation Institute—University Health Network, Toronto, ON M5G 2A2, Canada
| | | | - Tilak Dutta
- KITE Research Institute, Toronto Rehabilitation Institute—University Health Network, Toronto, ON M5G 2A2, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON M5S 3G9, Canada
- Rehabilitation Sciences Institute, University of Toronto, Toronto, ON M5G 1V7, Canada
| |
Collapse
|
33
|
Peng L, Liang R, Zhao A, Sun R, Yi F, Zhong J, Li R, Zhu S, Zhang S, Wu S. Amplifying Chinese physicians' emphasis on patients' psychological states beyond urologic diagnoses with ChatGPT - a multicenter cross-sectional study. Int J Surg 2024; 110:6501-6508. [PMID: 38954666 PMCID: PMC11487044 DOI: 10.1097/js9.0000000000001775] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2024] [Accepted: 05/29/2024] [Indexed: 07/04/2024]
Abstract
BACKGROUND Artificial intelligence (AI) technologies, particularly large language models (LLMs), have been widely employed by the medical community. In addressing the intricacies of urology, ChatGPT offers a novel possibility to aid in clinical decision-making. This study aimed to investigate the decision-making ability of LLMs in solving complex urology-related problems and assess their effectiveness in providing psychological support to patients with urological disorders. MATERIALS AND METHODS This study evaluated the clinical and psychological support capabilities of ChatGPT 3.5 and 4.0 in the field of urology. A total of 69 clinical and 30 psychological questions were posed to the AI models, and both urologists and psychologists evaluated their response. As a control, clinicians from Chinese medical institutions responded to closed-book conditions. Statistical analyses were conducted separately for each subgroup. RESULTS In multiple-choice tests covering diverse urological topics, ChatGPT 4.0 was performed comparably to the physician group, with no significant overall score difference. Subgroup analyses revealed variable performance based on disease type and physician experience, with ChatGPT 4.0 generally outperforming ChatGPT 3.5 and exhibiting competitive results against physicians. When assessing the psychological support capabilities of AI, it is evident that ChatGPT 4.0 outperforms ChatGPT 3.5 across all urology-related psychological problems. CONCLUSIONS The performance of LLMs in dealing with standardized clinical problems and providing psychological support has certain advantages over clinicians. AI stands out as a promising tool for potential clinical aid.
Collapse
Affiliation(s)
- Lei Peng
- Department of Urology, Lanzhou University Second Hospital, Lanzhou, Gansu
- Department of Urology, South China Hospital, Shenzhen University, Shenzhen, Guangdong
| | - Rui Liang
- Department of Urology, South China Hospital, Shenzhen University, Shenzhen, Guangdong
- Department of Urology, The First Affiliated Hospital of Soochow University
| | - Anguo Zhao
- Department of Urology, South China Hospital, Shenzhen University, Shenzhen, Guangdong
- Department of Urology, Dushu Lake Hospital Affiliated to Soochow University, Medical Center of Soochow University, Suzhou Dushu Lake Hospital, Suzhou, Jiangsu
| | - Ruonan Sun
- West China School of Medicine, Sichuan University, Chengdu
| | - Fulin Yi
- North Sichuan Medical College (University), Nanchong, Sichuan, People’s Republic of China
| | - Jianye Zhong
- Department of Urology, South China Hospital, Shenzhen University, Shenzhen, Guangdong
| | - Rongkang Li
- Department of Urology, Lanzhou University Second Hospital, Lanzhou, Gansu
- Department of Urology, South China Hospital, Shenzhen University, Shenzhen, Guangdong
| | - Shimao Zhu
- Department of Urology, South China Hospital, Shenzhen University, Shenzhen, Guangdong
| | - Shaohua Zhang
- Department of Urology, South China Hospital, Shenzhen University, Shenzhen, Guangdong
| | - Song Wu
- Department of Urology, Lanzhou University Second Hospital, Lanzhou, Gansu
- Department of Urology, South China Hospital, Shenzhen University, Shenzhen, Guangdong
| |
Collapse
|
34
|
Wang T, Hao J, Zhou J, Chen G, Shen H, Sun Q. Development and validation of a machine-learning model for predicting postoperative pneumonia in aneurysmal subarachnoid hemorrhage. Neurosurg Rev 2024; 47:668. [PMID: 39313739 DOI: 10.1007/s10143-024-02904-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2024] [Revised: 08/17/2024] [Accepted: 09/16/2024] [Indexed: 09/25/2024]
Abstract
Pneumonia is a common postoperative complication in patients with aneurysmal subarachnoid hemorrhage (aSAH), which is associated with poor prognosis and increased mortality. The aim of this study was to develop a predictive model for postoperative pneumonia (POP) in patients with aSAH. A retrospective analysis was conducted on 308 patients with aSAH who underwent surgery at the Neurosurgery Department of the First Affiliated Hospital of Soochow University. Univariate and multivariate logistic regression and lasso regression analysis were used to analyze the risk factors for POP. Receiver operating characteristic (ROC) curve, calibration curve, and decision curve analysis (DCA) were used to evaluate the constructed model. Finally, the effectiveness of modeling these six variables in different machine learning methods was investigated. In our patient cohort, 23.4% (n = 72/308) of patients experienced POP. Univariate, multivariate logistic regression analysis and lasso regression analysis revealed age, Hunt-Hess grade, mechanical ventilation, leukocyte count, lymphocyte count, and platelet count as independent risk factors for POP. Subsequently, these six factors were used to build the final model. We found that age, Hunt-Hess grade, mechanical ventilation, leukocyte count, lymphocyte count, and platelet count were independent risk factors for POP in patients with aSAH. Through validation and comparison with other studies and machine learning models, our novel predictive model has demonstrated high efficacy in effectively predicting the likelihood of pneumonia during the hospitalization of aSAH patients.
Collapse
Affiliation(s)
- Tong Wang
- Department of Neurosurgery & Brain and Nerve Research Laboratory, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, China
| | - Jiahui Hao
- Department of Neurosurgery & Brain and Nerve Research Laboratory, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, China
| | - Jialei Zhou
- Department of Neurosurgery & Brain and Nerve Research Laboratory, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, China
| | - Gang Chen
- Department of Neurosurgery & Brain and Nerve Research Laboratory, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, China.
- The First Affiliated Hospital of Soochow University Suzhou, 188 Shizi Street, Suzhou, 215006, Jiangsu Province, China.
| | - Haitao Shen
- Department of Neurosurgery & Brain and Nerve Research Laboratory, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, China
| | - Qing Sun
- Department of Neurosurgery & Brain and Nerve Research Laboratory, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, China.
| |
Collapse
|
35
|
Li Z, Huang J, Chen J, Zeng J, Jiang H, Ding L, Zhang T, Sun W, Lu R, Zhang Q, Liang L. A deep-learning pipeline for the diagnosis and grading of common blinding ophthalmic diseases based on lesion-focused classification model. Front Artif Intell 2024; 7:1444136. [PMID: 39324131 PMCID: PMC11422385 DOI: 10.3389/frai.2024.1444136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2024] [Accepted: 08/15/2024] [Indexed: 09/27/2024] Open
Abstract
Background Glaucoma (GLAU), Age-related Macular Degeneration (AMD), Retinal Vein Occlusion (RVO), and Diabetic Retinopathy (DR) are common blinding ophthalmic diseases worldwide. Purpose This approach is expected to enhance the early detection and treatment of common blinding ophthalmic diseases, contributing to the reduction of individual and economic burdens associated with these conditions. Methods We propose an effective deep-learning pipeline that combine both segmentation model and classification model for diagnosis and grading of four common blinding ophthalmic diseases and normal retinal fundus. Results In total, 102,786 fundus images of 75,682 individuals were used for training validation and external validation purposes. We test our model on internal validation data set, the micro Area Under the Receiver Operating Characteristic curve (AUROC) of which reached 0.995. Then, we fine-tuned the diagnosis model to classify each of the four disease into early and late stage, respectively, which achieved AUROCs of 0.597 (GL), 0.877 (AMD), 0.972 (RVO), and 0.961 (DR) respectively. To test the generalization of our model, we conducted two external validation experiments on Neimeng and Guangxi cohort, all of which maintained high accuracy. Conclusion Our algorithm demonstrates accurate artificial intelligence diagnosis pipeline for common blinding ophthalmic diseases based on Lesion-Focused fundus that overcomes the low-accuracy of the traditional classification method that based on raw retinal images, which has good generalization ability on diverse cases in different regions.
Collapse
Affiliation(s)
- Zhihuan Li
- Faculty of Medicine, Macau University of Science and Technology, Taipa, Macau, China
- Affiliated Hospital of Guangdong Medical University, Zhanjiang, China
- China Resources Power Intelligent Security Laboratory, Research Institute of Tsinghua University in Shenzhen, Shenzhen, China
| | - Junxiong Huang
- State Key Laboratory of Lunar and Planetary Sciences, Macau University of Science and Technology, Taipa, Macau, China
| | - Jingfang Chen
- Faculty of Medicine, Macau University of Science and Technology, Taipa, Macau, China
- Department of Research and Teaching, The Third People’s Hospital of Shenzhen, Shenzhen, China
| | - Jin Zeng
- The Institute for Sustainable Development, Macau University of Science and Technology, Taipa, Macau, China
- Guangzhou National Laboratory, Guangzhou, China
| | - Hong Jiang
- Faculty of Medicine, Macau University of Science and Technology, Taipa, Macau, China
- Statistical office, Zhuhai People’s Hospital (Zhuhai Clinical Medical College of Jinan University), Zhuhai, China
| | - Lin Ding
- Affiliated Hospital of Guangdong Medical University, Zhanjiang, China
| | - TianZi Zhang
- Department of Ophthalmology, Affiliated Hospital of Inner Mongolia University for the Nationalities, Tongliao, China
| | - Wen Sun
- Department of Ophthalmology Hainan Traditional Chinese Medicine Hospital, Haikou, China
| | - Rong Lu
- Yulin First People’s Hospital, Yulin, China
| | - Qiuli Zhang
- Affiliated Hospital of Guangdong Medical University, Zhanjiang, China
| | - Lizhong Liang
- Affiliated Hospital of Guangdong Medical University, Zhanjiang, China
- The Marine Biomedical Research Institute, School of Ocean and Tropical Medicine, Guangdong Medical University, Zhanjiang, Guangdong, China
| |
Collapse
|
36
|
Salloch S, Eriksen A. What Are Humans Doing in the Loop? Co-Reasoning and Practical Judgment When Using Machine Learning-Driven Decision Aids. THE AMERICAN JOURNAL OF BIOETHICS : AJOB 2024; 24:67-78. [PMID: 38767971 DOI: 10.1080/15265161.2024.2353800] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2024]
Abstract
Within the ethical debate on Machine Learning-driven decision support systems (ML_CDSS), notions such as "human in the loop" or "meaningful human control" are often cited as being necessary for ethical legitimacy. In addition, ethical principles usually serve as the major point of reference in ethical guidance documents, stating that conflicts between principles need to be weighed and balanced against each other. Starting from a neo-Kantian viewpoint inspired by Onora O'Neill, this article makes a concrete suggestion of how to interpret the role of the "human in the loop" and to overcome the perspective of rivaling ethical principles in the evaluation of AI in health care. We argue that patients should be perceived as "fellow workers" and epistemic partners in the interpretation of ML_CDSS outputs. We further highlight that a meaningful process of integrating (rather than weighing and balancing) ethical principles is most appropriate in the evaluation of medical AI.
Collapse
|
37
|
Zou X, He W, Huang Y, Ouyang Y, Zhang Z, Wu Y, Wu Y, Feng L, Wu S, Yang M, Chen X, Zheng Y, Jiang R, Chen T. AI-Driven Diagnostic Assistance in Medical Inquiry: Reinforcement Learning Algorithm Development and Validation. J Med Internet Res 2024; 26:e54616. [PMID: 39178403 PMCID: PMC11380057 DOI: 10.2196/54616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Revised: 04/13/2024] [Accepted: 07/04/2024] [Indexed: 08/25/2024] Open
Abstract
BACKGROUND For medical diagnosis, clinicians typically begin with a patient's chief concerns, followed by questions about symptoms and medical history, physical examinations, and requests for necessary auxiliary examinations to gather comprehensive medical information. This complex medical investigation process has yet to be modeled by existing artificial intelligence (AI) methodologies. OBJECTIVE The aim of this study was to develop an AI-driven medical inquiry assistant for clinical diagnosis that provides inquiry recommendations by simulating clinicians' medical investigating logic via reinforcement learning. METHODS We compiled multicenter, deidentified outpatient electronic health records from 76 hospitals in Shenzhen, China, spanning the period from July to November 2021. These records consisted of both unstructured textual information and structured laboratory test results. We first performed feature extraction and standardization using natural language processing techniques and then used a reinforcement learning actor-critic framework to explore the rational and effective inquiry logic. To align the inquiry process with actual clinical practice, we segmented the inquiry into 4 stages: inquiring about symptoms and medical history, conducting physical examinations, requesting auxiliary examinations, and terminating the inquiry with a diagnosis. External validation was conducted to validate the inquiry logic of the AI model. RESULTS This study focused on 2 retrospective inquiry-and-diagnosis tasks in the emergency and pediatrics departments. The emergency departments provided records of 339,020 consultations including mainly children (median age 5.2, IQR 2.6-26.1 years) with various types of upper respiratory tract infections (250,638/339,020, 73.93%). The pediatrics department provided records of 561,659 consultations, mainly of children (median age 3.8, IQR 2.0-5.7 years) with various types of upper respiratory tract infections (498,408/561,659, 88.73%). When conducting its own inquiries in both scenarios, the AI model demonstrated high diagnostic performance, with areas under the receiver operating characteristic curve of 0.955 (95% CI 0.953-0.956) and 0.943 (95% CI 0.941-0.944), respectively. When the AI model was used in a simulated collaboration with physicians, it notably reduced the average number of physicians' inquiries to 46% (6.037/13.26; 95% CI 6.009-6.064) and 43% (6.245/14.364; 95% CI 6.225-6.269) while achieving areas under the receiver operating characteristic curve of 0.972 (95% CI 0.970-0.973) and 0.968 (95% CI 0.967-0.969) in the scenarios. External validation revealed a normalized Kendall τ distance of 0.323 (95% CI 0.301-0.346), indicating the inquiry consistency of the AI model with physicians. CONCLUSIONS This retrospective analysis of predominantly respiratory pediatric presentations in emergency and pediatrics departments demonstrated that an AI-driven diagnostic assistant had high diagnostic performance both in stand-alone use and in simulated collaboration with clinicians. Its investigation process was found to be consistent with the clinicians' medical investigation logic. These findings highlight the diagnostic assistant's promise in assisting the decision-making processes of health care professionals.
Collapse
Affiliation(s)
- Xuan Zou
- Shenzhen Center for Disease Control and Prevention, Shenzhen, China
| | - Weijie He
- Department of Computer Science and Technology, Tsinghua University, Beijing, China
- Institute of Artificial Intelligence, Tsinghua University, Beijing, China
- Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China
| | - Yu Huang
- Jarvis Research Center, Tencent YouTu Lab, Shenzhen, China
| | - Yi Ouyang
- Jarvis Research Center, Tencent YouTu Lab, Shenzhen, China
| | - Zhen Zhang
- Shenzhen Center for Disease Control and Prevention, Shenzhen, China
| | - Yu Wu
- Shenzhen Center for Disease Control and Prevention, Shenzhen, China
| | - Yongsheng Wu
- Shenzhen Center for Disease Control and Prevention, Shenzhen, China
| | - Lili Feng
- Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua University, Beijing, China
| | - Sheng Wu
- Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua University, Beijing, China
| | | | - Xuyan Chen
- Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua University, Beijing, China
| | - Yefeng Zheng
- Jarvis Research Center, Tencent YouTu Lab, Shenzhen, China
| | - Rui Jiang
- Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China
- Department of Automation, Tsinghua University, Beijing, China
| | - Ting Chen
- Department of Computer Science and Technology, Tsinghua University, Beijing, China
- Institute of Artificial Intelligence, Tsinghua University, Beijing, China
- Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China
| |
Collapse
|
38
|
Sun L, Li J, Zeng S, Luo Q, Miao H, Liang Y, Cheng L, Sun Z, Tai WH, Han Y, Yin Y, Wu K, Zhang K. Artificial intelligence system for outcome evaluations of human in vitro fertilization-derived embryos. Chin Med J (Engl) 2024; 137:1939-1949. [PMID: 38997251 PMCID: PMC11332789 DOI: 10.1097/cm9.0000000000003162] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Indexed: 07/14/2024] Open
Abstract
BACKGROUND In vitro fertilization (IVF) has emerged as a transformative solution for infertility. However, achieving favorable live-birth outcomes remains challenging. Current clinical IVF practices in IVF involve the collection of heterogeneous embryo data through diverse methods, including static images and temporal videos. However, traditional embryo selection methods, primarily reliant on visual inspection of morphology, exhibit variability and are contingent on the experience of practitioners. Therefore, an automated system that can evaluate heterogeneous embryo data to predict the final outcomes of live births is highly desirable. METHODS We employed artificial intelligence (AI) for embryo morphological grading, blastocyst embryo selection, aneuploidy prediction, and final live-birth outcome prediction. We developed and validated the AI models using multitask learning for embryo morphological assessment, including pronucleus type on day 1 and the number of blastomeres, asymmetry, and fragmentation of blastomeres on day 3, using 19,201 embryo photographs from 8271 patients. A neural network was trained on embryo and clinical metadata to identify good-quality embryos for implantation on day 3 or day 5, and predict live-birth outcomes. Additionally, a 3D convolutional neural network was trained on 418 time-lapse videos of preimplantation genetic testing (PGT)-based ploidy outcomes for the prediction of aneuploidy and consequent live-birth outcomes. RESULTS These two approaches enabled us to automatically assess the implantation potential. By combining embryo and maternal metrics in an ensemble AI model, we evaluated live-birth outcomes in a prospective cohort that achieved higher accuracy than experienced embryologists (46.1% vs. 30.7% on day 3, 55.0% vs. 40.7% on day 5). Our results demonstrate the potential for AI-based selection of embryos based on characteristics beyond the observational abilities of human clinicians (area under the curve: 0.769, 95% confidence interval: 0.709-0.820). These findings could potentially provide a noninvasive, high-throughput, and low-cost screening tool to facilitate embryo selection and achieve better outcomes. CONCLUSIONS Our study underscores the AI model's ability to provide interpretable evidence for clinicians in assisted reproduction, highlighting its potential as a noninvasive, efficient, and cost-effective tool for improved embryo selection and enhanced IVF outcomes. The convergence of cutting-edge technology and reproductive medicine has opened new avenues for addressing infertility challenges and optimizing IVF success rates.
Collapse
Affiliation(s)
- Ling Sun
- Department of Reproductive Medicine, Guangzhou Women and Children’s Medical Center, Guangzhou Medical University, Guangzhou, Guangdong 510623, China
| | - Jiahui Li
- Department of Reproductive Medicine, Guangzhou Women and Children’s Medical Center, Guangzhou Medical University, Guangzhou, Guangdong 510623, China
- Guangzhou National Laboratory, Guangzhou, Guangdong 510000, China
| | - Simiao Zeng
- Department of Reproductive Medicine, Guangzhou Women and Children’s Medical Center, Guangzhou Medical University, Guangzhou, Guangdong 510623, China
- Guangzhou National Laboratory, Guangzhou, Guangdong 510000, China
| | - Qiangxiang Luo
- Department of Reproductive Medicine, Jiangmen Central Hospital, Jiangmen, Guangdong 529000, China
| | - Hanpei Miao
- Guangzhou National Laboratory, Guangzhou, Guangdong 510000, China
- Department of Ophthalmology, Dongguan People’s Hospital, The First School of Clinical Medicine, Southern Medical University, Dongguan, Guangdong 523000, China
| | - Yunhao Liang
- Department of Reproductive Medicine, Guangzhou Women and Children’s Medical Center, Guangzhou Medical University, Guangzhou, Guangdong 510623, China
| | - Linling Cheng
- Institute for Artificial Intelligence in Medicine, Faculty of Medicine, Macau University of Science and Technology, Macau Special Administrative Region 999078, China
| | - Zhuo Sun
- Wenzhou Eye Hospital, Institute for Advanced Studies on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Wa Hou Tai
- Institute for Artificial Intelligence in Medicine, Faculty of Medicine, Macau University of Science and Technology, Macau Special Administrative Region 999078, China
| | - Yibing Han
- Kiang Wu Hospital, Macau Special Administrative Region 999078, China
| | - Yun Yin
- Faculty of Business, City University of Macau, Macau Special Administrative Region 999078, China
| | - Keliang Wu
- State Key Laboratory of Reproductive Medicine and Offspring Health, Center for Reproductive Medicine, Institute of Women, Children and Reproductive Health and Key laboratory of Reproductive Endocrinology of Ministry of Education, Shandong University, Jinan, Shandong 250000,China
| | - Kang Zhang
- Department of Reproductive Medicine, Guangzhou Women and Children’s Medical Center, Guangzhou Medical University, Guangzhou, Guangdong 510623, China
- Guangzhou National Laboratory, Guangzhou, Guangdong 510000, China
- Institute for Artificial Intelligence in Medicine, Faculty of Medicine, Macau University of Science and Technology, Macau Special Administrative Region 999078, China
- Wenzhou Eye Hospital, Institute for Advanced Studies on Eye Health and Diseases, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| |
Collapse
|
39
|
Ren J, Wang H, Lai S, Shao Y, Che H, Xue Z, Qi X, Zhang S, Dai J, Wang S, Li K, Gan W, Si Q. Machine learning-based model to predict composite thromboembolic events among Chinese elderly patients with atrial fibrillation. BMC Cardiovasc Disord 2024; 24:420. [PMID: 39134969 PMCID: PMC11321189 DOI: 10.1186/s12872-024-04082-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2023] [Accepted: 07/30/2024] [Indexed: 08/15/2024] Open
Abstract
OBJECTIVE Accurate prediction of survival prognosis is helpful to guide clinical decision-making. The aim of this study was to develop a model using machine learning techniques to predict the occurrence of composite thromboembolic events (CTEs) in elderly patients with atrial fibrillation(AF). These events encompass newly diagnosed cerebral ischemia events, cardiovascular events, pulmonary embolism, and lower extremity arterial embolism. METHODS This retrospective study included 6,079 elderly hospitalized patients (≥ 75 years old) with AF admitted to the People's Liberation Army General Hospital in China from January 2010 to June 2022. Random forest imputation was used for handling missing data. In the descriptive statistics section, patients were divided into two groups based on the occurrence of CTEs, and differences between the two groups were analyzed using chi-square tests for categorical variables and rank-sum tests for continuous variables. In the machine learning section, the patients were randomly divided into a training dataset (n = 4,225) and a validation dataset (n = 1,824) in a 7:3 ratio. Four machine learning models (logistic regression, decision tree, random forest, XGBoost) were trained on the training dataset and validated on the validation dataset. RESULTS The incidence of composite thromboembolic events was 19.53%. The Least Absolute Shrinkage and Selection Operator (LASSO) method, using 5-fold cross-validation, was applied to the training dataset and identified a total of 18 features that exhibited a significant association with the occurrence of CTEs. The random forest model outperformed other models in terms of area under the curve (ACC: 0.9144, SEN: 0.7725, SPE: 0.9489, AUC: 0.927, 95% CI: 0.9105-0.9443). The random forest model also showed good clinical validity based on the clinical decision curve. The Shapley Additive exPlanations (SHAP) showed that the top five features associated with the model were history of ischemic stroke, high triglyceride (TG), high total cholesterol (TC), high plasma D-dimer, age. CONCLUSIONS This study proposes an accurate model to stratify patients with a high risk of CTEs. The random forest model has good performance. History of ischemic stroke, age, high TG, high TC and high plasma D-Dimer may be correlated with CTEs.
Collapse
Affiliation(s)
- Jiefeng Ren
- Department of Geriatric Cardiology, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
- Medical School of Chinese PLA, Beijing, 100853, China
| | - Haijun Wang
- Department of Geriatric Cardiology, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
| | - Song Lai
- Department of the Third Health Care, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
| | - Yi Shao
- Health Management Center, Shandong Provincial Qianfoshan Hospital, The First Affiliated Hospital of Shandong First Medical University, Jinan, 250012, Shandong, China
| | - Hebin Che
- Medical Big Data Research Center, Chinese PLA General Hospital, Fuxing Road 28#, Haidian district, Beijing, 100853, China
| | - Zaiyao Xue
- Department of Geriatric Cardiology, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
- Medical School of Chinese PLA, Beijing, 100853, China
| | - Xinlian Qi
- Department of Geriatric Cardiology, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
- Medical School of Chinese PLA, Beijing, 100853, China
| | - Sha Zhang
- Department of Geriatric Cardiology, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
- Medical School of Chinese PLA, Beijing, 100853, China
| | - Jinkun Dai
- Department of Geriatric Cardiology, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
- Beijing Goodwill Hessian Health Technology, Dongcheng District, Beijing, 100007, China
| | - Sai Wang
- Department of Geriatric Cardiology, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
- Beijing Goodwill Hessian Health Technology, Dongcheng District, Beijing, 100007, China
| | - Kunlian Li
- Department of Geriatric Cardiology, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
- Beijing Goodwill Hessian Health Technology, Dongcheng District, Beijing, 100007, China
| | - Wei Gan
- Department of Geriatric Cardiology, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China
- Beijing Goodwill Hessian Health Technology, Dongcheng District, Beijing, 100007, China
| | - Quanjin Si
- Department of the Third Health Care, National Clinical Research Center for Geriatric Diseases, Second Medical Center of Chinese PLA General Hospital, Beijing, 100853, China.
| |
Collapse
|
40
|
Huang YZ, Chen YM, Lin CC, Chiu HY, Chang YC. A nursing note-aware deep neural network for predicting mortality risk after hospital discharge. Int J Nurs Stud 2024; 156:104797. [PMID: 38788263 DOI: 10.1016/j.ijnurstu.2024.104797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Revised: 04/08/2024] [Accepted: 05/03/2024] [Indexed: 05/26/2024]
Abstract
BACKGROUND ICU readmissions and post-discharge mortality pose significant challenges. Previous studies used EHRs and machine learning models, but mostly focused on structured data. Nursing records contain crucial unstructured information, but their utilization is challenging. Natural language processing (NLP) can extract structured features from clinical text. This study proposes the Crucial Nursing Description Extractor (CNDE) to predict post-ICU discharge mortality rates and identify high-risk patients for unplanned readmission by analyzing electronic nursing records. OBJECTIVE Developed a deep neural network (NurnaNet) with the ability to perceive nursing records, combined with a bio-clinical medicine pre-trained language model (BioClinicalBERT) to analyze the electronic health records (EHRs) in the MIMIC III dataset to predict the death of patients within six month and two year risk. DESIGN A cohort and system development design was used. SETTING(S) Based on data extracted from MIMIC-III, a database of critically ill in the US between 2001 and 2012, the results were analyzed. PARTICIPANTS We calculated patients' age using admission time and date of birth information from the MIMIC dataset. Patients under 18 or over 89 years old, or who died in the hospital, were excluded. We analyzed 16,973 nursing records from patients' ICU stays. METHODS We have developed a technology called the Crucial Nursing Description Extractor (CNDE), which extracts key content from text. We use the logarithmic likelihood ratio to extract keywords and combine BioClinicalBERT. We predict the survival of discharged patients after six months and two years and evaluate the performance of the model using precision, recall, the F1-score, the receiver operating characteristic curve (ROC curve), the area under the curve (AUC), and the precision-recall curve (PR curve). RESULTS The research findings indicate that NurnaNet achieved good F1-scores (0.67030, 0.70874) within six months and two years. Compared to using BioClinicalBERT alone, there was an improvement in performance of 2.05 % and 1.08 % for predictions within six months and two years, respectively. CONCLUSIONS CNDE can effectively reduce long-form records and extract key content. NurnaNet has a good F1-score in analyzing the data of nursing records, which helps to identify the risk of death of patients after leaving the hospital and adjust the regular follow-up and treatment plan of relevant medical care as soon as possible.
Collapse
Affiliation(s)
- Yong-Zhen Huang
- Graduate Institute of Data Science, College of Management, Taipei Medical University, Taipei, Taiwan; Department of Nursing, National Taiwan University Cancer Center, Taipei, Taiwan.
| | - Yan-Ming Chen
- Graduate Institute of Data Science, College of Management, Taipei Medical University, Taipei, Taiwan.
| | - Chih-Cheng Lin
- Graduate Institute of Data Science, College of Management, Taipei Medical University, Taipei, Taiwan.
| | - Hsiao-Yean Chiu
- School of Nursing, College of Nursing, Taipei Medical University, Taipei, Taiwan; Department of Nursing, Taipei Medical University Hospital, Taipei, Taiwan; Research Center of Sleep Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan.
| | - Yung-Chun Chang
- Graduate Institute of Data Science, College of Management, Taipei Medical University, Taipei, Taiwan; Clinical Big Data Research Center, Taipei Medical University Hospital, Taipei, Taiwan.
| |
Collapse
|
41
|
Wang L, Wu YH, Ren Y, Sun FF, Tao SH, Lin HX, Zhang CS, Tang W, Chen ZG, Chen C, Zhang LD. Establishment and Verification of an Artificial Intelligence Prediction Model for Children With Sepsis. Pediatr Infect Dis J 2024; 43:736-742. [PMID: 38717173 DOI: 10.1097/inf.0000000000004376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 07/19/2024]
Abstract
BACKGROUND Early identification of high-risk groups of children with sepsis is beneficial to reduce sepsis mortality. This article used artificial intelligence (AI) technology to predict the risk of death effectively and quickly in children with sepsis in the pediatric intensive care unit (PICU). STUDY DESIGN This retrospective observational study was conducted in the PICUs of the First Affiliated Hospital of Sun Yat-sen University from December 2016 to June 2019 and Shenzhen Children's Hospital from January 2019 to July 2020. The children were divided into a death group and a survival group. Different machine language (ML) models were used to predict the risk of death in children with sepsis. RESULTS A total of 671 children with sepsis were enrolled. The accuracy (ACC) of the artificial neural network model was better than that of support vector machine, logical regression analysis, Bayesian, K nearest neighbor method and decision tree models, with a training set ACC of 0.99 and a test set ACC of 0.96. CONCLUSIONS The AI model can be used to predict the risk of death due to sepsis in children in the PICU, and the artificial neural network model is better than other AI models in predicting mortality risk.
Collapse
Affiliation(s)
- Li Wang
- From the Pediatric Intensive Care Unit, The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, Guangdong, China
| | - Yu-Hui Wu
- Pediatric Intensive Care Unit, Shenzhen Children's Hospital, Shenzhen, Guangdong, China
| | - Yong Ren
- Scientific Research Project Department, Guangdong Artificial Intelligence and Digital Economy Laboratory (Guangzhou), Pazhou Lab, Guangzhou, Guangdong, China
- Shensi lab, Shenzhen Institute for Advanced Study, UESTC, Shenzhen, Guangdong, China
- Center for Digestive Disease, The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, Guangdong, China
| | - Fan-Fan Sun
- From the Pediatric Intensive Care Unit, The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, Guangdong, China
| | - Shao-Hua Tao
- Pediatric Intensive Care Unit, Zhujiang Hospital of Southern Medical University, Guangzhou, Guangdong, China
| | - Hong-Xin Lin
- From the Pediatric Intensive Care Unit, The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, Guangdong, China
| | - Chuang-Sen Zhang
- From the Pediatric Intensive Care Unit, The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, Guangdong, China
| | - Wen Tang
- Pediatric Intensive Care Unit, The First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, Guangdong, China
| | - Zhuang-Gui Chen
- Pediatric Intensive Care Unit, The Third Affiliated Hospital of Sun Yat-Sen University, Guangzhou, Guangdong, China
| | - Chun Chen
- Pediatric Hematology Laboratory, Division of Hematology/Oncology, Department of Pediatrics, The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, Guangdong, China
| | - Li-Dan Zhang
- From the Pediatric Intensive Care Unit, The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, Guangdong, China
| |
Collapse
|
42
|
Benzinger L, Epping J, Ursin F, Salloch S. Artificial Intelligence to support ethical decision-making for incapacitated patients: a survey among German anesthesiologists and internists. BMC Med Ethics 2024; 25:78. [PMID: 39026308 PMCID: PMC11256615 DOI: 10.1186/s12910-024-01079-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2024] [Accepted: 07/03/2024] [Indexed: 07/20/2024] Open
Abstract
BACKGROUND Artificial intelligence (AI) has revolutionized various healthcare domains, where AI algorithms sometimes even outperform human specialists. However, the field of clinical ethics has remained largely untouched by AI advances. This study explores the attitudes of anesthesiologists and internists towards the use of AI-driven preference prediction tools to support ethical decision-making for incapacitated patients. METHODS A questionnaire was developed and pretested among medical students. The questionnaire was distributed to 200 German anesthesiologists and 200 German internists, thereby focusing on physicians who often encounter patients lacking decision-making capacity. The questionnaire covered attitudes toward AI-driven preference prediction, availability and utilization of Clinical Ethics Support Services (CESS), and experiences with ethically challenging situations. Descriptive statistics and bivariate analysis was performed. Qualitative responses were analyzed using content analysis in a mixed inductive-deductive approach. RESULTS Participants were predominantly male (69.3%), with ages ranging from 27 to 77. Most worked in nonacademic hospitals (82%). Physicians generally showed hesitance toward AI-driven preference prediction, citing concerns about the loss of individuality and humanity, lack of explicability in AI results, and doubts about AI's ability to encompass the ethical deliberation process. In contrast, physicians had a more positive opinion of CESS. Availability of CESS varied, with 81.8% of participants reporting access. Among those without access, 91.8% expressed a desire for CESS. Physicians' reluctance toward AI-driven preference prediction aligns with concerns about transparency, individuality, and human-machine interaction. While AI could enhance the accuracy of predictions and reduce surrogate burden, concerns about potential biases, de-humanisation, and lack of explicability persist. CONCLUSIONS German physicians frequently encountering incapacitated patients exhibit hesitance toward AI-driven preference prediction but hold a higher esteem for CESS. Addressing concerns about individuality, explicability, and human-machine roles may facilitate the acceptance of AI in clinical ethics. Further research into patient and surrogate perspectives is needed to ensure AI aligns with patient preferences and values in complex medical decisions.
Collapse
Affiliation(s)
- Lasse Benzinger
- Institute for Ethics, History and Philosophy of Medicine, Hannover Medical School (MHH), Carl-Neuberg-Str. 1, Hannover, 30625, Germany.
| | - Jelena Epping
- Department of Medical Sociology, Hannover Medical School (MHH), Carl-Neuberg-Str. 1, Hannover, 30625, Germany
| | - Frank Ursin
- Institute for Ethics, History and Philosophy of Medicine, Hannover Medical School (MHH), Carl-Neuberg-Str. 1, Hannover, 30625, Germany
| | - Sabine Salloch
- Institute for Ethics, History and Philosophy of Medicine, Hannover Medical School (MHH), Carl-Neuberg-Str. 1, Hannover, 30625, Germany
| |
Collapse
|
43
|
Liu X, Fang M, Wang K, Zhu J, Chen Z, He L, Liang S, Deng Y, Chen C. Machine learning-based model to predict severe acute kidney injury after total aortic arch replacement for acute type A aortic dissection. Heliyon 2024; 10:e34171. [PMID: 39071670 PMCID: PMC11280131 DOI: 10.1016/j.heliyon.2024.e34171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 06/26/2024] [Accepted: 07/04/2024] [Indexed: 07/30/2024] Open
Abstract
Background Severe acute kidney injury (AKI) after total aortic arch replacement (TAAR) is related to adverse outcomes in patients with acute type A aortic dissection (ATAAD). However, the early prediction of severe AKI remains a challenge. This study aimed to develop a novel model to predict severe AKI after TAAR in ATAAD patients using machine learning algorithms. Methods A total of 572 ATAAD patients undergoing TAAR were enrolled in this retrospective study, and randomly divided into a training set (70 %) and a validation set (30 %). Lasso regression, support vector machine-recursive feature elimination and random forest algorithms were used to screen indicators for severe AKI (defined as AKI stage III) in the training set, respectively. Then the intersection indicators were selected to construct models through artificial neural network (ANN) and logistic regression. The AUC-ROC curve was employed to ascertain the prediction efficacy of the ANN and logistic regression models. Results The incidence of severe AKI after TAAR was 22.9 % among ATAAD patients. The intersection predictors identified by different machine learning algorithms were baseline serum creatinine and ICU admission variables, including serum cystatin C, procalcitonin, aspartate transaminase, platelet, lactic dehydrogenase, urine N-acetyl-β-d-glucosidase and Acute Physiology and Chronic Health Evaluation II score. The ANN model showed a higher AUC-ROC than logistic regression (0.938 vs 0.908, p < 0.05). Furthermore, the ANN model could predict 89.1 % of severe AKI cases beforehand. In the validation set, the superior performance of the ANN model was further confirmed in terms of discrimination ability (AUC = 0.916), calibration curve analysis and decision curve analysis. Conclusion This study developed a novel and reliable clinical prediction model for severe AKI after TAAR in ATAAD patients using machine learning algorithms. Importantly, the ANN model showed a higher predictive ability for severe AKI than logistic regression.
Collapse
Affiliation(s)
- Xiaolong Liu
- Department of Critical Care Medicine, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, China
| | - Miaoxian Fang
- Department of Intensive Care Unit of Cardiac Surgery, Guangdong Cardiovascular Institute, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, China
| | - Kai Wang
- Department of Critical Care Medicine, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, China
- School of Medicine, South China University of Technology, Guangzhou, 510000, China
| | - Junjiang Zhu
- Department of Critical Care Medicine, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, China
| | - Zeling Chen
- Department of Critical Care Medicine, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, China
| | - Linling He
- Department of Critical Care Medicine, Shenzhen People's Hospital, The Second Clinical Medical College of Jinan University, The First Affiliated Hospital of Southern University of Science and Technology, Shenzhen, 518020, China
| | - Silin Liang
- Department of Critical Care Medicine, Shenzhen People's Hospital, The Second Clinical Medical College of Jinan University, The First Affiliated Hospital of Southern University of Science and Technology, Shenzhen, 518020, China
| | - Yiyu Deng
- Department of Critical Care Medicine, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, China
| | - Chunbo Chen
- Department of Intensive Care Unit of Cardiac Surgery, Guangdong Cardiovascular Institute, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, China
- Department of Emergency, Shenzhen People's Hospital, The Second Clinical Medical College of Jinan University, The First Affiliated Hospital of Southern University of Science and Technology, Shenzhen, 518020, China
| |
Collapse
|
44
|
Tang BH, Zhang XF, Fu SM, Yao BF, Zhang W, Wu YE, Zheng Y, Zhou Y, van den Anker J, Huang HR, Hao GX, Zhao W. Machine Learning Approach in Dosage Individualization of Isoniazid for Tuberculosis. Clin Pharmacokinet 2024; 63:1055-1063. [PMID: 38990504 DOI: 10.1007/s40262-024-01400-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/01/2024] [Indexed: 07/12/2024]
Abstract
INTRODUCTION Isoniazid is a first-line antituberculosis agent with high variability, which would profit from individualized dosing. Concentrations of isoniazid at 2 h (C2h), as an indicator of safety and efficacy, are important for optimizing therapy. OBJECTIVE The objective of this study was to establish machine learning (ML) models to predict the C2h, that can be used for establishing an individualized dosing regimen in clinical practice. METHODS Published population pharmacokinetic (PopPK) models for adults were searched based on PubMed and ultimately four reliable models were selected for simulating individual C2h datasets under different conditions (demographics, genotype, ethnicity, etc.). Machine learning models were trained on simulated C2h obtained from the four PopPK models. Five different algorithms were used for ML model building to predict C2h. Real-world data were used for predictive performance evaluations. Virtual trials were used to compare ML-optimized doses with PopPK model-optimized doses. RESULTS Categorical boosting (CatBoost) exhibited the highest prediction ability. Target C2h can be predicted using the ML model combined with the dosing regimen and three covariates (N-acetyltransferase 2 [NAT2] genotypes, weight and race [Asians and Africans]). Real-world data validation results showed that the ML model can achieve an overall prediction accuracy of 93.4%. Using the final ML model, the mean absolute prediction error value decreased by 45.7% relative to the average of PopPK models. Using the ML-optimized dosing regimen, the probability of target attainment increased by 43.7% relative to the PopPK model-optimized dosing regimens. CONCLUSION Machine learning models were developed with great predictive performance, which can be used to determine the individualized initial dose of isoniazid in adult patients.
Collapse
Affiliation(s)
- Bo-Hao Tang
- Department of Pharmacy, The Second Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Xin-Fang Zhang
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Shu-Meng Fu
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Bu-Fan Yao
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Wei Zhang
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Yue-E Wu
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Yi Zheng
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Yue Zhou
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - John van den Anker
- Division of Clinical Pharmacology, Children's National Hospital, Washington, DC, USA
- Departments of Pediatrics, Pharmacology and Physiology, Genomics and Precision Medicine, The George Washington University School of Medicine and Health Sciences, Washington, DC, USA
- Department of Pediatric Pharmacology and Pharmacometrics, University of Basel Children's Hospital, Basel, Switzerland
| | - Hai-Rong Huang
- National Clinical Laboratory on Tuberculosis, Beijing Key Laboratory on Drug-Resistant Tuberculosis, Beijing Chest Hospital, Beijing Tuberculosis and Thoracic Tumor Research Institute, Capital Medical University, Beijing, China
| | - Guo-Xiang Hao
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Wei Zhao
- Department of Pharmacy, The Second Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China.
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education), NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China.
- NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, Qilu Hospital of Shandong University, Shandong University, Jinan, China.
| |
Collapse
|
45
|
Sharma CM, Chariar VM. Diagnosis of mental disorders using machine learning: Literature review and bibliometric mapping from 2012 to 2023. Heliyon 2024; 10:e32548. [PMID: 38975193 PMCID: PMC11225745 DOI: 10.1016/j.heliyon.2024.e32548] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2024] [Revised: 06/05/2024] [Accepted: 06/05/2024] [Indexed: 07/09/2024] Open
Abstract
Background Mental disorders (MDs) are becoming a leading burden in non-communicable diseases (NCDs). As per the World Health Organization's 2022 assessment report, there was a steep increase of 25 % in MDs during the COVID-19 pandemic. Early diagnosis of MDs can significantly improve treatment outcome and save disability-adjusted life years (DALYs). In recent times, the application of machine learning (ML) and deep learning (DL)) has shown promising results in the diagnosis of MDs, and the field has witnessed a huge research output in the form of research publications. Therefore, a bibliometric mapping along with a review of recent advancements is required. Methods This study presents a bibliometric analysis and review of the research, published over the last 10 years. Literature searches were conducted in the Scopus database for the period from January 1, 2012, to June 9, 2023. The data was filtered and screened to include only relevant and reliable publications. A total of 2811 journal articles were found. The data was exported to a comma-separated value (CSV) format for further analysis. Furthermore, a review of 40 selected studies was performed. Results The popularity of ML techniques in diagnosing MDs has been growing, with an annual research growth rate of 17.05 %. The Journal of Affective Disorders published the most documents (n = 97), while Wang Y. (n = 64) has published the most articles. Lotka's law is observed, with a minority of authors contributing the majority of publications. The top affiliating institutes are the West China Hospital of Sichuan University followed by the University of California, with China and the US dominating the top 10 institutes. While China has more publications, papers affiliated with the US receive more citations. Depression and schizophrenia are the primary focuses of ML and deep learning (DL) in mental disease detection. Co-occurrence network analysis reveals that ML is associated with depression, schizophrenia, autism, anxiety, ADHD, obsessive-compulsive disorder, and PTSD. Popular algorithms include support vector machine (SVM) classifier, decision tree classifier, and random forest classifier. Furthermore, DL is linked to neuroimaging techniques such as MRI, fMRI, and EEG, as well as bipolar disorder. Current research trends encompass DL, LSTM, generalized anxiety disorder, feature fusion, and convolutional neural networks.
Collapse
Affiliation(s)
- Chandra Mani Sharma
- CRDT, Indian Institute of Technology Delhi, Hauz Khas, New Delhi, 110016, India
- School of Computer Science, UPES, Dehradun, Uttarakhand, India
| | | |
Collapse
|
46
|
Wang Y, Fu W, Zhang Y, Wang D, Gu Y, Wang W, Xu H, Ge X, Ye C, Fang J, Su L, Wang J, He W, Zhang X, Feng R. Constructing and implementing a performance evaluation indicator set for artificial intelligence decision support systems in pediatric outpatient clinics: an observational study. Sci Rep 2024; 14:14482. [PMID: 38914707 PMCID: PMC11196575 DOI: 10.1038/s41598-024-64893-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Accepted: 06/13/2024] [Indexed: 06/26/2024] Open
Abstract
Artificial intelligence (AI) decision support systems in pediatric healthcare have a complex application background. As an AI decision support system (AI-DSS) can be costly, once applied, it is crucial to focus on its performance, interpret its success, and then monitor and update it to ensure ongoing success consistently. Therefore, a set of evaluation indicators was explicitly developed for AI-DSS in pediatric healthcare, enabling continuous and systematic performance monitoring. The study unfolded in two stages. The first stage encompassed establishing the evaluation indicator set through a literature review, a focus group interview, and expert consultation using the Delphi method. In the second stage, weight analysis was conducted. Subjective weights were calculated based on expert opinions through analytic hierarchy process, while objective weights were determined using the entropy weight method. Subsequently, subject and object weights were synthesized to form the combined weight. In the two rounds of expert consultation, the authority coefficients were 0.834 and 0.846, Kendall's coordination coefficient was 0.135 in Round 1 and 0.312 in Round 2. The final evaluation indicator set has three first-class indicators, fifteen second-class indicators, and forty-seven third-class indicators. Indicator I-1(Organizational performance) carries the highest weight, followed by Indicator I-2(Societal performance) and Indicator I-3(User experience performance) in the objective and combined weights. Conversely, 'Societal performance' holds the most weight among the subjective weights, followed by 'Organizational performance' and 'User experience performance'. In this study, a comprehensive and specialized set of evaluation indicators for the AI-DSS in the pediatric outpatient clinic was established, and then implemented. Continuous evaluation still requires long-term data collection to optimize the weight proportions of the established indicators.
Collapse
Affiliation(s)
- Yingwen Wang
- Nursing Department, Children's Hospital of Fudan University, Shanghai, 201102, China
| | - Weijia Fu
- Medical Information Center, Children's Hospital of Fudan University, Shanghai, 201102, China
| | - Yuejie Zhang
- School of Computer Science, Fudan University, Shanghai, 200438, China
| | - Daoyang Wang
- School of Public, Health Fudan University, Shanghai, 200032, China
| | - Ying Gu
- Nursing Department, Children's Hospital of Fudan University, Shanghai, 201102, China
| | - Weibing Wang
- School of Public, Health Fudan University, Shanghai, 200032, China
| | - Hong Xu
- Nephrology Department, Children's Hospital of Fudan University, Shanghai, 201102, China
| | - Xiaoling Ge
- Statistical and Data Management Center, Children's Hospital of Fudan University, Shanghai, 201102, China
| | - Chengjie Ye
- Medical Information Center, Children's Hospital of Fudan University, Shanghai, 201102, China
| | - Jinwu Fang
- School of Public, Health Fudan University, Shanghai, 200032, China
| | - Ling Su
- Statistical and Data Management Center, Children's Hospital of Fudan University, Shanghai, 201102, China
| | - Jiayu Wang
- National Health Commission Key Laboratory of Neonatal Diseases (Fudan University), Children's Hospital of Fudan University, Shanghai, 201102, China
| | - Wen He
- Respiratory Department, Children's Hospital of Fudan University, Shanghai, 201102, China
| | - Xiaobo Zhang
- Respiratory Department, Children's Hospital of Fudan University, Shanghai, 201102, China.
| | - Rui Feng
- School of Computer Science, Fudan University, Shanghai, 200438, China.
- School of Computer Science, Fudan University, 2005 Songhu Road, Shanghai, 200438, China.
| |
Collapse
|
47
|
Sengupta S, Rao R, Kaufman Z, Stuhlmiller TJ, Wong KK, Kesari S, Shapiro MA, Kramer GA. A Health Care Clinical Data Platform for Rapid Deployment of Artificial Intelligence and Machine Learning Algorithms for Cancer Care and Oncology Clinical Trials. N C Med J 2024; 85:270-273. [PMID: 39466099 DOI: 10.18043/001c.120572] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/29/2024]
Abstract
The xCures platform aggregates, organizes, structures, and normalizes clinical EMR data across care sites, utilizing advanced technologies for near real-time access. The platform generates data in a format to support clinical care, accelerate research, and promote artificial intelligence/ machine learning algorithm development, highlighted by a clinical decision support algorithm for precision oncology.
Collapse
Affiliation(s)
- Soma Sengupta
- Department of Neurosurgery, School of Medicine, University of North Carolina at Chapel Hill
| | - Rohan Rao
- Ronald Reagan UCLA Medical Center, University of California, Los Angeles
| | | | | | | | - Santosh Kesari
- Department of Translational Neurosciences, Saint John's Cancer Institute, Saint John's Health Center, Santa Monica, CA
| | | | | |
Collapse
|
48
|
Lee VV, van der Lubbe SCC, Goh LH, Valderas JM. Harnessing ChatGPT for Thematic Analysis: Are We Ready? J Med Internet Res 2024; 26:e54974. [PMID: 38819896 PMCID: PMC11179012 DOI: 10.2196/54974] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Revised: 02/28/2024] [Accepted: 03/20/2024] [Indexed: 06/01/2024] Open
Abstract
ChatGPT (OpenAI) is an advanced natural language processing tool with growing applications across various disciplines in medical research. Thematic analysis, a qualitative research method to identify and interpret patterns in data, is one application that stands to benefit from this technology. This viewpoint explores the use of ChatGPT in three core phases of thematic analysis within a medical context: (1) direct coding of transcripts, (2) generating themes from a predefined list of codes, and (3) preprocessing quotes for manuscript inclusion. Additionally, we explore the potential of ChatGPT to generate interview transcripts, which may be used for training purposes. We assess the strengths and limitations of using ChatGPT in these roles, highlighting areas where human intervention remains necessary. Overall, we argue that ChatGPT can function as a valuable tool during analysis, enhancing the efficiency of the thematic analysis and offering additional insights into the qualitative data. While ChatGPT may not adequately capture the full context of each participant, it can serve as an additional member of the analysis team, contributing to researcher triangulation through knowledge building and sensemaking.
Collapse
Affiliation(s)
- V Vien Lee
- Division of Family Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Stephanie C C van der Lubbe
- Division of Family Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Lay Hoon Goh
- Division of Family Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- Department of Family Medicine, National University Health System, Singapore, Singapore
| | - Jose Maria Valderas
- Division of Family Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- Department of Family Medicine, National University Health System, Singapore, Singapore
- Centre for Research in Health Systems Performance, National University of Singapore, Singapore, Singapore
| |
Collapse
|
49
|
Roberts K, Chin AT, Loewy K, Pompeii L, Shin H, Rider NL. Natural language processing of clinical notes enables early inborn error of immunity risk ascertainment. THE JOURNAL OF ALLERGY AND CLINICAL IMMUNOLOGY. GLOBAL 2024; 3:100224. [PMID: 38439946 PMCID: PMC10910118 DOI: 10.1016/j.jacig.2024.100224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 12/24/2023] [Accepted: 01/21/2024] [Indexed: 03/06/2024]
Abstract
Background There are now approximately 450 discrete inborn errors of immunity (IEI) described; however, diagnostic rates remain suboptimal. Use of structured health record data has proven useful for patient detection but may be augmented by natural language processing (NLP). Here we present a machine learning model that can distinguish patients from controls significantly in advance of ultimate diagnosis date. Objective We sought to create an NLP machine learning algorithm that could identify IEI patients early during the disease course and shorten the diagnostic odyssey. Methods Our approach involved extracting a large corpus of IEI patient clinical-note text from a major referral center's electronic health record (EHR) system and a matched control corpus for comparison. We built text classifiers with simple machine learning methods and trained them on progressively longer time epochs before date of diagnosis. Results The top performing NLP algorithm effectively distinguished cases from controls robustly 36 months before ultimate clinical diagnosis (area under precision recall curve > 0.95). Corpus analysis demonstrated that statistically enriched, IEI-relevant terms were evident 24+ months before diagnosis, validating that clinical notes can provide a signal for early prediction of IEI. Conclusion Mining EHR notes with NLP holds promise for improving early IEI patient detection.
Collapse
Affiliation(s)
- Kirk Roberts
- McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Tex
| | - Aaron T. Chin
- Division of Immunology, Allergy, and Rheumatology, University of California, Los Angeles, Calif
| | | | - Lisa Pompeii
- Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio
| | - Harold Shin
- College of Osteopathic Medicine, Liberty University, Lynchburg, Va
| | - Nicholas L. Rider
- Division of Health System & Implementation Science, Virginia Tech Carilion School of Medicine, Roanoke, Va
- Section of Allergy and Immunology, Carilion Clinic, Roanoke, Va
| |
Collapse
|
50
|
Nunez JJ, Leung B, Ho C, Ng RT, Bates AT. Predicting which patients with cancer will see a psychiatrist or counsellor from their initial oncology consultation document using natural language processing. COMMUNICATIONS MEDICINE 2024; 4:69. [PMID: 38589545 PMCID: PMC11001970 DOI: 10.1038/s43856-024-00495-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2023] [Accepted: 03/28/2024] [Indexed: 04/10/2024] Open
Abstract
BACKGROUND Patients with cancer often have unmet psychosocial needs. Early detection of who requires referral to a counsellor or psychiatrist may improve their care. This work used natural language processing to predict which patients will see a counsellor or psychiatrist from a patient's initial oncology consultation document. We believe this is the first use of artificial intelligence to predict psychiatric outcomes from non-psychiatric medical documents. METHODS This retrospective prognostic study used data from 47,625 patients at BC Cancer. We analyzed initial oncology consultation documents using traditional and neural language models to predict whether patients would see a counsellor or psychiatrist in the 12 months following their initial oncology consultation. RESULTS Here, we show our best models achieved a balanced accuracy (receiver-operating-characteristic area-under-curve) of 73.1% (0.824) for predicting seeing a psychiatrist, and 71.0% (0.784) for seeing a counsellor. Different words and phrases are important for predicting each outcome. CONCLUSION These results suggest natural language processing can be used to predict psychosocial needs of patients with cancer from their initial oncology consultation document. Future research could extend this work to predict the psychosocial needs of medical patients in other settings.
Collapse
Affiliation(s)
- John-Jose Nunez
- BC Cancer, Vancouver, BC, Canada.
- Department of Computer Science, University of British Columbia, Vancouver, BC, Canada.
- Department of Psychiatry, University of British Columbia, Vancouver, BC, Canada.
| | | | | | - Raymond T Ng
- Department of Computer Science, University of British Columbia, Vancouver, BC, Canada
| | - Alan T Bates
- BC Cancer, Vancouver, BC, Canada
- Department of Psychiatry, University of British Columbia, Vancouver, BC, Canada
| |
Collapse
|