Observational Study
Copyright ©The Author(s) 2025. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastroenterol. Feb 14, 2025; 31(6): 102090
Published online Feb 14, 2025. doi: 10.3748/wjg.v31.i6.102090
Evaluating large language models as patient education tools for inflammatory bowel disease: A comparative study
Yan Zhang, Xiao-Han Wan, Qing-Zhou Kong, Han Liu, Jun Liu, Jing Guo, Xiao-Yun Yang, Xiu-Li Zuo, Yan-Qing Li
Yan Zhang, Xiao-Han Wan, Qing-Zhou Kong, Han Liu, Jun Liu, Jing Guo, Xiao-Yun Yang, Xiu-Li Zuo, Yan-Qing Li, Department of Gastroenterology, Qilu Hospital of Shandong University, Jinan 250012, Shandong Province, China
Yan Zhang, Xiao-Han Wan, Qing-Zhou Kong, Han Liu, Jun Liu, Jing Guo, Xiao-Yun Yang, Xiu-Li Zuo, Yan-Qing Li, Laboratory of Translational Gastroenterology, Qilu Hospital of Shandong University, Jinan 250012, Shandong Province, China
Yan Zhang, Xiao-Han Wan, Qing-Zhou Kong, Han Liu, Jun Liu, Jing Guo, Xiao-Yun Yang, Xiu-Li Zuo, Yan-Qing Li, Robot Engineering Laboratory for Precise Diagnosis and Therapy of GI Tumor, Qilu Hospital of Shandong University, Jinan 250012, Shandong Province, China
Yan Zhang, Xiao-Han Wan, Qing-Zhou Kong, Han Liu, Jun Liu, Jing Guo, Xiao-Yun Yang, Xiu-Li Zuo, Yan-Qing Li, Shandong Provincial Clinical Research Center for Digestive Disease, Qilu Hospital of Shandong University, Jinan 250012, Shandong Province, China
Co-first authors: Yan Zhang and Xiao-Han Wan.
Author contributions: Zhang Y and Wan XH contribute equally to this study as co-first authors; Zhang Y and Wan XH participated in the conceptualization, data curation, formal analysis, investigation, methodology, and writing of the original draft; Kong QZ participated in methodology; Liu J, Guo J and Liu H made the evaluation scores; Yang XY and Zuo XL participated in the conceptualization and methodology; Li YQ participated in the conceptualization, supervision, writing of the manuscript, and editing; all authors contributed to the article and approved the submitted version.
Supported by the China Health Promotion Foundation Young Doctors' Research Foundation for Inflammatory Bowel Disease, the Taishan Scholars Program of Shandong Province, China, No. tsqn202306343; and National Natural Science Foundation of China, No. 82270578.
Institutional review board statement: This study did not involve human or animal experiments. Therefore, ethics committee approval was not required.
Informed consent statement: This study did not involve human or animal experiments. Therefore, informed consent was not required.
Conflict-of-interest statement: All the authors report having no relevant conflicts of interest for this article.
STROBE statement: The authors have read the STROBE Statement—checklist of items, and the manuscript was prepared and revised according to the STROBE Statement—checklist of items.
Data sharing statement: Dataset available from the corresponding author at liyanqing@sdu.edu.cn. Participants gave informed consent for data sharing.
Open Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: https://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Yan-Qing Li, PhD, Professor, Department of Gastroenterology, Qilu Hospital of Shandong University, No. 107 Wenhuaxi Road, Jinan 250012, Shandong Province, China. liyanqing@sdu.edu.cn
Received: October 8, 2024
Revised: December 8, 2024
Accepted: December 23, 2024
Published online: February 14, 2025
Processing time: 93 Days and 23.6 Hours
Core Tip

Core Tip: This study evaluated the performance of three large language models, ChatGPT-4.0, Claude-3-Opus, and Gemini-1.5-Pro, in providing accurate and comprehensible information for patients with inflammatory bowel disease. Although all three models showed satisfactory accuracy and relevance, Claude-3-Opus excelled in terms of readability and patient comprehensibility. However, discrepancies in the provision of complex medical advice highlight the need for further optimization and specialized model development to ensure safe and accurate patient education.