Wang SS, Gao H, Lin PY, Qian TC, Du Y, Xu L. Evaluating chat generative pretrained transformer in answering questions on endoscopic mucosal resection and endoscopic submucosal dissection. World J Gastrointest Oncol 2025; 17(10): 109792 [PMID: 41114106 DOI: 10.4251/wjgo.v17.i10.109792]
Corresponding Author of This Article
Lei Xu, MD, PhD, Department of Gastroenterology, The First Affiliated Hospital of Ningbo University, No. 59 Liuting Street, Ningbo 315010, Zhejiang Province, China. xulei22@163.com
Research Domain of This Article
Gastroenterology & Hepatology
Article-Type of This Article
Observational Study
Open-Access Policy of This Article
This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Oct 15, 2025 (publication date) through Oct 25, 2025
Times Cited of This Article
Times Cited (0)
Journal Information of This Article
Publication Name
World Journal of Gastrointestinal Oncology
ISSN
1948-5204
Publisher of This Article
Baishideng Publishing Group Inc, 7041 Koll Center Parkway, Suite 160, Pleasanton, CA 94566, USA
Share the Article
Wang SS, Gao H, Lin PY, Qian TC, Du Y, Xu L. Evaluating chat generative pretrained transformer in answering questions on endoscopic mucosal resection and endoscopic submucosal dissection. World J Gastrointest Oncol 2025; 17(10): 109792 [PMID: 41114106 DOI: 10.4251/wjgo.v17.i10.109792]
World J Gastrointest Oncol. Oct 15, 2025; 17(10): 109792 Published online Oct 15, 2025. doi: 10.4251/wjgo.v17.i10.109792
Evaluating chat generative pretrained transformer in answering questions on endoscopic mucosal resection and endoscopic submucosal dissection
Shi-Song Wang, Hui Gao, Peng-Yao Lin, Tian-Chen Qian, Ying Du, Lei Xu
Shi-Song Wang, Hui Gao, Tian-Chen Qian, Ying Du, Lei Xu, Department of Gastroenterology, The First Affiliated Hospital of Ningbo University, Ningbo 315010, Zhejiang Province, China
Shi-Song Wang, Peng-Yao Lin, Ying Du, Health Science Center, Ningbo University, Ningbo 315010, Zhejiang Province, China
Tian-Chen Qian, Department of Gastroenterology, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou 310003, Zhejiang Province, China
Author contributions: Xu L conceived the study design; Wang SS and Gao H performed the statistical analysis; Wang SS and Du Y wrote the manuscript; Qian TC and Lin PY reviewed the manuscript; All authors approved the submitted draft.
Supported by Ningbo Top Medical and Health Research Program, No. 2023020612; the Ningbo Leading Medical & Healthy Discipline Project, No. 2022-S04; the Medical Health Science and Technology Project of Zhejiang Provincial Health Commission, No. 2022KY315; and Ningbo Science and Technology Public Welfare Project, No. 2023S133.
Institutional review board statement: Since the study did not involve human or animal data and all ChatGPT answers were public, there was no need for Ethics Committee approval.
Informed consent statement: As this study does not involve human or animal data and all ChatGPT responses are publicly accessible, informed consent was not required.
Conflict-of-interest statement: All the authors report no relevant conflicts of interest for this article.
STROBE statement: The authors have read the STROBE Statement—checklist of items, and the manuscript was prepared and revised according to the STROBE Statement—checklist of items.
Data sharing statement: Technical appendix, statistical code, and dataset available from the corresponding author.
Open Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: https://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Lei Xu, MD, PhD, Department of Gastroenterology, The First Affiliated Hospital of Ningbo University, No. 59 Liuting Street, Ningbo 315010, Zhejiang Province, China. xulei22@163.com
Received: May 22, 2025 Revised: June 17, 2025 Accepted: August 27, 2025 Published online: October 15, 2025 Processing time: 145 Days and 19.4 Hours
Abstract
BACKGROUND
With the rising use of endoscopic submucosal dissection (ESD) and endoscopic mucosal resection (EMR), patients are increasingly questioning various aspects of these endoscopic procedures. At the same time, conversational artificial intelligence (AI) tools like chat generative pretrained transformer (ChatGPT) are rapidly emerging as sources of medical information.
AIM
To evaluate ChatGPT’s reliability and usefulness regarding ESD and EMR for patients and healthcare professionals.
METHODS
In this study, 30 specific questions related to ESD and EMR were identified. Then, these questions were repeatedly entered into ChatGPT, with two independent answers generated for each question. A Likert scale was used to rate the accuracy, completeness, and comprehensibility of the responses. Meanwhile, a binary category (high/Low) was used to evaluate each aspect of the two responses generated by ChatGPT and the response retrieved from Google.
RESULTS
By analyzing the average scores of the three raters, our findings indicated that the responses generated by ChatGPT received high ratings for accuracy (mean score of 5.14 out of 6), completeness (mean score of 2.34 out of 3), and comprehensibility (mean score of 2.96 out of 3). Kendall’s coefficients of concordance indicated good agreement among raters (all P < 0.05). For the responses generated by Google, more than half were classified by experts as having low accuracy and low completeness.
CONCLUSION
ChatGPT provided accurate and reliable answers in response to questions about ESD and EMR. Future studies should address ChatGPT’s current limitations by incorporating more detailed and up-to-date medical information. This could establish AI chatbots as significant resource for both patients and health care professionals.
Core Tip: This study evaluated the reliability and usefulness of chat generative pretrained transformer in addressing questions related to endoscopic submucosal dissection and endoscopic mucosal resection. A set of thirty targeted questions was repeatedly entered, and responses were independently rated for accuracy, completeness, and comprehensibility. Compared with Google, chat generative pretrained transformer produced more accurate, detailed, and easier to understand answers, with consistent agreement among evaluators. The findings indicate that chat generative pretrained transformer may serve as a valuable and accessible source of medical information for both patients and healthcare professionals.
