Evaluating chat generative pretrained transformer in answering questions on endoscopic mucosal resection and endoscopic submucosal dissection

doi:10.4251/wjgo.v17.i10.109792

Advanced Search

BPG is committed to discovery and dissemination of knowledge

Home / Archive / Volume 17, Issue 10

This Article

Peer-Review Report of This Article

CrossCheck and Google Search of This Article

Academic Rules and Norms of This Article

Supplementary Materials of This Article

Citation of this article

Corresponding Author of This Article

Research Domain of This Article

Article-Type of This Article

Open-Access Policy of This Article

Times Cited Counts in Google of This Article

Number of Hits and Downloads for This Article

Total Article Views (629)

All Articles published online

The chart showing PDF series, HTML series, Figures (1-3) series, Tables (1-2) series.

Item

Count

PDF

HTML

120

Figures (1-3)

Tables (1-2)

Sum=227

Publishing Process of This Article

The chart showing Browse series, Download series.

Item

Count

Browse

Download

283

Sum=350

Oct 15, 2025 (publication date) through Feb 26, 2026

Times Cited of This Article

Times Cited (1)

Journal Information of This Article

Publication Name

World Journal of Gastrointestinal Oncology

ISSN

1948-5204

Publisher of This Article

Baishideng Publishing Group Inc, 7041 Koll Center Parkway, Suite 160, Pleasanton, CA 94566, USA

Observational Study

World J Gastrointest Oncol. Oct 15, 2025; 17(10): 109792
Published online Oct 15, 2025. doi: 10.4251/wjgo.v17.i10.109792

Evaluating chat generative pretrained transformer in answering questions on endoscopic mucosal resection and endoscopic submucosal dissection

Shi-Song Wang, Hui Gao, Peng-Yao Lin, Tian-Chen Qian, Ying Du, Lei Xu

Shi-Song Wang, Hui Gao, Tian-Chen Qian, Ying Du, Lei Xu, Department of Gastroenterology, The First Affiliated Hospital of Ningbo University, Ningbo 315010, Zhejiang Province, China

Shi-Song Wang, Peng-Yao Lin, Ying Du, Health Science Center, Ningbo University, Ningbo 315010, Zhejiang Province, China

Tian-Chen Qian, Department of Gastroenterology, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou 310003, Zhejiang Province, China

Author contributions: Xu L conceived the study design; Wang SS and Gao H performed the statistical analysis; Wang SS and Du Y wrote the manuscript; Qian TC and Lin PY reviewed the manuscript; All authors approved the submitted draft.

Supported by Ningbo Top Medical and Health Research Program, No. 2023020612; the Ningbo Leading Medical & Healthy Discipline Project, No. 2022-S04; the Medical Health Science and Technology Project of Zhejiang Provincial Health Commission, No. 2022KY315; and Ningbo Science and Technology Public Welfare Project, No. 2023S133.

Institutional review board statement: Since the study did not involve human or animal data and all ChatGPT answers were public, there was no need for Ethics Committee approval.

Informed consent statement: As this study does not involve human or animal data and all ChatGPT responses are publicly accessible, informed consent was not required.

Conflict-of-interest statement: All the authors report no relevant conflicts of interest for this article.

STROBE statement: The authors have read the STROBE Statement—checklist of items, and the manuscript was prepared and revised according to the STROBE Statement—checklist of items.

Data sharing statement: Technical appendix, statistical code, and dataset available from the corresponding author.

Corresponding author: Lei Xu, MD, PhD, Department of Gastroenterology, The First Affiliated Hospital of Ningbo University, No. 59 Liuting Street, Ningbo 315010, Zhejiang Province, China. xulei22@163.com

Received: May 22, 2025
Revised: June 17, 2025
Accepted: August 27, 2025
Published online: October 15, 2025
Processing time: 145 Days and 19.4 Hours

Abstract

BACKGROUND

With the rising use of endoscopic submucosal dissection (ESD) and endoscopic mucosal resection (EMR), patients are increasingly questioning various aspects of these endoscopic procedures. At the same time, conversational artificial intelligence (AI) tools like chat generative pretrained transformer (ChatGPT) are rapidly emerging as sources of medical information.

AIM

To evaluate ChatGPT’s reliability and usefulness regarding ESD and EMR for patients and healthcare professionals.

METHODS

In this study, 30 specific questions related to ESD and EMR were identified. Then, these questions were repeatedly entered into ChatGPT, with two independent answers generated for each question. A Likert scale was used to rate the accuracy, completeness, and comprehensibility of the responses. Meanwhile, a binary category (high/Low) was used to evaluate each aspect of the two responses generated by ChatGPT and the response retrieved from Google.

RESULTS

By analyzing the average scores of the three raters, our findings indicated that the responses generated by ChatGPT received high ratings for accuracy (mean score of 5.14 out of 6), completeness (mean score of 2.34 out of 3), and comprehensibility (mean score of 2.96 out of 3). Kendall’s coefficients of concordance indicated good agreement among raters (all P < 0.05). For the responses generated by Google, more than half were classified by experts as having low accuracy and low completeness.

CONCLUSION

ChatGPT provided accurate and reliable answers in response to questions about ESD and EMR. Future studies should address ChatGPT’s current limitations by incorporating more detailed and up-to-date medical information. This could establish AI chatbots as significant resource for both patients and health care professionals.

Keywords: Endoscopic submucosal dissection; Endoscopic mucosal dissection; Artificial intelligence; Chat generative pretrained transformer; Patient education; Google

Core Tip: This study evaluated the reliability and usefulness of chat generative pretrained transformer in addressing questions related to endoscopic submucosal dissection and endoscopic mucosal resection. A set of thirty targeted questions was repeatedly entered, and responses were independently rated for accuracy, completeness, and comprehensibility. Compared with Google, chat generative pretrained transformer produced more accurate, detailed, and easier to understand answers, with consistent agreement among evaluators. The findings indicate that chat generative pretrained transformer may serve as a valuable and accessible source of medical information for both patients and healthcare professionals.