Copyright: ©Author(s) 2026. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license. No commercial re-use. See permissions. Published by Baishideng Publishing Group Inc.
Do bots provide correct and adequate guidance regarding acidity: A blinded comparison rated by patients and physicians
Kartikay Goyal, Manjeet Kumar Goyal, Varna Taranikanti, Praneet Wander, Rishi Chowdhary, Shivam Kalra, Gargi Prashar, Ashita Rukmini Vuthaluru, Omesh Goyal
Kartikay Goyal, Department of Medicine, Government Medical College and Hospital, Chandigarh 160030, Chandīgarh, India
Manjeet Kumar Goyal, Department of Internal Medicine, Cleveland Clinic Akron General Hospital, Akron, OH 44308, United States
Varna Taranikanti, Department of Foundational Medical Studies, Oakland University William Beaumont School of Medicine Rochester, Rochester, MI 48309, United States
Praneet Wander, Division of Gastroenterology, St Mary’s Hospital, Trinity Health of New England, Waterbury, CT 06708, United States
Rishi Chowdhary, Department of Medicine, Metro Health Medical Center, Cleveland, OH 44109, United States
Shivam Kalra, Department of Internal Medicine, Trident Medical Center, Charleston, SC 29405, United States
Gargi Prashar, Omesh Goyal, Department of Gastroenterology, Dayanand Medical College and Hospital, Ludhiana 141001, Punjab, India
Ashita Rukmini Vuthaluru, Department of Anesthesiology, All India Institute of Medical Sciences, New Delhi 110029, Delhi, India
Co-first authors: Kartikay Goyal and Manjeet Kumar Goyal.
Author contributions: Goyal K and Goyal MK jointly designed the study, conducted the analysis, and drafted the manuscript, underlying the co-first authorship designation; Taranikanti V, Wander P, and Gargi P contributed to the analysis, drafting of the manuscript, and literature review; Wander P and Chowdhary R assisted with the manuscript preparation and literature review; Kalra S and Vuthaluru AR provided critical feedback and clinical insights; Goyal O supervised the overall study design and provided senior guidance.
AI contribution statement: AI tools (Grammarly) were used solely for basic grammar correction and improving language clarity. They were not used for data analysis, content generation, or substantive writing.
Institutional review board statement: This study was reviewed and approved by the Ethics Committee of Dayanand Medical College and Hospital, Ludhiana (IEC No. DMCH-21/615).
Informed consent statement: The study consist of patient with dyspepsia reading and rating the responses of large language models. This was to check the relativity of references for the patients. All study participants, or their legal guardian, provided informed written consent prior to study enrollment.
Conflict-of-interest statement: All authors declare that they have no conflict of interest to disclose.
STROBE statement: The authors have read the STROBE Statement – checklist of items, and the manuscript was prepared and revised according to the STROBE Statement – checklist of items.
Data sharing statement: Technical appendix, statistical code, and dataset available from the corresponding author upon reasonable request.
Corresponding author: Omesh Goyal, Professor, Department of Gastroenterology, Dayanand Medical College and Hospital, Tagore Nagar, Ludhiana, Punjab 141001, India.
goyalomesh@yahoo.co.in
Received: November 2, 2025
Revised: December 22, 2025
Accepted: January 28, 2026
Published online: September 20, 2026
Processing time: 252 Days and 8.5 Hours
BACKGROUND
Large language models (LLMs) are increasingly accessed by patients for gastrointestinal health information. Despite their growing use, concerns persist regarding accuracy, empathy, actionability, and readability of responses generated by LLMs.
AIM
To assess the responses generated by ChatGPT-5, Gemini-2.5, and Claude-4 for common patient questions on “acidity” (heartburn/dyspepsia/gastroesophageal reflux disease).
METHODS
Thirty-nine frequently asked questions were submitted to each model. Responses were independently rated by three gastroenterologists for accuracy, comprehensiveness, empathy, and actionability; and by 20 patients for empathy, comprehensiveness, actionability, compassion, and usefulness. Readability indices were also analyzed.
RESULTS
Significant inter-model differences were observed across multiple physician-rated domains. Gemini-2.5 and Claude-4 achieved higher mean scores for accuracy, comprehensiveness, and actionability compared with ChatGPT-5 (P < 0.05), while Claude-4 demonstrated the highest empathy scores. Patient ratings indicated uniformly high comprehensibility across all models; however, Gemini-2.5 and Claude-4 responses were perceived as more actionable than those generated by ChatGPT-5. Readability analysis showed that ChatGPT-5 produced the most accessible responses, corresponding approximately to a high-school reading level, whereas Gemini-2.5 and Claude-4 generated more linguistically complex content.
CONCLUSION
These findings underscore the need for careful model selection and suggest that hybrid approaches integrating complementary model strengths may optimize safe and effective artificial intelligence -assisted patient education in gastroenterology.
Core Tip: This is among the first head-to-head evaluations of ChatGPT-5, Gemini-2.5, and Claude-4 for patient education on acidity (gastroesophageal reflux disease). Physicians rated Gemini-2.5 and Claude-4 highest in accuracy, empathy, and actionability, while patients appreciated ChatGPT-5’s readability. The study underscores trade-offs between clarity and clinical richness and highlights how large language models can complement but not replace clinician-guided patient education.
