Copyright
©The Author(s) 2025. Published by Baishideng Publishing Group Inc. All rights reserved.
Accuracy of artificial intelligence in meta-analysis: A comparative study of ChatGPT 4.0 and traditional methods in data synthesis
Aman Goyal, Muhammad Daoud Tariq, Areeba Ahsan, Muhammad Hamza Khan, Amna Zaheer, Hritvik Jain, Surabhi Maheshwari, Andrei Brateanu
Aman Goyal, Department of Internal Medicine, Seth GS Medical College and KEM Hospital, Mumbai 400012, Maharashtra, India
Muhammad Daoud Tariq, Areeba Ahsan, Department of Internal Medicine, Foundation University Medical College, Islamabad 44000, Pakistan
Muhammad Hamza Khan, Department of Internal Medicine, Karachi Medical and Dental College, Karachi 74700, Pakistan
Amna Zaheer, Department of Internal Medicine, Liaquat National Hospital and Medical College, Karachi 74800, Pakistan
Hritvik Jain, Department of Internal Medicine, All India Institute of Medical Sciences, Jodhpur 400022, India
Surabhi Maheshwari, Department of Internal Medicine, University of Alabama/Heersink School of Medicine, Montgomery, AL 36116, United States
Andrei Brateanu, Department of Internal Medicine, Cleveland Clinic Foundation, Cleveland, OH 44195, United States
Co-corresponding authors: Aman Goyal and Andrei Brateanu.
Author contributions: Goyal A designed research, performed research, analyzed data, and wrote the paper; Tariq MD performed research and analyzed data; Ahsan A wrote the paper; Khan MH wrote the paper; Zaheer A wrote the paper; Jain H wrote the paper; Maheshwari S supervised research, validated findings, and wrote the paper; Brateanu A supervised research, validated findings, conceptualized the study, and wrote the paper. Goyal A and Brateanu A are co-corresponding authors, as the former is expected to change their email and workplace in the near future. Since both authors contributed equally to the conceptualization and have a strong understanding of the topic, assigning two corresponding authors ensures smooth communication between journal readers and the authors in the future.
Conflict-of-interest statement: The authors have no conflicts of interest statement to declare.
PRISMA 2009 Checklist statement: The authors have read the PRISMA 2009 Checklist, and the manuscript was prepared and revised according to the PRISMA 2009 Checklist.
Open Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:
https://creativecommons.org/Licenses/by-nc/4.0/ Corresponding author: Aman Goyal, MD, Department of Internal Medicine, Seth GS Medical College and KEM Hospital, Acharya Donde Marg, Mumbai 400012, Maharashtra, India.
amanmgy@gmail.com
Received: October 14, 2024
Revised: March 15, 2025
Accepted: April 7, 2025
Published online: December 20, 2025
Processing time: 294 Days and 13.4 Hours
BACKGROUND
Meta-analysis is a critical tool in evidence-based medicine, particularly in cardiology, where it synthesizes data from multiple studies to inform clinical decisions. This study explored the potential of using ChatGPT to streamline and enhance the meta-analysis process.
AIM
To investigate the potential of ChatGPT to conduct meta-analyses in interventional cardiology by comparing the results of ChatGPT-generated analyses with those of randomly selected, human-conducted meta-analyses on the same topic.
METHODS
We systematically searched PubMed for meta-analyses on interventional cardiology published in 2024. Five meta-analyses were randomly chosen. ChatGPT 4.0 was used to perform meta-analyses on the extracted data. We compared the results from ChatGPT with the original meta-analyses, focusing on key effect sizes, such as risk ratios (RR), hazard ratios, and odds ratios, along with their confidence intervals (CI) and P values.
RESULTS
The ChatGPT results showed high concordance with those of the original meta-analyses. For most outcomes, the effect measures and P values generated by ChatGPT closely matched those of the original studies, except for the RR of stent thrombosis in the Sreenivasan et al study, where ChatGPT reported a non-significant effect size, while the original study found it to be statistically significant. While minor discrepancies were observed in specific CI and P values, these differences did not alter the overall conclusions drawn from the analyses.
CONCLUSION
Our findings suggest the potential of ChatGPT in conducting meta-analyses in interventional cardiology. However, further research is needed to address the limitations of transparency and potential data quality issues, ensuring that AI-generated analyses are robust and trustworthy for clinical decision-making.
Core Tip: This study explored the potential of using ChatGPT 4.0 to conduct meta-analyses. Five meta-analyses were systematically selected. ChatGPT 4.0 was used to perform meta-analyses on the extracted data, and the results were compared with the original meta-analyses. The results generated by ChatGPT 4.0 showed high concordance with those of the original meta-analyses. For most outcomes, the effect measures and p-values generated by ChatGPT 4.0 closely matched those of the original studies. Our findings suggest the potential of ChatGPT 4.0 in conducting meta-analyses in cardiology.
