05230413

October 12, 2021, 08:18

Reader’s expertise on the topic of the manuscript

Does the reader have a conflict of interest?

The overall quality of the manuscript, based on the above-listed criteria, should be evaluated and classified according to the following five categories

Language quality (style, grammar, and spelling) should be evaluated and classified according to the following five categories.

This is an important article to review the role of AI/ML in gastroenterology and hepatology. AI will open a new era of clinical practice in gastroenterology soon. In this point of view, I would like to comment several things. Whether diagnostic/prognostic prediction model is AI/ML or classic logistic regression or not, prediction model for binary/categorical outcomes provides probability (from 0 to 1) of events of interest. It is cliché for AI/ML model developers to use AUC of ROC curve analysis as a critical measure to compare the performances of candidate models and they are likely to adopt a model of high AUC value. However, not only accuracy in discrimination but also precision in the calibration of the predicted probability must be included in the evaluation of the model’s performance. Metrics of calibration are in general calibration slope and intercept, which is a similar concept of bias and variance. The reason why I would like to emphasize calibration is that the AI is a black-box model in binary prediction. We cannot understand their algorithms made from the training dataset. There are several measures such as factor importance or SHAP plot, to visualize the impact of predictors. However, they do not provide 'why and how'. These are utmost questions in medical decision. For example, your cancer patient will die within 2 years in prediction model of AUC=0.95. Unless you understand the mechanism and condition of your patient's death, you cannot provide any medical decision. Why and How; this is so-called 'interpretability'. Unless clinician interpret the phenomenon, they may hesitate to adopt the results of prediction models. However, highly interpretable AI/ML models are not available yet. In this point of view, calibration should be at least another key property of a prediction model. Reliability of calibration may give clinician more confidence in predicted results. In addition, measures of overall performance such as R square and Brier scoring, should be provided. Especially, brier scoring is a similar metric concept of mean square error in binary classification. If my candidate model with excellent AUC has extremely poor calibration and poor Brier score, I would hesitate to use it. Nonetheless, AI/ML will be rapidly becoming the canon in many hospitals.

First, thank you very much for your professional comments on the article published in World Journal of Gastroenterology. Second, we read your comments with great interest. You are welcome to format your valuable comments into a Letter to the Editor and submit it online to World Journal of Gastroenterology at https://www.f6publishing.com. There are no restrictions on the number of words, figures (color, B/W) or authors for a Letter to the Editor. In addition, the article processing charge will be exempted for this Letter to the Editor. As with all articles published by the Baishideng Publishing Group, the Letter to the Editor will be published online after completing peer review. The guidelines for a Letter to the Editor can be found at: https://www.wjgnet.com/bpg/GerInfo/219. Finally, we look forward to receiving your high-quality Letter to the Editor, which will promote academic communication and lead the development of this discipline.