Letter to the Editor Open Access
Copyright ©The Author(s) 2023. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Crit Care Med. Mar 9, 2023; 12(2): 89-91
Published online Mar 9, 2023. doi: 10.5492/wjccm.v12.i2.89
The artificial intelligence evidence-based medicine pyramid
Valentina Bellini, Elena Bignami, Department of Medicine and Surgery, University of Parma, Anesthesiology, Critical Care and Pain Medicine Division, Parma 43126, Italy
Federico Coccolini, Department of General, Emergency and Trauma Surgery, Pisa University Hospital, Pisa 56124, Italy
Francesco Forfori, Department of Anesthesia and Intensive Care, University of Pisa, Pisa 53126, Italy
ORCID number: Elena Bignami (0000-0001-9985-4126).
Author contributions: Bellini V, Coccolini F, Forfori F and Bignami E made substantial contributions to the conception and design of the study as well as the acquisition and interpretation of data; Bellini V and Bignami E drafted the article; Coccolini F and Forfori F made critical revisions related to important intellectual content within the manuscript; Bellini V, Coccolini F, Forfori F and Bignami E approved the final draft of the article.
Conflict-of-interest statement: All the authors report having no relevant conflicts of interest for this article.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: https://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Elena Bignami, MD, Professor, Department of Medicine and Surgery, University of Parma, Anesthesiology, Critical Care and Pain Medicine Division, Viale Gramsci 14, Parma 43126, Italy. elenagiovanna.bignami@unipr.it
Received: October 14, 2022
Peer-review started: October 14, 2022
First decision: November 14, 2022
Revised: November 21, 2022
Accepted: February 1, 2023
Article in press: February 1, 2023
Published online: March 9, 2023
Processing time: 143 Days and 19 Hours

Abstract

Several studies exist in the literature regarding the exploitation of artificial intelligence in intensive care. However, an important gap between clinical research and daily clinical practice still exists that can only be bridged by robust validation studies carried out by multidisciplinary teams.

Key Words: Artificial intelligence; Intensive care; Intensive care unit; Evidence-based medicine; Clinical research

Core Tip: Artificial intelligence (AI) use in intensive care is now a reality. However, there is still an important discrepancy between the results found in the scientific literature and the day-to-day clinical implementation of this technology. One reason for this is that the AI evidence pyramid in intensive care has only just begun to emerge. We need to focus on the next steps in AI pyramid evidence, amplifying the external validation of models and increasing the number of randomized clinical trials. Only robust validation studies carried out by multidisciplinary teams will help bridge this existing gap between clinical research and clinical practice.



TO THE EDITOR

We read with great interest the editorial by Luo et al[1] where the authors cogently present the main results regarding the use of artificial intelligence (AI) in the intensive care unit (ICU) for decision making and resource allocation. They simultaneously exposed the current limitations of the large-scale use of AI clinical tools in this setting. We share many of the reflections set out by Luo et al[1]. The presence of AI in medicine science and clinical practice has become a reality. Knowing how this new technology can assist the medical profession and how clinicians might take advantage of it are characteristics that are now required and are likely to be of assistance as far as personal career development is concerned[2]. However, the gap between the excellent results derived from biomedical research and the rare use in clinical practice is clear to everyone[3]. While this is probably the biggest deterrent to AI application on a daily professional basis, we must not stop considering it as a valuable ally. On the contrary, we need to ask clinical researchers to find answers for how these models can help intensivists carry out day-to-day activities.

Without external validation, the positive performance of these models in observational studies is no longer sufficient. This, however, should not lead to the erroneous conviction that AI implementation in the ICU should remain purely a scientific speculation, as its application outside the clinical reality regularly disproves this hypothesis. Intelligent vocal assistants and accurate search engines are just two examples of the efficient support offered to us by well-devised AI. The first results from clinical trials point in the same direction, with an example being the hypotension prediction index[4]. This is an algorithm implemented to predict hypotension, even before adverse events occur. Since its marketing, a number of clinical trials have tried to interpret its possible usefulness in clinical practice with most results showing a lower incidence of hypotensive events when compared with standard care[5-7].

We should bear in mind that anything stemming from evidence-based medicine (EBM) has a history based on the progressive collection of increasingly solid results, and the application of AI in the ICU follows the same path (Figure 1). We began with the intuition that AI might be useful in critical patients. Subsequently, stronger results, initially from retrospective followed by prospective observational studies, appeared. In the literature, a few clinical trials as well as sporadic systematic reviews and meta-analyses are available[8,9]. Presently, we are only halfway up the pyramid of the AI scientific evidence we initially imagined, and it is therefore logical that the use of AI tools is not widespread. This phenomenon is consistent with the concept of EBM. At this point, we need to focus on the second part of the pyramid, increasing the external validation of models and multiplying the number of randomized clinical trials.

Figure 1
Figure 1 A pyramid for artificial intelligence scientific evidence is proposed. Starting from the bottom and moving to the top, emerging results are becoming increasingly solid and strong. The two lowest rungs are the theory followed by the third, fourth and fifth steps that represent studies analyzing the use of artificial intelligence (AI) in clinical practice. From creation of the model with internal validation, we move towards external validation studies and the creation of usable real instruments (AI tools). The penultimate step [randomized controlled trials (RCTs)] and the tip of the pyramid (meta-analysis and systematic reviews) represent the strongest methodological analysis to reach conclusions on the real impact of this technology on healthcare systems. If we then imagine the support base of the pyramid we have the necessary tools for each step of clinical research in AI applied to the intensive care unit: Electronic health record, solid big data systems, internet of things technologies and models of eXplainable AI.

Furthermore, we must not underestimate the fact that this gap can only be bridged by the intervention of multidisciplinary teams. As with the creation of the AI surgical department in anesthesiology[10], similar systems need to be considered for the ICU. Engineers, data scientists and intensivists must create units capable of managing each phase of the AI application in the ICU, from the design and then to the creation and exploitation of AI clinical instruments. This cooperation should also take place in the post-marketing phase, with constant verification of the quality and safety of AI tools together with continuous systems updates. In conclusion, it is not surprising that AI is not yet widely used in daily ICU activities. We are still at the very beginning of the EBM pyramid, and the gap between bytes and the bedside will only be bridged by robust validation studies carried out by multidisciplinary teams.

Footnotes

Provenance and peer review: Unsolicited article; Externally peer reviewed.

Peer-review model: Single blind

Specialty type: Medicine, research and experimental

Country/Territory of origin: Italy

Peer-review report’s scientific quality classification

Grade A (Excellent): 0

Grade B (Very good): B

Grade C (Good): C

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Hanada E, Japan; Morya AK, India S-Editor: Xing YX L-Editor: Filipodia P-Editor: Xing YX

References
1.  Luo MH, Huang DL, Luo JC, Su Y, Li JK, Tu GW, Luo Z. Data science in the intensive care unit. World J Crit Care Med. 2022;11:311-316.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in CrossRef: 1]  [Cited by in F6Publishing: 1]  [Article Influence: 0.5]  [Reference Citation Analysis (1)]
2.  Basu K, Sinha R, Ong A, Basu T. Artificial Intelligence: How is It Changing Medical Sciences and Its Future? Indian J Dermatol. 2020;65:365-370.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5]  [Cited by in F6Publishing: 35]  [Article Influence: 8.8]  [Reference Citation Analysis (0)]
3.  Shipley E, Joddrell M, Lip GY, Zheng Y. Bridging the Gap Between Artificial Intelligence Research and Clinical Practice in Cardiovascular Science: What the Clinician Needs to Know. Arrhythm Electrophysiol Rev. 2022;11:e03.  [PubMed]  [DOI]  [Cited in This Article: ]  [Reference Citation Analysis (0)]
4.  Davies SJ, Vistisen ST, Jian Z, Hatib F, Scheeren TWL. Ability of an Arterial Waveform Analysis-Derived Hypotension Prediction Index to Predict Future Hypotensive Events in Surgical Patients. Anesth Analg. 2020;130:352-359.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 132]  [Cited by in F6Publishing: 112]  [Article Influence: 28.0]  [Reference Citation Analysis (0)]
5.  Wijnberge M, Geerts BF, Hol L, Lemmers N, Mulder MP, Berge P, Schenk J, Terwindt LE, Hollmann MW, Vlaar AP, Veelo DP. Effect of a Machine Learning-Derived Early Warning System for Intraoperative Hypotension vs Standard Care on Depth and Duration of Intraoperative Hypotension During Elective Noncardiac Surgery: The HYPE Randomized Clinical Trial. JAMA. 2020;323:1052-1060.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 174]  [Cited by in F6Publishing: 259]  [Article Influence: 64.8]  [Reference Citation Analysis (0)]
6.  Tsoumpa M, Kyttari A, Matiatou S, Tzoufi M, Griva P, Pikoulis E, Riga M, Matsota P, Sidiropoulou T. The Use of the Hypotension Prediction Index Integrated in an Algorithm of Goal Directed Hemodynamic Treatment during Moderate and High-Risk Surgery. J Clin Med. 2021;10.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4]  [Cited by in F6Publishing: 28]  [Article Influence: 9.3]  [Reference Citation Analysis (0)]
7.  Maheshwari K, Shimada T, Yang D, Khanna S, Cywinski JB, Irefin SA, Ayad S, Turan A, Ruetzler K, Qiu Y, Saha P, Mascha EJ, Sessler DI. Hypotension Prediction Index for Prevention of Hypotension during Moderate- to High-risk Noncardiac Surgery. Anesthesiology. 2020;133:1214-1222.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 36]  [Cited by in F6Publishing: 40]  [Article Influence: 10.0]  [Reference Citation Analysis (0)]
8.  van de Sande D, van Genderen ME, Huiskens J, Gommers D, van Bommel J. Moving from bytes to bedside: a systematic review on the use of artificial intelligence in the intensive care unit. Intensive Care Med. 2021;47:750-760.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 101]  [Cited by in F6Publishing: 97]  [Article Influence: 32.3]  [Reference Citation Analysis (0)]
9.  Fleuren LM, Klausch TLT, Zwager CL, Schoonmade LJ, Guo T, Roggeveen LF, Swart EL, Girbes ARJ, Thoral P, Ercole A, Hoogendoorn M, Elbers PWG. Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med. 2020;46:383-400.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 310]  [Cited by in F6Publishing: 282]  [Article Influence: 70.5]  [Reference Citation Analysis (0)]
10.  Valente M, Bellini V, Del Rio P, Freyrie A, Bignami E. Artificial Intelligence Is the Future of Surgical Departments … Are We Ready? Angiology. 2022;33197221121192.  [PubMed]  [DOI]  [Cited in This Article: ]  [Reference Citation Analysis (0)]