Current role and future perspectives of artificial intelligence in echocardiography

Abstract

Echocardiography is an essential tool in diagnostic cardiology and is fundamental to clinical care. Artificial intelligence (AI) can help health care providers serving as a valuable diagnostic tool for physicians in the field of echocardiography specially on the automation of measurements and interpretation of results. In addition, it can help expand the capabilities of research and discover alternative pathways in medical management specially on prognostication. In this review article, we describe the current role and future perspectives of AI in echocardiography.

Key Words: Echocardiography; Artificial intelligence; Machine learning; Deep learning; Prognosis

Core Tip: Artificial intelligence (AI) is the process of having a computational program that can perform tasks of human intelligence (e.g., pattern recognition) by mimicking human thought processes. Echocardiography is an essential tool in diagnostic cardiology and is fundamental to clinical care. AI could help the health care providers by one side serving as a valuable diagnostic tool for physicians in the field of echocardiography specially on the automation of measurements and by the other side helping on the interpretation of results. The current role of this technique is described in this review and its future perspectives are covered through the text highlighting the obstacles and advantages expected on this implementation.

INTRODUCTION

Artificial intelligence (AI) is the process of having a computational program that can perform tasks of human intelligence by mimicking human thought processes[1]. The AI applications in cardiology are showing that not complex devices like electrocardiography (ECG) machine could generate big amounts of potential data transforming the ECG into a powerful tool for prediction[2].

On the same side, the use of AI techniques in cardiovascular imaging into the process of daily decision-making will enhance the delivery of care, and AI has been influencing in the last years every field of cardiac imaging in all phases from the beginning with acquisition to the last step of reporting[3-5]. But for sure it will be basic for the specialty that cardiologists retain the final step in the control of the system, take care of the decisions, and have the authority to amend algorithms in the cases that these get mistaken.

In radiology departments, the likely influence of AI on the imminent advance of this area of medical specialization is crystal clear. The use of computers to assist with radiologic image interpretation tasks is a practice that will endure for a considerable period, one example of this that we could see it on cardiothoracic imaging where the subset most commonly utilized in medical imaging is machine learning (ML) and the more complex deep learning (DL). Much scientific research has been done focusing on the use of ML for pattern recognition to identify and diagnose potentially a great amount of pathologies[6].

ML models based on AI are being applied quickly to the assessment of magnetic resonance or cardiac computed tomography (CT) as it is an independent tool completely different from the manual approach that the echocardiography needs. For echocardiography, the quality of the imaging obtained during the study is the most critical element for a good interpretation; however, for cardiac imaging with magnetic resonance or CT, the obtained imaging quality is seldom an issue as it is not obtained manually. The workforce for performing echocardiography will prevail for a long period of time as is needed to obtain good quality imaging in patients with adequate acoustic windows.

Echocardiography has a critical role in the diagnosis and management of patients that develop a cardiovascular disease. It allows for real-time imaging of the cardiac structures and fast detection of various anomalies[7]. Despite the large amount of echocardiographic interpretation and evaluation guidelines, the quantification and diagnosis are mainly based on a subjective review of the images obtained by 2- and 3-dimensional (3D) echocardiography, and due to this, we could say that echocardiography is still an error-prone and imperfect technique. Another element is that there exists a high level of inter-operator variation in the interpretation of echocardiography findings, which has been for ages a long-standing problem[8]. This inter-operator variation can lead to incorrect interpretations and diagnoses, especially when the poor-quality images obtained previously are interpreted. Furthermore, critical care specialists (cardiologist, intensivists, and anaesthesiologists) frequently have limited time to improve echocardiographic images and measurements in the critical care patients and have to make analyses quickly in unstable patients due to hemodynamic reasons. The need for fast, automated analysis of echocardiography data that are less dependent on operator effort is crucial for critical care scenarios.

Although the application of AI in echocardiography is still in its first steps to greater implementation, the use of this technology in the future has great potential and is presumed to support the improvement of the efficiency and accuracy of the manual tracing. The technology based on AI can help in a more standardized evaluation of echocardiographic images/videos to decrease human errors by producing automated, consistent, and accurate interpretations.

Current and future applications of AI in echocardiography are shown in Figure 1, and these applications of the AI technology in echocardiography go through multiple areas, including the process of image acquisition, the image interpretation tasks, text interpretation, performing diagnosis, and prognosis evaluation.

Figure 1 Current and future applications of artificial intelligence in echocardiography. AI: Artificial intelligence; CAD: Cardiovascular diseases; 2D/3D: 2 dimensional/3 dimensional.

CURRENT ROLE OF AI IN ECHOCARDIOGRAPHY

Imaging techniques like echocardiography have many advantages in relation with other techniques, and the main advantages of echocardiography include a wide availability, the potential portability, and nowadays the affordability that was not real before. However, when it comes to interpreting echocardiographic images, there is a notable level of variability among observers compared to other cardiovascular imaging techniques. In this regard, AI has the potential to play a valuable role by not only reducing observer variability but also enhancing diagnostic accuracy[9].

On the other hand, the utilization of AI in this field has been constrained by the intricate multi-view format of echocardiography and the indispensable requirement for human expertise in both image acquisition and interpretation. In daily clinical practice, this is mostly evident, as happened for new multidimensional imaging technologies that are not easily adopted, such as 3D echocardiography and speckle-tracking[1,5,10-17].

Image acquisition

In numerous clinical settings, echocardiography is not available because of an absence of qualified workforce. In these locations, nonexpert operators may perform limited exams [point-of-care ultrasound (POCUS)] using portable or handheld machines, but quality is no homogeneous, with risks for misleading and nondiagnostic imaging[18]. POCUS is commonly utilized in intensive care units; emergency departments; preoperative and outpatient clinics; and areas medically underserved, from rural areas of US to low- and middle-income countries to man operated space flights. POCUS also enabled frontline clinicians to acquire echocardiograms in patients with coronavirus disease 2019 during the pandemic, restricting the exposure of sonographers[19]. During this period, technology based on AI allowed acquisition of diagnostic-quality ultrasonographic studies by users with a minimal training in these locations, and data from one study showed this recently with novice users, demonstrating that they could obtain 10-view transthoracic echocardiographic studies of diagnostic quality using DL-based software[20] approved by the United States Food and Drug Administration[21], that guides the operator on the recording of the right views.

Image interpretation

In comparison with image acquisition technology, echocardiographic interpretation using AI has evolved in recent years based on a correct automated image interpretation and classification[22,23]. Several studies have confirmed and validated the use of AI for automated quantification of left ventricle and right ventricle volumes or ejection fraction, global longitudinal strain, and atrial size or function from both 2D and 3D acquisitions[23-31].

One of the notable applications of AI in the field of echocardiography is the assessment of myocardial thickening and endocardial excursion, which enables the detection of regional wall motion abnormalities (RWMAs). This application holds great significance, particularly in managing ischemic coronary artery disease[9]. The assessment of RWMAs typically relies on the visual interpretation of the operator, which is subjective and experience-dependent. This evaluation holds great importance, especially when assessing patients with chest pain in the emergency room. To mitigate the possibility of incorrect interpretations, AI models have been developed to detect and quantify RWMAs, providing a more objective approach[3]. These ML and DL models exist, designed to evaluate RWMAs and diagnose ischemic cardiomyopathy[32,33]. The sensitivity of RWMAs evaluation models for automated diagnosis of ischemic heart disease is related to the area under the receiver-operating characteristic curve by AI algorithm. It should be remarked that AI algorithm “experience and expertise” are based on the available real-world datasets, that will have the same limits, such as the relatively higher ratios of wrong classification of the left anterior descending coronary artery perfusion territory described in the latest publications[33].

Report composition

The natural language processing algorithms have been created for large-scale extraction of the text data from echocardiographic reports; however, the widespread application has been limited due to the portability of each algorithm, with a potential degradation of algorithm performance when it is applied to an external data set[34,35].

Another step is the automatic interpretation of echo measurements that for sure is the next step for the automation of the echocardiography workflow. One good example of this strategy is the screening for valvular heart diseases (VHDs) using the data obtained through Doppler echocardiography video recordings automatically analyzed; the research of this interesting investigation created a three-step DL framework for the automatic screening of the echocardiographic videos for the detection of mitral regurgitation (MR) and stenosis (MS), and the detection of aortic regurgitation (AR) and stenosis (AS), and this DL algorithm categorizes the echocardiographic views, detects the presence of VHDs, and, when present, measures essential metrics related with the valvular severities. The DL algorithm was trained initially with 1335 exams, then was validated with 311, and finally was tested in 434 individuals using retrospectively selected studies from five hospitals. A prospectively collected set of 1374 consecutive echocardiograms was used later as the real-world data set for testing the algorithm. Disease categorization accuracy obtained was high, showing the following areas under the curve: 0.88 (95% confidence interval: 0.86-0.90) for MR; 0.99 (95% confidence interval: 0.97-0.99) for MS; 0.90 (95% confidence interval: 0.88-0.92) for AR; and 0.97 (95% confidence interval: 0.95-0.99) for AS in the prospective test data set[36].

Diagnostic process

In this aspect, a continuous progress has been made, and one of the first successful approaches was published by Zhang et al[23] that used a model based on DL to create a fully automated echocardiogram interpretation program, that included view identification (a kind of chamber view detection), image segmentation (detection of the different parts of the image), quantification of structure and function (automatic measurements), and disease detection (after the integration of prior data). By analyzing over 14000 echocardiographic studies, the algorithm achieved an impressive 96% accuracy in recognizing and distinguishing between various echocardiographic view classifications, such as parasternal long-axis and short-axis views. Furthermore, it demonstrated an accuracy ranging from 72% to 90% in accurately segmenting the image.

Additionally, the authors of the research presented that the algorithm for automated quantification of cardiac structure and function was similar to or even superior to the manual measurements across 11 internal consistency metrics, and that unexpectedly the convolutional neural networks were also successfully trained to detect hypertrophic cardiomyopathy, cardiac amyloidosis, and pulmonary artery hypertension, with a high accuracy. Even though the accuracy has not reached the level of the experts, the potential application of the DL models to echocardiography interpretation is a very promising tool for the detection of uncommon diseases.

Another important research is the one centered on common diseases like the potential automation of the detection of severe coronary artery disease with an echocardiographic system using AI. This innovation demonstrates the potential of validating an AI system for automating the analysis of stress echocardiography, thereby assisting clinicians in their interpretation. To achieve this, an automated image processing pipeline was developed to extract new geometric and kinematic features from a dataset of stress echocardiograms. The dataset was collected as part of a large, prospective, multicenter, multivendor study conducted in the United Kingdom. Using the extracted features, a ML algorithm was trained to identify patients with severe coronary artery disease based on invasive coronary angiography. Through cross-fold validation, the algorithm achieved a satisfactory classification accuracy in identifying patients with severe coronary artery disease in the training dataset. It utilized 31 unique geometric and kinematic features and demonstrated a sensitivity of 84.4% and specificity of 92.7%. Importantly, this accuracy was also observed in the independent validation dataset from the United States. By providing automated classifications to clinicians during the interpretation of stress echocardiograms, this method has the potential to enhance accuracy, improve inter-reader agreement, and boost reader confidence in the near future[37]. Another area of research of great interest is the VHD for the acceleration of echocardiography workflows[36,38-40].

Prognostication

In the scientific literature, we could find nonrandomized studies on the use of AI in echocardiography to predict outcomes, going from the response to cardiac resynchronization therapy to in-hospital mortality[41-43]. Samad et al[44] employed an ML framework for the prediction of all-cause mortality combining the information from echocardiographic measurements and electronic medical information of 171510 patients. A random forest model was compared with a logistic regression model based on a range of analytic approaches employing echocardiographic and clinical variables to predict outcomes. The random forest models had a superior prediction accuracy (all areas under the curve > 0.82) over common clinical risk scores (areas under the curve = 0.69-0.79) and did better than logistic regression models (P < 0.001) on all survival durations[44].

FUTURE PERSPECTIVES OF AI IN ECHOCARDIOGRAPHY

For the prediction of the future, probably we must focus on the potential gaps and limitations of AI, knowing that elements will guide us on the new advances that we must expect in the years to come.

The utilization of automated tracing and recognition of structures faces several limitations, as outlined in Table 1. However, ongoing advancements in this technology aim to address these limitations and enhance reproducibility and user experience. It is important to note that most studies have primarily focused on patients in sinus rhythm, with limited knowledge regarding the application of automation in patients with arrhythmia, significant conduction disease, and paced rhythm. Erroneous border tracings by the computer were found to be more prevalent in poor and fair-quality images, and the automated software did not function effectively in numerous cases. This poses a significant obstacle to the further development of this software. On the other hand, contour corrections appear to enhance the accuracy of automated analysis, resulting in a stronger correlation with cardiac MRI. However, this does increase the analysis time and may reduce workflow efficiency. In conclusion, the automation processes have demonstrated a favorable correlation with cardiac MRI volumes, as traditional 2D measurements have been shown to potentially underestimate chamber volumes[13].

Table 1 Current applications, strengths, and limitations of artificial intelligence in echocardiography.

Applications of AI in echocardiography	Strengths	Limitations
View interpretation and classification	View identification and classification among thousands of images. Possibility of quantification of both structure and function. Possibility of disease diagnosis	Lack of learning process clarification. Possibility of imperfect classification. Image quality is often suboptimal, and nonstructural echocardiographic data need careful preprocessing by the specialist to build the definitive model. Non-standardized intermediate off-axis and continuously rotational and sweeping views, which can be clinically very helpful, even though of low technical quality, are difficult to be managed by AI models
Measuring anatomy and morphofunctional structure	Building a patient similarity model (e.g., for predicting major cardiac events). Comparing automatic analysis between echocardiography and other imaging modalities	Possibility of suboptimal image quality. Possibility of limited number and representativeness of datasets. Current inferiority of automatic compared to semi-automatic software. Frequently inadequate standardization
Wall motion abnormalities detection	Reducing the potential operator-dependent misreading. Detecting different patterns of responses to stress. Possibility of integration with other technologies (e.g., strain technology)	AI algorithms are based on the existing real world datasets, that bring with them the same limits and misclassification risks. Possibility of suboptimal images quality (which implies the exclusion of some acquisitions, hence limited authenticity). Presence of arrhythmias (difficult to be managed by AI models)

AI: Artificial intelligence.

Nowadays, most of the studies of AI in echocardiography are constructed with retrospective data and centered largely on the performance of AI in specific diagnostic tasks. These studies cover from small exploratory studies (like the 2017 study made by Sanchez-Martinez et al[45] with 55 patients) to larger studies (like the 2018 study created by Madani et al[22] evaluating > 200000 images or the 2021 study by Solomon et al[46] evaluating > 900000 echocardiography reports). There is a need for prospective studies to show the feasibility of the AI algorithms in the cardiovascular field[47], and the first AI-ECG prospective study based on DL (unsupervised system) has been recently published[48] showing that the study arm had increased diagnostic accuracy.

Nowadays, the utilization of AI in the field of echocardiography is characterized by a scarcity of randomized control trials. The objective of the Prospective Randomized Controlled Trial Evaluating the Use of Artificial Intelligence in Stress Echocardiography (ISRCTN15113915) trial is to evaluate the benefit of using an AI model for a complex modality like stress echocardiography. If we think how the upcoming studies in AI in echocardiography look like, we could affirm that it is likely that AI will be used for the assessment of diagnostic test performance in the field of the automatization of measurements and automatic interpretations of the set of images obtained during the echocardiographic studies. Nevertheless, future studies in the field of echocardiography will explore the capability of AI to identify diagnostic or prognostic insights that may elude human comprehension, as has been done for ECG exams, and AI has the potential to transform ECG into a tool that can accurately detect conditions that were previously undetectable using conventional methods[48].

As fresh data emerges in this field, it becomes crucial, without any hesitation, to establish a comprehensive plan and framework for integrating AI into daily clinical practice, including its incorporation into clinical practice guidelines. These guidelines should encompass various domains, undoubtedly encompassing the utilization of AI for diagnosis assistance as well as clinical management and prognostication. One indispensable aspect of these clinical guidelines is to provide clinicians with a curated body of evidence that they can be relied upon for patient care. This evidence must possess the following attributes: Validity, accuracy, generalizability, safety, and fairness. The initial step involves promoting the standardization of study design and reporting when utilizing AI in conjunction with echocardiography. The CONSORT-AI initiative has already taken a pivotal stride in this direction, and it is recommended that this or similar standardized reporting formats be endorsed in all future studies[49].

ML has given us a greater ability to build predictive models, and now we are facing due to these advances a translational crisis[50]. Two studies found four essential stages in the process of creating a predictive model that could advance to a practical use: Development, validation, association with actual patient outcomes, and knowledge transition to a wide applicability[51,52]. They revised more than 800 predictive models for cardiovascular diseases previously published and found that a great part of them were improperly validated and only 0.1% were widely used finally in clinical practice.

One open question needs to be answered: Will the use of AI replace echocardiographers? Probably we could answer that not anytime soon. When considering the outcomes generated by AI, it is essential to interpret them in conjunction with the additional information obtained from echocardiography and stress testing. Nonetheless, AI holds promising potential for enhancing the reproducibility and efficiency of echocardiography. Cardiologists should make an effort to comprehend the AI tools and be ready to validate their effectiveness. For instance, the integration of AI into stress echocardiography should not be perceived as a threat but rather as an exceptional opportunity to further amplify the advantages of an already highly valuable test[53].

Considering the multitude of encouraging and promising outcomes, it begs the question why we have not implemented them in clinical practice. Should not our ethical obligation be to utilize every available resource to provide the best patient care? Or perhaps we doubt the authenticity of these findings? Alternatively, could it be that we are simply apprehensive about the potential threat to our roles if AI technology starts assuming some of our current tasks? Throughout human history, it has been demonstrated that our opinions hold little significance in the grand scheme of things. We may disagree with or even resist these technological advancements, but ultimately, they will prevail. Why? Because, in the long run, they prove to be beneficial for everyone involved[54].

Another concern that arises is how to address situations when there is a discrepancy between the machine and human opinions. As the accurate validation of algorithms holds immense importance, it is vital to underscore the necessity for it. While utilizing AI to supplement clinical decision-making, it is imperative for the physician to exercise their judgment while maintaining a degree of modesty. Unsupervised ML, specifically the advanced methods used in DL, may have limitations in terms of explanation ability (the ability to clarify how the algorithm produced its results). Despite these challenges, the potential of AI applications in healthcare is promising, and its capacity to enhance medical practices is remarkable.

CONCLUSION

Echocardiography is an essential asset in diagnostic cardiology and is really necessary to clinical care. AI is helping healthcare providers as a worthy diagnostic tool for physicians in the field of echocardiography specially on the automation of measurements and interpretation of results. Furthermore, it will help to expand the abilities of research and find alternative pathways in medical management specially on prognostication. For sure many obstacles or barriers need to be broken to reach a whole integration of AI in the echocardiography lab workflow.