1
|
Souza LA, Passos LA, Santana MCS, Mendel R, Rauber D, Ebigbo A, Probst A, Messmann H, Papa JP, Palm C. Layer-selective deep representation to improve esophageal cancer classification. Med Biol Eng Comput 2024; 62:3355-3372. [PMID: 38848031 DOI: 10.1007/s11517-024-03142-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2023] [Accepted: 05/25/2024] [Indexed: 10/17/2024]
Abstract
Even though artificial intelligence and machine learning have demonstrated remarkable performances in medical image computing, their accountability and transparency level must be improved to transfer this success into clinical practice. The reliability of machine learning decisions must be explained and interpreted, especially for supporting the medical diagnosis. For this task, the deep learning techniques' black-box nature must somehow be lightened up to clarify its promising results. Hence, we aim to investigate the impact of the ResNet-50 deep convolutional design for Barrett's esophagus and adenocarcinoma classification. For such a task, and aiming at proposing a two-step learning technique, the output of each convolutional layer that composes the ResNet-50 architecture was trained and classified for further definition of layers that would provide more impact in the architecture. We showed that local information and high-dimensional features are essential to improve the classification for our task. Besides, we observed a significant improvement when the most discriminative layers expressed more impact in the training and classification of ResNet-50 for Barrett's esophagus and adenocarcinoma classification, demonstrating that both human knowledge and computational processing may influence the correct learning of such a problem.
Collapse
Affiliation(s)
- Luis A Souza
- Department of Informatics, Espírito Santo Federal University, Vitória, Brazil.
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg, Germany.
| | - Leandro A Passos
- CMI Lab, School of Engineering and Informatics, University of Wolverhampton, Wolverhampton, UK
| | | | - Robert Mendel
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg, Germany
| | - David Rauber
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg, Germany
| | - Alanna Ebigbo
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Andreas Probst
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Helmut Messmann
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - João Paulo Papa
- Department of Computing, São Paulo State University, Bauru, Brazil
| | - Christoph Palm
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg, Germany
| |
Collapse
|
2
|
Kang SM, Lee GP, Kim YJ, Kim KO, Kim KG. Deep Learning Models for Anatomical Location Classification in Esophagogastroduodenoscopy Images and Videos: A Quantitative Evaluation with Clinical Data. Diagnostics (Basel) 2024; 14:2360. [PMID: 39518328 PMCID: PMC11545494 DOI: 10.3390/diagnostics14212360] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2024] [Revised: 10/14/2024] [Accepted: 10/21/2024] [Indexed: 11/16/2024] Open
Abstract
BACKGROUND/OBJECTIVES During gastroscopy, accurately identifying the anatomical locations of the gastrointestinal tract is crucial for developing diagnostic aids, such as lesion localization and blind spot alerts. METHODS This study utilized a dataset of 31,403 still images from 1000 patients with normal findings to annotate the anatomical locations within the images and develop a classification model. The model was then applied to videos of 20 esophagogastroduodenoscopy procedures, where it was validated for real-time location prediction. To address instability of predictions caused by independent frame-by-frame assessment, we implemented a hard-voting-based post-processing algorithm that aggregates results from seven consecutive frames, improving the overall accuracy. RESULTS Among the tested models, InceptionV3 demonstrated superior performance for still images, achieving an F1 score of 79.79%, precision of 80.57%, and recall of 80.08%. For video data, the InceptionResNetV2 model performed best, achieving an F1 score of 61.37%, precision of 73.08%, and recall of 57.21%. These results indicate that the deep learning models not only achieved high accuracy in position recognition for still images but also performed well on video data. Additionally, the post-processing algorithm effectively stabilized the predictions, highlighting its potential for real-time endoscopic applications. CONCLUSIONS This study demonstrates the feasibility of predicting the gastrointestinal tract locations during gastroscopy and suggests a promising path for the development of advanced diagnostic aids to assist clinicians. Furthermore, the location information generated by this model can be leveraged in future technologies, such as automated report generation and supporting follow-up examinations for patients.
Collapse
Affiliation(s)
- Seong Min Kang
- Medical Device R&D Center, Gachon University Gil Hospital, Incheon 21565, Republic of Korea;
| | - Gi Pyo Lee
- Department of Biomedical Engineering, Gachon University, Seongnam-si 13120, Republic of Korea;
| | - Young Jae Kim
- Gachon Biomedical & Convergence Institute, Gachon University Gil Medical Center, Incheon 21565, Republic of Korea;
| | - Kyoung Oh Kim
- Department of Internal Medicine, Gachon University Gil Hospital, Incheon 21565, Republic of Korea;
| | - Kwang Gi Kim
- Department of Biomedical Engineering, Gil Medical Center, Gachon University College of Medicine, Incheon 21565, Republic of Korea
| |
Collapse
|
3
|
Pornvoraphat P, Tiankanon K, Pittayanon R, Nupairoj N, Vateekul P, Rerknimitr R. Real-time gastric intestinal metaplasia segmentation using a deep neural network designed for multiple imaging modes on high-resolution images. Knowl Based Syst 2024; 300:112213. [DOI: 10.1016/j.knosys.2024.112213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2025]
|
4
|
Janaki R, Lakshmi D. Hybrid model-based early diagnosis of esophageal disorders using convolutional neural network and refined logistic regression. EURASIP JOURNAL ON IMAGE AND VIDEO PROCESSING 2024; 2024:19. [DOI: 10.1186/s13640-024-00634-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/20/2024] [Accepted: 07/28/2024] [Indexed: 01/04/2025]
|
5
|
Hou M, Wang J, Liu T, Li Z, Hounye AH, Liu X, Wang K, Chen S. A graph-optimized deep learning framework for recognition of Barrett’s esophagus and reflux esophagitis. MULTIMEDIA TOOLS AND APPLICATIONS 2024; 83:83747-83767. [DOI: 10.1007/s11042-024-18910-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 01/12/2024] [Accepted: 03/11/2024] [Indexed: 01/03/2025]
|
6
|
Pornvoraphat P, Tiankanon K, Pittayanon R, Sunthornwetchapong P, Vateekul P, Rerknimitr R. Real-time gastric intestinal metaplasia diagnosis tailored for bias and noisy-labeled data with multiple endoscopic imaging. Comput Biol Med 2023; 154:106582. [PMID: 36738708 DOI: 10.1016/j.compbiomed.2023.106582] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 01/11/2023] [Accepted: 01/22/2023] [Indexed: 01/26/2023]
Abstract
This work presents real-time segmentation viz. gastric intestinal metaplasia (GIM). Recently, GIM segmentation of endoscopic images has been carried out to differentiate GIM from a healthy stomach. However, real-time detection is difficult to achieve. Conditions are challenging, and include multiple color modes (white light endoscopy and narrow-band imaging), other abnormal lesions (erosion and ulcer), noisy labels etc. Herein, our model is based on BiSeNet and can overcome the many issues regarding GIM. Application of auxiliary head and additional loss are seen to improve performance as well as enhance multiple color modes accurately. Further, multiple pre-processing techniques are utilized for leveraging detection performance: namely, location-wise negative sampling, jigsaw augmentation, and label smoothing. Finally, the decision threshold can be adjusted separately for each color mode. Work undertaken at King Chulalongkorn Memorial Hospital examined 940 histologically proven GIM images and 1239 non-GIM images, obtained over 173 frames per second (FPS). In terms of accuracy, our model is seen to outperform all baselines. Our results demonstrate sensitivity, specificity, positive predictive, negative predictive, accuracy, and mean intersection over union (IoU), achieving GIM segmentation values of 91%, 96%, 91%, 91%, 96%, and 55%, respectively.
Collapse
Affiliation(s)
- Passin Pornvoraphat
- Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Pathumwan, Bangkok, Thailand
| | - Kasenee Tiankanon
- Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital, Bangkok, Thailand
| | - Rapat Pittayanon
- Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital, Bangkok, Thailand
| | - Phanukorn Sunthornwetchapong
- Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Pathumwan, Bangkok, Thailand
| | - Peerapon Vateekul
- Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Pathumwan, Bangkok, Thailand.
| | - Rungsun Rerknimitr
- Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital, Bangkok, Thailand.
| |
Collapse
|
7
|
Alharbe NR, Munshi RM, Khayyat MM, Khayyat MM, Abdalaha Hamza SH, Aljohani AA. Atom Search Optimization with the Deep Transfer Learning-Driven Esophageal Cancer Classification Model. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:4629178. [PMID: 36156959 PMCID: PMC9507698 DOI: 10.1155/2022/4629178] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Revised: 07/13/2022] [Accepted: 08/10/2022] [Indexed: 11/20/2022]
Abstract
Esophageal cancer (EC) is a commonly occurring malignant tumor that significantly affects human health. Earlier recognition and classification of EC or premalignant lesions can result in highly effective targeted intervention. Accurate detection and classification of distinct stages of EC provide effective precision therapy planning and improve the 5-year survival rate. Automated recognition of EC can aid physicians in improving diagnostic performance and accuracy. However, the classification of EC is challenging due to identical endoscopic features, like mucosal erosion, hyperemia, and roughness. The recent developments of deep learning (DL) and computer-aided diagnosis (CAD) models have been useful for designing accurate EC classification models. In this aspect, this study develops an atom search optimization with a deep transfer learning-driven EC classification (ASODTL-ECC) model. The presented ASODTL-ECC model mainly examines the medical images for the existence of EC in a timely and accurate manner. To do so, the presented ASODTL-ECC model employs Gaussian filtering (GF) as a preprocessing stage to enhance image quality. In addition, the deep convolution neural network- (DCNN-) based residual network (ResNet) model is applied as a feature extraction approach. Besides, ASO with an extreme learning machine (ELM) model is utilized for identifying the presence of EC, showing the novelty of the work. The performance of the ASODTL-ECC model is assessed and compared with existing models under several medical images. The experimental results pointed out the improved performance of the ASODTL-ECC model over recent approaches.
Collapse
Affiliation(s)
| | - Raafat M. Munshi
- Department of Medical Laboratory Technology (MLT), Faculty of Applied Medical Sciences, King Abdulaziz University, Rabigh, Saudi Arabia
| | - Manal M. Khayyat
- Department of Information Systems, College of Computers and Information Systems, Umm Al-Qura University, Makkah, Saudi Arabia
| | - Mashael M. Khayyat
- Department of Information Systems and Technology, Faculty of Computer Science and Engineering, University of Jeddah, Jeddah, Saudi Arabia
| | - Saadia Hassan Abdalaha Hamza
- Department of Computer Science College of Science and Humanities in Al-Sulail, Prince Sattam Bin Abdulaziz University, Saudi Arabia
| | | |
Collapse
|
8
|
Zhou Y, Yuan X, Zhang X, Liu W, Wu Y, Yen GG, Hu B, Yi Z. Evolutionary Neural Architecture Search for Automatic Esophageal Lesion Identification and Segmentation. IEEE TRANSACTIONS ON ARTIFICIAL INTELLIGENCE 2022; 3:436-450. [DOI: 10.1109/tai.2021.3134600] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Affiliation(s)
- Yao Zhou
- Center of Intelligent Medicine, College of Computer Science, Sichuan University, Chengdu, China
| | - Xianglei Yuan
- Department of Gastroenterology, West China Hospital, Sichuan University, Chengdu, China
| | - Xiaozhi Zhang
- Center of Intelligent Medicine, College of Computer Science, Sichuan University, Chengdu, China
| | - Wei Liu
- Department of Gastroenterology, West China Hospital, Sichuan University, Chengdu, China
| | - Yu Wu
- Center of Intelligent Medicine, College of Computer Science, Sichuan University, Chengdu, China
| | - Gary G. Yen
- School of Electrical and Computer Engineering, Oklahoma State University, Stillwater, OK, USA
| | - Bing Hu
- Department of Gastroenterology, West China Hospital, Sichuan University, Chengdu, China
| | - Zhang Yi
- Center of Intelligent Medicine, College of Computer Science, Sichuan University, Chengdu, China
| |
Collapse
|
9
|
Wang Y, Zhu C, Wang Y, Sun J, Ling D, Wang L. Survival risk prediction model for ESCC based on relief feature selection and CNN. Comput Biol Med 2022; 145:105460. [PMID: 35364307 DOI: 10.1016/j.compbiomed.2022.105460] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Revised: 03/23/2022] [Accepted: 03/24/2022] [Indexed: 01/10/2023]
Abstract
Esophageal squamous cell carcinoma (ESCC) is a common malignant tumor of the digestive system with poor prognosis and high mortality. It is of great significance to predict the prognosis risk of patients with cancer by using medical pathology information. To take full advantage of the clinic pathological information of ESCC patients and improve the accuracy of postoperative survival risk prediction, this paper proposes an ESCC survival risk prediction model based on Relief feature selection and convolutional neural network (CNN). Firstly, statistical analysis methods and relief feature selection algorithm are used to extract the important risk factors related to the survival risk of patients. Then, One-dimensional convolutional neural network (1D-CNN) is used to establish the survival risk prediction model of patients with esophageal cancer. Finally, the data of patients with esophageal cancer provided by the First Affiliated Hospital of Zhengzhou University is used to assess the performance of the model. The results show that the model proposed in this paper has a high accuracy rate, which can effectively predict the postoperative survival risk of the patient through the clinical phenotypic index of the patient.
Collapse
Affiliation(s)
- Yanfeng Wang
- School of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, 45002, China
| | - Chuanqian Zhu
- School of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, 45002, China
| | - Yan Wang
- School of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, 45002, China.
| | - Junwei Sun
- School of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, 45002, China
| | - Dan Ling
- School of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, 45002, China
| | - Lidong Wang
- State Key Laboratory of Esophageal Cancer Prevention, Treatment and Henan Key Laboratory for Esophageal Cancer Research of The First Affiliated Hospital, Zhengzhou University, Zhengzhou, 450052, China
| |
Collapse
|
10
|
AIM in Endoscopy Procedures. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
11
|
Correia FP, Lourenço LC. Artificial intelligence application in diagnostic gastrointestinal endoscopy - Deus ex machina? World J Gastroenterol 2021; 27:5351-5361. [PMID: 34539137 PMCID: PMC8409168 DOI: 10.3748/wjg.v27.i32.5351] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Revised: 05/15/2021] [Accepted: 07/19/2021] [Indexed: 02/06/2023] Open
Abstract
The close relationship of medicine with technology and the particular interest in this symbiosis in recent years has led to the development of several computed artificial intelligence (AI) systems aimed at various areas of medicine. A number of studies have demonstrated that those systems allow accurate diagnoses with histological precision, thus facilitating decision-making by clinicians in real time. In the field of gastroenterology, AI has been applied in the diagnosis of pathologies of the entire digestive tract and their attached glands, and are increasingly accepted for the detection of colorectal polyps and confirming their histological classification. Studies have shown high accuracy, sensitivity, and specificity in relation to expert endoscopists, and mainly in relation to those with less experience. Other applications that are increasingly studied and with very promising results are the investigation of dysplasia in patients with Barrett's esophagus and the endoscopic and histological assessment of colon inflammation in patients with ulcerative colitis. In some cases AI is thus better than or at least equal to human abilities. However, additional studies are needed to reinforce the existing data, and mainly to determine the applicability of this technology in other indications. This review summarizes the state of the art of AI in gastroenterological pathology.
Collapse
Affiliation(s)
- Fábio Pereira Correia
- Department of Gastroenterology, Hospital Prof. Dr Fernando Fonseca, Lisbon 2720-276, Portugal
| | - Luís Carvalho Lourenço
- Department of Gastroenterology, Hospital Prof. Dr Fernando Fonseca, Lisbon 2720-276, Portugal
| |
Collapse
|
12
|
McShane R, Arya S, Stewart AJ, Caie P, Bates M. Prognostic features of the tumour microenvironment in oesophageal adenocarcinoma. Biochim Biophys Acta Rev Cancer 2021; 1876:188598. [PMID: 34332022 DOI: 10.1016/j.bbcan.2021.188598] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2021] [Revised: 07/26/2021] [Accepted: 07/26/2021] [Indexed: 12/12/2022]
Abstract
Oesophageal adenocarcinoma (OAC) is a disease with an incredibly poor survival rate and a complex makeup. The growth and spread of OAC tumours are profoundly influenced by their surrounding microenvironment and the properties of the tumour itself. Constant crosstalk between the tumour and its microenvironment is key to the survival of the tumour and ultimately the death of the patient. The tumour microenvironment (TME) is composed of a complex milieu of cell types including cancer associated fibroblasts (CAFs) which make up the tumour stroma, endothelial cells which line blood and lymphatic vessels and infiltrating immune cell populations. These various cell types and the tumour constantly communicate through environmental cues including fluctuations in pH, hypoxia and the release of mitogens such as cytokines, chemokines and growth factors, many of which help promote malignant progression. Eventually clusters of tumour cells such as tumour buds break away and spread through the lymphatic system to nearby lymph nodes or enter the circulation forming secondary metastasis. Collectively, these factors need to be considered when assessing and treating patients clinically. This review aims to summarise the ways in which these various factors are currently assessed and how they relate to patient treatment and outcome at an individual level.
Collapse
Affiliation(s)
| | - Swati Arya
- School of Medicine, University of St Andrews, Fife, UK
| | | | - Peter Caie
- School of Medicine, University of St Andrews, Fife, UK
| | - Mark Bates
- Department of Surgery, Trinity Translational Medicine Institute, St. James's Hospital, Dublin 8, Ireland; Trinity St James's Cancer Institute, St James's Hospital, Dublin 8, Ireland.
| |
Collapse
|
13
|
Zhang L, Sun B, Zhou X, Wei Q, Liang S, Luo G, Li T, Lü M. Barrett's Esophagus and Intestinal Metaplasia. Front Oncol 2021; 11:630837. [PMID: 34221959 PMCID: PMC8252963 DOI: 10.3389/fonc.2021.630837] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Accepted: 05/31/2021] [Indexed: 02/05/2023] Open
Abstract
Intestinal metaplasia refers to the replacement of the differentiated and mature normal mucosal epithelium outside the intestinal tract by the intestinal epithelium. This paper briefly describes the etiology and clinical significance of intestinal metaplasia in Barrett’s esophagus. This article summarizes the impact of intestinal metaplasia on the diagnosis, monitoring, and treatment of Barrett’s esophagus according to different guidelines. We also briefly explore the basis for the endoscopic diagnosis of intestinal metaplasia in Barrett’s esophagus. The identification techniques of goblet cells in Barrett’s esophagus are also elucidated by some scholars. Additionally, we further elaborate on the current treatment methods related to Barrett’s esophagus.
Collapse
Affiliation(s)
- Lu Zhang
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou City, China
| | - Binyu Sun
- Department of Endoscope, Public Health Clinical Medical Center of Chengdu, Chengdu City, China
| | - Xi Zhou
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou City, China
| | - QiongQiong Wei
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou City, China
| | - Sicheng Liang
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou City, China
| | - Gang Luo
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou City, China
| | - Tao Li
- Department of Anesthesiology, West China Hospital, Sichuan University, Chengdu City, China
| | - Muhan Lü
- Department of Gastroenterology, The Affiliated Hospital of Southwest Medical University, Luzhou City, China
| |
Collapse
|
14
|
de Souza LA, Mendel R, Strasser S, Ebigbo A, Probst A, Messmann H, Papa JP, Palm C. Convolutional Neural Networks for the evaluation of cancer in Barrett's esophagus: Explainable AI to lighten up the black-box. Comput Biol Med 2021; 135:104578. [PMID: 34171639 DOI: 10.1016/j.compbiomed.2021.104578] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Revised: 06/11/2021] [Accepted: 06/12/2021] [Indexed: 01/10/2023]
Abstract
Even though artificial intelligence and machine learning have demonstrated remarkable performances in medical image computing, their level of accountability and transparency must be provided in such evaluations. The reliability related to machine learning predictions must be explained and interpreted, especially if diagnosis support is addressed. For this task, the black-box nature of deep learning techniques must be lightened up to transfer its promising results into clinical practice. Hence, we aim to investigate the use of explainable artificial intelligence techniques to quantitatively highlight discriminative regions during the classification of early-cancerous tissues in Barrett's esophagus-diagnosed patients. Four Convolutional Neural Network models (AlexNet, SqueezeNet, ResNet50, and VGG16) were analyzed using five different interpretation techniques (saliency, guided backpropagation, integrated gradients, input × gradients, and DeepLIFT) to compare their agreement with experts' previous annotations of cancerous tissue. We could show that saliency attributes match best with the manual experts' delineations. Moreover, there is moderate to high correlation between the sensitivity of a model and the human-and-computer agreement. The results also lightened that the higher the model's sensitivity, the stronger the correlation of human and computational segmentation agreement. We observed a relevant relation between computational learning and experts' insights, demonstrating how human knowledge may influence the correct computational learning.
Collapse
Affiliation(s)
- Luis A de Souza
- Department of Computing, São Carlos Federal University - UFSCar, Brazil; Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany
| | - Robert Mendel
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany
| | - Sophia Strasser
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany
| | - Alanna Ebigbo
- Medizinische Klinik III, Universitätsklinikum Augsburg, Germany
| | - Andreas Probst
- Medizinische Klinik III, Universitätsklinikum Augsburg, Germany
| | - Helmut Messmann
- Medizinische Klinik III, Universitätsklinikum Augsburg, Germany
| | - João P Papa
- Department of Computing, São Paulo State University, UNESP, Brazil.
| | - Christoph Palm
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany; Regensburg Center of Health Sciences and Technology (RCHST), OTH Regensburg, Germany
| |
Collapse
|
15
|
Zhang SM, Wang YJ, Zhang ST. Accuracy of artificial intelligence-assisted detection of esophageal cancer and neoplasms on endoscopic images: A systematic review and meta-analysis. J Dig Dis 2021; 22:318-328. [PMID: 33871932 PMCID: PMC8361665 DOI: 10.1111/1751-2980.12992] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/22/2021] [Revised: 03/02/2021] [Accepted: 04/15/2021] [Indexed: 12/14/2022]
Abstract
OBJECTIVE To investigate systematically previous studies on the accuracy of artificial intelligence (AI)-assisted diagnostic models in detecting esophageal neoplasms on endoscopic images so as to provide scientific evidence for the effectiveness of these models. METHODS A literature search was conducted on the PubMed, EMBASE and Cochrane Library databases for studies on the AI-assisted detection of esophageal neoplasms on endoscopic images published up to December 2020. A bivariate mixed-effects regression model was used to calculate the pooled diagnostic efficacy of AI-assisted system. Subgroup analyses and meta-regression analyses were performed to explore the sources of heterogeneity. The effectiveness of AI-assisted models was also compared with that of the endoscopists. RESULTS Sixteen studies were included in the systematic review and meta-analysis. The pooled sensitivity, specificity, positive and negative likelihood ratios, diagnostic odds ratio and area under the summary receiver operating characteristic curve regarding AI-assisted detection of esophageal neoplasms were 94% (95% confidence interval [CI] 92%-96%), 85% (95% CI 73%-92%), 6.40 (95% CI 3.38-12.11), 0.06 (95% CI 0.04-0.10), 98.88 (95% CI 39.45-247.87) and 0.97 (95% CI 0.95-0.98), respectively. AI-based models performed better than endoscopists in terms of the pooled sensitivity (94% [95% CI 84%-98%] vs 82% [95% CI 77%-86%, P < 0.01). CONCLUSIONS The use of AI results in increased accuracy in detecting early esophageal cancer. However, most of the included studies have a retrospective study design, thus further validation with prospective trials is required.
Collapse
Affiliation(s)
- Si Min Zhang
- Department of GastroenterologyBeijing Friendship Hospital, Capital Medical UniversityBeijingChina,National Clinical Research Center for Digestive DiseasesBeijingChina,Beijing Digestive Disease CenterBeijingChina
| | - Yong Jun Wang
- Department of GastroenterologyBeijing Friendship Hospital, Capital Medical UniversityBeijingChina,National Clinical Research Center for Digestive DiseasesBeijingChina,Beijing Digestive Disease CenterBeijingChina
| | - Shu Tian Zhang
- Department of GastroenterologyBeijing Friendship Hospital, Capital Medical UniversityBeijingChina,National Clinical Research Center for Digestive DiseasesBeijingChina,Beijing Digestive Disease CenterBeijingChina
| |
Collapse
|
16
|
Choi SJ, Khan MA, Choi HS, Choo J, Lee JM, Kwon S, Keum B, Chun HJ. Development of artificial intelligence system for quality control of photo documentation in esophagogastroduodenoscopy. Surg Endosc 2021; 36:57-65. [PMID: 33415420 DOI: 10.1007/s00464-020-08236-6] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2020] [Accepted: 12/08/2020] [Indexed: 11/24/2022]
Abstract
BACKGROUND Esophagogastroduodenoscopy (EGD) is generally a safe procedure, but adverse events often occur. This highlights the necessity of the quality control of EGD. Complete visualization and photo documentation of upper gastrointestinal (UGI) tracts are important measures in quality control of EGD. To evaluate these measures in large scale, we developed an AI-driven quality control system for EGD through convolutional neural networks (CNNs) using archived endoscopic images. METHODS We retrospectively collected and labeled images from 250 EGD procedures, a total of 2599 images from eight locations of the UGI tract, using the European Society of Gastrointestinal Endoscopy (ESGE) photo documentation methods. The label confirmed by five experts was considered the gold standard. We developed a CNN model for multi-class classification of EGD images to one of the eight locations and binary classification of each EGD procedure based on its completeness. RESULTS Our CNN model successfully classified the EGD images into one of the eight regions of UGI tracts with 97.58% accuracy, 97.42% sensitivity, 99.66% specificity, 97.50% positive predictive value (PPV), and 99.66% negative predictive value (NPV). Our model classified the completeness of EGD with 89.20% accuracy, 89.20% sensitivity, 100.00% specificity, 100.00% PPV, and 64.94% NPV. We analyzed the credibility of our model using a probability heatmap. CONCLUSIONS We constructed a CNN model that could be used in the quality control of photo documentation in EGD. Our model needs further validation with a large dataset, and we expect our model to help both endoscopists and patients by improving the quality of EGD procedures.
Collapse
Affiliation(s)
- Seong Ji Choi
- Department of Internal Medicine, Hanyang University College of Medicine, Seoul, Republic of Korea
| | - Mohammad Azam Khan
- Graduate School of Artificial Intelligence, KAIST, Daejeon, Republic of Korea
| | - Hyuk Soon Choi
- Division of Gastroenterology and Hepatology, Department of Internal Medicine, Korea University College of Medicine, Seoul, Republic of Korea.
| | - Jaegul Choo
- Graduate School of Artificial Intelligence, KAIST, Daejeon, Republic of Korea.
| | - Jae Min Lee
- Division of Gastroenterology and Hepatology, Department of Internal Medicine, Korea University College of Medicine, Seoul, Republic of Korea
| | - Soonwook Kwon
- Department of Anatomy, School of Medicine, Catholic University of Daegu, Daegu, Republic of Korea
| | - Bora Keum
- Division of Gastroenterology and Hepatology, Department of Internal Medicine, Korea University College of Medicine, Seoul, Republic of Korea
| | - Hoon Jai Chun
- Division of Gastroenterology and Hepatology, Department of Internal Medicine, Korea University College of Medicine, Seoul, Republic of Korea
| |
Collapse
|
17
|
Kolb JM, Wani S. Barrett's esophagus: current standards in advanced imaging. Transl Gastroenterol Hepatol 2021; 6:14. [PMID: 33409408 DOI: 10.21037/tgh.2020.02.10] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/21/2019] [Accepted: 01/21/2020] [Indexed: 12/13/2022] Open
Abstract
Esophageal adenocarcinoma (EAC) continues to be one of the fastest rising incident cancers in the Western population with the majority of patients presenting with late stage disease and associated with a dismal 5-year survival rate. Barrett's esophagus (BE) is the only identifiable precursor lesion to EAC. Strategies to screen for and survey BE are critical to detect earlier cancers and reduce morbidity and mortality related to EAC. A high-quality endoscopic examination with careful inspection of the Barrett's segment and adherence to the Seattle protocol for tissue sampling are critical. Advanced imaging modalities offer the potential to improve dysplasia detection, predict histopathology in real time and guide endoscopic eradication therapy (EET). Several technologies have been studied and although most are not yet recommended for routine clinical practice, high definition white light endoscopy (HD-WLE) as well as chromoendoscopy (including virtual chromoendoscopy) improved dysplasia detection in numerous studies supporting their use. Future studies should evaluate the role of artificial intelligence in optimizing detection of dysplasia in BE patients.
Collapse
Affiliation(s)
- Jennifer M Kolb
- Division of Gastroenterology & Hepatology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Sachin Wani
- Division of Gastroenterology & Hepatology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| |
Collapse
|
18
|
Ghatwary N, Zolgharni M, Janan F, Ye X. Learning Spatiotemporal Features for Esophageal Abnormality Detection From Endoscopic Videos. IEEE J Biomed Health Inform 2021; 25:131-142. [PMID: 32750901 DOI: 10.1109/jbhi.2020.2995193] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Esophageal cancer is categorized as a type of disease with a high mortality rate. Early detection of esophageal abnormalities (i.e. precancerous and early cancerous) can improve the survival rate of the patients. Recent deep learning-based methods for selected types of esophageal abnormality detection from endoscopic images have been proposed. However, no methods have been introduced in the literature to cover the detection from endoscopic videos, detection from challenging frames and detection of more than one esophageal abnormality type. In this paper, we present an efficient method to automatically detect different types of esophageal abnormalities from endoscopic videos. We propose a novel 3D Sequential DenseConvLstm network that extracts spatiotemporal features from the input video. Our network incorporates 3D Convolutional Neural Network (3DCNN) and Convolutional Lstm (ConvLstm) to efficiently learn short and long term spatiotemporal features. The generated feature map is utilized by a region proposal network and ROI pooling layer to produce a bounding box that detects abnormality regions in each frame throughout the video. Finally, we investigate a post-processing method named Frame Search Conditional Random Field (FS-CRF) that improves the overall performance of the model by recovering the missing regions in neighborhood frames within the same clip. We extensively validate our model on an endoscopic video dataset that includes a variety of esophageal abnormalities. Our model achieved high performance using different evaluation metrics showing 93.7% recall, 92.7% precision, and 93.2% F-measure. Moreover, as no results have been reported in the literature for the esophageal abnormality detection from endoscopic videos, to validate the robustness of our model, we have tested the model on a publicly available colonoscopy video dataset, achieving the polyp detection performance in a recall of 81.18%, precision of 96.45% and F-measure 88.16%, compared to the state-of-the-art results of 78.84% recall, 90.51% precision and 84.27% F-measure using the same dataset. This demonstrates that the proposed method can be adapted to different gastrointestinal endoscopic video applications with a promising performance.
Collapse
|
19
|
Abstract
Barrett's esophagus is the precursor lesion for esophageal adenocarcinoma. The goals of endoscopic surveillance are to detect dysplasia and early esophageal adenocarcinoma in order to improve patient outcomes. Despite the ongoing debate regarding the efficacy of surveillance, all current gastrointestinal societies recommend surveillance at this time. Optimal surveillance technique includes adequate inspection time, evaluation using high-definition white light and chromoendoscopy, appropriate documentation of the metaplastic segment using the Prague C & M criteria as well as the Paris classification should lesions be found, utilization of the Seattle biopsy protocol, and endoscopic resection of visible lesions.
Collapse
Affiliation(s)
- Joseph R. Triggs
- Clinical Instructor, Division of Gastroenterology. Hospital of the University of Pennsylvania. University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Gary W. Falk
- Professor of Medicine, Division of Gastroenterology, Hospital of the University of Pennsylvania. University of Pennsylvania Perelman School of Medicine Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
20
|
Marzullo A, Moccia S, Calimeri F, De Momi E. AIM in Endoscopy Procedures. Artif Intell Med 2021. [DOI: 10.1007/978-3-030-58080-3_164-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2022]
|
21
|
Sinonquel P, Eelbode T, Bossuyt P, Maes F, Bisschops R. Artificial intelligence and its impact on quality improvement in upper and lower gastrointestinal endoscopy. Dig Endosc 2021; 33:242-253. [PMID: 33145847 DOI: 10.1111/den.13888] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Revised: 10/14/2020] [Accepted: 11/01/2020] [Indexed: 12/24/2022]
Abstract
Artificial intelligence (AI) and its application in medicine has grown large interest. Within gastrointestinal (GI) endoscopy, the field of colonoscopy and polyp detection is the most investigated, however, upper GI follows the lead. Since endoscopy is performed by humans, it is inherently an imperfect procedure. Computer-aided diagnosis may improve its quality by helping prevent missing lesions and supporting optical diagnosis for those detected. An entire evolution in AI systems has been established in the last decades, resulting in optimization of the diagnostic performance with lower variability and matching or even outperformance of expert endoscopists. This shows a great potential for future quality improvement of endoscopy, given the outstanding diagnostic features of AI. With this narrative review, we highlight the potential benefit of AI to improve overall quality in daily endoscopy and describe the most recent developments for characterization and diagnosis as well as the recent conditions for regulatory approval.
Collapse
Affiliation(s)
- Pieter Sinonquel
- Department of Gastroenterology and Hepatology, University Hospitals Leuven, Leuven, Belgium.,Departments of, Department of, Translational Research in Gastrointestinal Diseases (TARGID), KU Leuven, Leuven, Belgium
| | - Tom Eelbode
- Medical Imaging Research Center (MIRC), University Hospitals Leuven, Leuven, Belgium.,Department of Electrical Engineering (ESAT/PSI), KU Leuven, Leuven, Belgium
| | - Peter Bossuyt
- Department of Gastroenterology and Hepatology, University Hospitals Leuven, Leuven, Belgium.,Department of Gastroenterology and Hepatology, Imelda Hospital, Bonheiden, Belgium
| | - Frederik Maes
- Medical Imaging Research Center (MIRC), University Hospitals Leuven, Leuven, Belgium.,Department of Electrical Engineering (ESAT/PSI), KU Leuven, Leuven, Belgium
| | - Raf Bisschops
- Department of Gastroenterology and Hepatology, University Hospitals Leuven, Leuven, Belgium.,Departments of, Department of, Translational Research in Gastrointestinal Diseases (TARGID), KU Leuven, Leuven, Belgium
| |
Collapse
|
22
|
Yan T, Wong PK, Choi IC, Vong CM, Yu HH. Intelligent diagnosis of gastric intestinal metaplasia based on convolutional neural network and limited number of endoscopic images. Comput Biol Med 2020; 126:104026. [PMID: 33059237 DOI: 10.1016/j.compbiomed.2020.104026] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2020] [Revised: 09/28/2020] [Accepted: 09/29/2020] [Indexed: 12/24/2022]
Abstract
BACKGROUND Gastric intestinal metaplasia (GIM) is a precancerous lesion of gastric cancer. Currently, diagnosis of GIM is based on the experience of a physician, which is liable to interobserver variability. Thus, an intelligent diagnostic (ID) system, based on narrow-band and magnifying narrow-band images, was constructed to provide objective assistance in the diagnosis of GIM. METHOD We retrospectively collected 1880 endoscopic images (1048 GIM and 832 non-GIM) via biopsy from 336 patients confirmed histologically as GIM or non-GIM, from the Kiang Wu Hospital, Macau. We developed an ID system with these images using a modified convolutional neural network algorithm. A separate test dataset containing 477 pathologically confirmed images (242 GIM and 235 non-GIM) from 80 patients was used to test the performance of the ID system. Experienced endoscopists also examined the same test dataset, for comparison with the ID system. One of the challenges faced in this study was that it was difficult to obtain a large number of training images. Thus, data augmentation and transfer learning were applied together. RESULTS The area under the receiver operating characteristic curve was 0.928 for the pre-patient analysis of the ID system, while the sensitivities, specificities, and accuracies of the ID system against those of the human experts were (91.9% vs. 86.5%, p-value = 1.000) (86.0% vs. 81.4%, p-value = 0.754), and (88.8% vs. 83.8%, p-value = 0.424), respectively. Even though the three indices of the ID system were slightly higher than those of the human experts, there were no significant differences. CONCLUSIONS In this pilot study, a novel ID system was developed to diagnose GIM. This system exhibits promising diagnostic performance. It is believed that the proposed system has the potential for clinical application in the future.
Collapse
Affiliation(s)
- Tao Yan
- School of Mechanical Engineering, Hubei University of Arts and Science, Xiangyang, 441053, China; Department of Electromechanical Engineering, University of Macau, Taipa, 999078, Macau, China
| | - Pak Kin Wong
- Department of Electromechanical Engineering, University of Macau, Taipa, 999078, Macau, China.
| | | | - Chi Man Vong
- Department of Computer and Information Science, University of Macau, Taipa, 999078, Macau, China
| | | |
Collapse
|
23
|
de Souza LA, Passos LA, Mendel R, Ebigbo A, Probst A, Messmann H, Palm C, Papa JP. Assisting Barrett's esophagus identification using endoscopic data augmentation based on Generative Adversarial Networks. Comput Biol Med 2020; 126:104029. [PMID: 33059236 DOI: 10.1016/j.compbiomed.2020.104029] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2020] [Revised: 09/08/2020] [Accepted: 09/30/2020] [Indexed: 12/12/2022]
Abstract
Barrett's esophagus figured a swift rise in the number of cases in the past years. Although traditional diagnosis methods offered a vital role in early-stage treatment, they are generally time- and resource-consuming. In this context, computer-aided approaches for automatic diagnosis emerged in the literature since early detection is intrinsically related to remission probabilities. However, they still suffer from drawbacks because of the lack of available data for machine learning purposes, thus implying reduced recognition rates. This work introduces Generative Adversarial Networks to generate high-quality endoscopic images, thereby identifying Barrett's esophagus and adenocarcinoma more precisely. Further, Convolution Neural Networks are used for feature extraction and classification purposes. The proposed approach is validated over two datasets of endoscopic images, with the experiments conducted over the full and patch-split images. The application of Deep Convolutional Generative Adversarial Networks for the data augmentation step and LeNet-5 and AlexNet for the classification step allowed us to validate the proposed methodology over an extensive set of datasets (based on original and augmented sets), reaching results of 90% of accuracy for the patch-based approach and 85% for the image-based approach. Both results are based on augmented datasets and are statistically different from the ones obtained in the original datasets of the same kind. Moreover, the impact of data augmentation was evaluated in the context of image description and classification, and the results obtained using synthetic images outperformed the ones over the original datasets, as well as other recent approaches from the literature. Such results suggest promising insights related to the importance of proper data for the accurate classification concerning computer-assisted Barrett's esophagus and adenocarcinoma detection.
Collapse
Affiliation(s)
- Luis A de Souza
- Department of Computing, São Carlos Federal University, UFSCar, Brazil; Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany
| | - Leandro A Passos
- Department of Computing, São Paulo State University, UNESP, Brazil
| | - Robert Mendel
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany; Regensburg Center of Health Sciences and Technology (RCHST), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany
| | - Alanna Ebigbo
- Department of Gastroenterology, University Hospital Augsburg, Germany
| | - Andreas Probst
- Department of Gastroenterology, University Hospital Augsburg, Germany
| | - Helmut Messmann
- Department of Gastroenterology, University Hospital Augsburg, Germany
| | - Christoph Palm
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany; Regensburg Center of Health Sciences and Technology (RCHST), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany
| | - João P Papa
- Department of Computing, São Paulo State University, UNESP, Brazil.
| |
Collapse
|
24
|
The Application of Artificial Intelligence in Prostate Cancer Management—What Improvements Can Be Expected? A Systematic Review. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10186428] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Artificial Intelligence (AI) is progressively remodeling our daily life. A large amount of information from “big data” now enables machines to perform predictions and improve our healthcare system. AI has the potential to reshape prostate cancer (PCa) management thanks to growing applications in the field. The purpose of this review is to provide a global overview of AI in PCa for urologists, pathologists, radiotherapists, and oncologists to consider future changes in their daily practice. A systematic review was performed, based on PubMed MEDLINE, Google Scholar, and DBLP databases for original studies published in English from January 2009 to January 2019 relevant to PCa, AI, Machine Learning, Artificial Neural Networks, Convolutional Neural Networks, and Natural-Language Processing. Only articles with full text accessible were considered. A total of 1008 articles were reviewed, and 48 articles were included. AI has potential applications in all fields of PCa management: analysis of genetic predispositions, diagnosis in imaging, and pathology to detect PCa or to differentiate between significant and non-significant PCa. AI also applies to PCa treatment, whether surgical intervention or radiotherapy, skills training, or assessment, to improve treatment modalities and outcome prediction. AI in PCa management has the potential to provide a useful role by predicting PCa more accurately, using a multiomic approach and risk-stratifying patients to provide personalized medicine.
Collapse
|
25
|
He Q, Bano S, Ahmad OF, Yang B, Chen X, Valdastri P, Lovat LB, Stoyanov D, Zuo S. Deep learning-based anatomical site classification for upper gastrointestinal endoscopy. Int J Comput Assist Radiol Surg 2020; 15:1085-1094. [PMID: 32377939 PMCID: PMC7316667 DOI: 10.1007/s11548-020-02148-5] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Accepted: 03/31/2020] [Indexed: 02/07/2023]
Abstract
Purpose Upper gastrointestinal (GI) endoscopic image documentation has provided an efficient, low-cost solution to address quality control for endoscopic reporting. The problem is, however, challenging for computer-assisted techniques, because different sites have similar appearances. Additionally, across different patients, site appearance variation may be large and inconsistent. Therefore, according to the British and modified Japanese guidelines, we propose a set of oesophagogastroduodenoscopy (EGD) images to be routinely captured and evaluate its efficiency for deep learning-based classification methods. Methods A novel EGD image dataset standardising upper GI endoscopy to several steps is established following landmarks proposed in guidelines and annotated by an expert clinician. To demonstrate the discrimination of proposed landmarks that enable the generation of an automated endoscopic report, we train several deep learning-based classification models utilising the well-annotated images. Results We report results for a clinical dataset composed of 211 patients (comprising a total of 3704 EGD images) acquired during routine upper GI endoscopic examinations. We find close agreement between predicted labels using our method and the ground truth labelled by human experts. We observe the limitation of current static image classification scheme for EGD image classification. Conclusion Our study presents a framework for developing automated EGD reports using deep learning. We demonstrate that our method is feasible to address EGD image classification and can lead towards improved performance and additionally qualitatively demonstrate its performance on our dataset.
Collapse
Affiliation(s)
- Qi He
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, China
| | - Sophia Bano
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK
| | - Omer F Ahmad
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK
| | - Bo Yang
- General Hospital, Tianjin Medical University, Tianjin, China
| | - Xin Chen
- General Hospital, Tianjin Medical University, Tianjin, China
| | - Pietro Valdastri
- School of Electronic and Electrical Engineering, University of Leeds, Leeds, UK
| | - Laurence B Lovat
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK
| | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK
| | - Siyang Zuo
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, China.
| |
Collapse
|
26
|
Franklin J, Jankowski J. Recent advances in understanding and preventing oesophageal cancer. F1000Res 2020; 9. [PMID: 32399195 PMCID: PMC7194479 DOI: 10.12688/f1000research.21971.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/03/2020] [Indexed: 12/24/2022] Open
Abstract
Oesophageal cancer is a common cancer that continues to have a poor survival. This is largely in part due to its late diagnosis and early metastatic spread. Currently, screening is limited to patients with multiple risk factors via a relatively invasive technique. However, there is a large proportion of patients diagnosed with oesophageal cancer who have not been screened. This has warranted the development of new screening techniques that could be implemented more widely and lead to earlier identification and subsequently improvements in survival rates. This article also explores progress in the surveillance of Barrett’s oesophagus, a pre-malignant condition for the development of oesophageal adenocarcinoma. In recent years, advances in early endoscopic intervention have meant that more patients are considered at an earlier stage for potentially curative treatment.
Collapse
Affiliation(s)
- James Franklin
- Gastroenterology and Endoscopy Department, Kings Mill Hospital NHS Foundation Trust, Sutton-in-Ashfield, Nottinghamshire, NG17 4JL, UK
| | - Janusz Jankowski
- Gastroenterology and Endoscopy Department, Kings Mill Hospital NHS Foundation Trust, Sutton-in-Ashfield, Nottinghamshire, NG17 4JL, UK
- University of Liverpool, Liverpool, UK
- University of Roehampton, London, UK
| |
Collapse
|
27
|
de Groof AJ, Struyvenberg MR, van der Putten J, van der Sommen F, Fockens KN, Curvers WL, Zinger S, Pouw RE, Coron E, Baldaque-Silva F, Pech O, Weusten B, Meining A, Neuhaus H, Bisschops R, Dent J, Schoon EJ, de With PH, Bergman JJ. Deep-Learning System Detects Neoplasia in Patients With Barrett's Esophagus With Higher Accuracy Than Endoscopists in a Multistep Training and Validation Study With Benchmarking. Gastroenterology 2020; 158:915-929.e4. [PMID: 31759929 DOI: 10.1053/j.gastro.2019.11.030] [Citation(s) in RCA: 219] [Impact Index Per Article: 43.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/15/2019] [Revised: 10/31/2019] [Accepted: 11/18/2019] [Indexed: 02/06/2023]
Abstract
BACKGROUND & AIMS We aimed to develop and validate a deep-learning computer-aided detection (CAD) system, suitable for use in real time in clinical practice, to improve endoscopic detection of early neoplasia in patients with Barrett's esophagus (BE). METHODS We developed a hybrid ResNet-UNet model CAD system using 5 independent endoscopy data sets. We performed pretraining using 494,364 labeled endoscopic images collected from all intestinal segments. Then, we used 1704 unique esophageal high-resolution images of rigorously confirmed early-stage neoplasia in BE and nondysplastic BE, derived from 669 patients. System performance was assessed by using data sets 4 and 5. Data set 5 was also scored by 53 general endoscopists with a wide range of experience from 4 countries to benchmark CAD system performance. Coupled with histopathology findings, scoring of images that contained early-stage neoplasia in data sets 2-5 were delineated in detail for neoplasm position and extent by multiple experts whose evaluations served as the ground truth for segmentation. RESULTS The CAD system classified images as containing neoplasms or nondysplastic BE with 89% accuracy, 90% sensitivity, and 88% specificity (data set 4, 80 patients and images). In data set 5 (80 patients and images) values for the CAD system vs those of the general endoscopists were 88% vs 73% accuracy, 93% vs 72% sensitivity, and 83% vs 74% specificity. The CAD system achieved higher accuracy than any of the individual 53 nonexpert endoscopists, with comparable delineation performance. CAD delineations of the area of neoplasm overlapped with those from the BE experts in all detected neoplasia in data sets 4 and 5. The CAD system identified the optimal site for biopsy of detected neoplasia in 97% and 92% of cases (data sets 4 and 5, respectively). CONCLUSIONS We developed, validated, and benchmarked a deep-learning computer-aided system for primary detection of neoplasia in patients with BE. The system detected neoplasia with high accuracy and near-perfect delineation performance. The Netherlands National Trials Registry, Number: NTR7072.
Collapse
Affiliation(s)
- Albert J de Groof
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
| | - Maarten R Struyvenberg
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
| | - Joost van der Putten
- Department of Electrical Engineering, Video Coding & Architectures group, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Fons van der Sommen
- Department of Electrical Engineering, Video Coding & Architectures group, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Kiki N Fockens
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
| | - Wouter L Curvers
- Department of Gastroenterology and Hepatology, Catharina Hospital Eindhoven, Eindhoven, The Netherlands
| | - Sveta Zinger
- Department of Electrical Engineering, Video Coding & Architectures group, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Roos E Pouw
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
| | - Emmanuel Coron
- Institut des Maladies de l'Appareil Digestif, University Hospital of Nantes place Alexis Ricordeau, Nantes, France
| | - Francisco Baldaque-Silva
- Department of Digestive Diseases, Karolinska University Hospital and Karolinska Institute, Stockholm, Sweden
| | - Oliver Pech
- Gastroenterology and Interventional Endoscopy, Krankenhaus Barmherzige Brüder, Regensburg, Germany
| | - Bas Weusten
- Department of Gastroenterology and Hepatology, St. Antonius Hospital, Nieuwegein, The Netherlands
| | | | - Horst Neuhaus
- Internal Medicine, Evangelisches Krankenhaus Düsseldorf, Düsseldorf, Germany
| | - Raf Bisschops
- Department of Gastroenterology and Hepatology, University Hospitals Leuven, Leuven, Belgium
| | - John Dent
- Department of Medicine, University of Adelaide and Royal Adelaide Hospital, Adelaide, South Australia
| | - Erik J Schoon
- Department of Gastroenterology and Hepatology, Catharina Hospital Eindhoven, Eindhoven, The Netherlands
| | - Peter H de With
- Department of Electrical Engineering, Video Coding & Architectures group, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Jacques J Bergman
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands.
| |
Collapse
|
28
|
Adadi A, Adadi S, Berrada M. Gastroenterology Meets Machine Learning: Status Quo and Quo Vadis. Adv Bioinformatics 2019; 2019:1870975. [PMID: 31065266 PMCID: PMC6466966 DOI: 10.1155/2019/1870975] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2018] [Accepted: 02/24/2019] [Indexed: 12/16/2022] Open
Abstract
Machine learning has undergone a transition phase from being a pure statistical tool to being one of the main drivers of modern medicine. In gastroenterology, this technology is motivating a growing number of studies that rely on these innovative methods to deal with critical issues related to this practice. Hence, in the light of the burgeoning research on the use of machine learning in gastroenterology, a systematic review of the literature is timely. In this work, we present the results gleaned through a systematic review of prominent gastroenterology literature using machine learning techniques. Based on the analysis of 88 journal articles, we delimit the scope of application, we discuss current limitations including bias, lack of transparency, accountability, and data availability, and we put forward future avenues.
Collapse
Affiliation(s)
- Amina Adadi
- Computer and Interdisciplinary Physics Laboratory, Sidi Mohamed Ben Abdellah University, Fez 30050, Morocco
| | - Safae Adadi
- Service of Hepatology and Gastroenterology, Hassan II University Hospital of Fez, Sidi Mohamed Ben Abdellah University, Fez, Morocco
| | - Mohammed Berrada
- Computer and Interdisciplinary Physics Laboratory, Sidi Mohamed Ben Abdellah University, Fez 30050, Morocco
| |
Collapse
|
29
|
Sanghi V, Thota PN. Barrett's esophagus: novel strategies for screening and surveillance. Ther Adv Chronic Dis 2019; 10:2040622319837851. [PMID: 30937155 PMCID: PMC6435879 DOI: 10.1177/2040622319837851] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2018] [Accepted: 02/19/2019] [Indexed: 12/14/2022] Open
Abstract
Barrett’s esophagus is the precursor lesion for esophageal adenocarcinoma. Screening and surveillance of Barrett’s esophagus are undertaken with the goal of earlier detection and lowering the mortality from esophageal adenocarcinoma. The widely used technique is standard esophagogastroduodenoscopy with biopsies per the Seattle protocol for screening and surveillance of Barrett’s esophagus. Surveillance intervals vary depending on the degree of dysplasia with endoscopic eradication therapy confined to patients with Barrett’s esophagus and confirmed dysplasia. In this review, we present various novel techniques for screening of Barrett’s esophagus such as unsedated transnasal endoscopy, cytosponge with trefoil factor-3, balloon cytology, esophageal capsule endoscopy, liquid biopsy, electronic nose, and oral microbiome. In addition, advanced imaging techniques such as narrow band imaging, dye-based chromoendoscopy, confocal laser endomicroscopy, volumetric laser endomicroscopy, and wide-area transepithelial sampling with computer-assisted three-dimensional analysis developed for better detection of dysplasia are also reviewed.
Collapse
Affiliation(s)
- Vedha Sanghi
- Department of Internal Medicine, Cleveland Clinic, Cleveland, OH, USA
| | - Prashanthi N Thota
- Esophageal Center, Department of Gastroenterology and Hepatology, Cleveland Clinic, 9500 Euclid Avenue, Cleveland, OH 44195, USA
| |
Collapse
|
30
|
Ghatwary N, Ahmed A, Grisan E, Jalab H, Bidaut L, Ye X. In-vivo Barrett's esophagus digital pathology stage classification through feature enhancement of confocal laser endomicroscopy. J Med Imaging (Bellingham) 2019; 6:014502. [PMID: 30840732 DOI: 10.1117/1.jmi.6.1.014502] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2018] [Accepted: 02/05/2019] [Indexed: 12/15/2022] Open
Abstract
Barrett's esophagus (BE) is a premalignant condition that has an increased risk to turn into esophageal adenocarcinoma. Classification and staging of the different changes (BE in particular) in the esophageal mucosa are challenging since they have a very similar appearance. Confocal laser endomicroscopy (CLE) is one of the newest endoscopy tools that is commonly used to identify the pathology type of the suspected area of the esophageal mucosa. However, it requires a well-trained physician to classify the image obtained from CLE. An automatic stage classification of esophageal mucosa is presented. The proposed model enhances the internal features of CLE images using an image filter that combines fractional integration with differentiation. Various features are then extracted on a multiscale level, to classify the mucosal tissue into one of its four types: normal squamous (NS), gastric metaplasia (GM), intestinal metaplasia (IM or BE), and neoplasia. These sets of features are used to train two conventional classifiers: support vector machine (SVM) and random forest. The proposed method was evaluated on a dataset of 96 patients with 557 images of different histopathology types. The SVM classifier achieved the best performance with 96.05% accuracy based on a leave-one-patient-out cross-validation. Additionally, the dataset was divided into 60% training and 40% testing; the model achieved an accuracy of 93.72% for the testing data using the SVM. The presented model showed superior performance when compared with four state-of-the-art methods. Accurate classification is essential for the intestinal metaplasia grade, which most likely develops into esophageal cancer. Not only does our method come to the aid of physicians for more accurate diagnosis by acting as a second opinion, but it also acts as a training method for junior physicians who need practice in using CLE. Consequently, this work contributes to an automatic classification that facilitates early intervention and decreases samples of required biopsy.
Collapse
Affiliation(s)
- Noha Ghatwary
- University of Lincoln, Computer Science Department, Brayford Pool, Lincoln, United Kingdom.,Arab Academy for Science and Technology, Computer Engineering Department, Alexandria, Egypt
| | - Amr Ahmed
- University of Nottingham, Computer Science Department, Semenyih, Selangor, Malaysia
| | - Enrico Grisan
- University of Padova, Department of Information Engineering, Padova, Italy
| | - Hamid Jalab
- University of Malaya, Department of Computer System and Technology, Kuala Lumpur, Malaysia
| | - Luc Bidaut
- University of Lincoln, Computer Science Department, Brayford Pool, Lincoln, United Kingdom
| | - Xujiong Ye
- University of Lincoln, Computer Science Department, Brayford Pool, Lincoln, United Kingdom
| |
Collapse
|
31
|
Passos LA, de Souza Jr. LA, Mendel R, Ebigbo A, Probst A, Messmann H, Palm C, Papa JP. Barrett’s esophagus analysis using infinity Restricted Boltzmann Machines. JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION 2019; 59:475-485. [DOI: 10.1016/j.jvcir.2019.01.043] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/30/2023]
|
32
|
Early esophageal adenocarcinoma detection using deep learning methods. Int J Comput Assist Radiol Surg 2019; 14:611-621. [PMID: 30666547 PMCID: PMC6420905 DOI: 10.1007/s11548-019-01914-4] [Citation(s) in RCA: 65] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2018] [Accepted: 01/07/2019] [Indexed: 02/08/2023]
Abstract
Purpose This study aims to adapt and evaluate the performance of different state-of-the-art deep learning object detection methods to automatically identify esophageal adenocarcinoma (EAC) regions from high-definition white light endoscopy (HD-WLE) images. Method Several state-of-the-art object detection methods using Convolutional Neural Networks (CNNs) were adapted to automatically detect abnormal regions in the esophagus HD-WLE images, utilizing VGG’16 as the backbone architecture for feature extraction. Those methods are Regional-based Convolutional Neural Network (R-CNN), Fast R-CNN, Faster R-CNN and Single-Shot Multibox Detector (SSD). For the evaluation of the different methods, 100 images from 39 patients that have been manually annotated by five experienced clinicians as ground truth have been tested. Results Experimental results illustrate that the SSD and Faster R-CNN networks show promising results, and the SSD outperforms other methods achieving a sensitivity of 0.96, specificity of 0.92 and F-measure of 0.94. Additionally, the Average Recall Rate of the Faster R-CNN in locating the EAC region accurately is 0.83. Conclusion In this paper, recent deep learning object detection methods are adapted to detect esophageal abnormalities automatically. The evaluation of the methods proved its ability to locate abnormal regions in the esophagus from endoscopic images. The automatic detection is a crucial step that may help early detection and treatment of EAC and also can improve automatic tumor segmentation to monitor its growth and treatment outcome.
Collapse
|
33
|
de Souza LA, Afonso LCS, Ebigbo A, Probst A, Messmann H, Mendel R, Hook C, Palm C, Papa JP. Learning visual representations with optimum-path forest and its applications to Barrett’s esophagus and adenocarcinoma diagnosis. Neural Comput Appl 2019. [DOI: 10.1007/s00521-018-03982-0] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|