Understanding deep learning in capsule endoscopy: Can artificial intelligence enhance clinical practice?

Table 1 Deep learning applications in wireless capsule endoscopy for classifying gastrointestinal disorders

Ref.	Class/outcome variable	Deep network architecture	Device/image resolution	Training and internal validation dataset	Testing/external validation dataset	Accuracy (%)/AUC	Sensitivity (%)/specificity (%)
Majid et al[19], 2020, NA	Multiple lesions (bleeding, esophagitis, ulcer, polyp)	CNN with classical features fusion and selection	NA/224 × 224 pixels	70% of 12889 images from multiple databases	30% of 12889 images from multiple databases	96.5/NA	96.5/NA
Ding et al[20], 2019, China	Multiple SB lesions1	CNN (ResNet 152)	SB-CE by Ankon Technologies/480 × 480 pixels	158235 images from 1970 patients	113268334 images from 5000 patients	NA/NA	99.88/100 (per patient); 99.90/100 (per lesion)
Iakovidis et al[21], 2018, NA	Multiple SB lesions2	CNN and iterative cluster unification	(1) NA/489 × 409 pixels; and (2) MiroCam CE/320 × 320 pixels	(1) 465 images from 1063 volunteers; and (2) 852 images	(1) 233 images from 1063 volunteers; and (2) 344 images	(1) 89.9/0.963; and (2) 77.5/0.814	(1) 90.7/88.2; and (2) 36.2/91.3
Aoki et al[22], 2020, Japan	Bleeding (blood content)	CNN (ResNet50)	Pillcam SB2 or SB3 CE / 224 × 224 pixels	27847 images from 41 patients	10208 images from 25 patients	99.89/0.9998	96.63/99.96
Tsuboi et al[23], 2019, Japan	Bleeding (SB angioectasia)	CNN (SSD)	Pillcam SB2 or SB3 CE/300 × 300 pixels	2237 images from 141 patients	10488 images from 28 patients	NA/0.998	98.8/98.4
Leenhardt et al[24], 2019, France	Bleeding (SB angioectasia)	CNN-based semantic segmentation	Pillcam SB3 CE / NA	600 images	600 images	NA/NA	96/100
Li et al[25], 2017, China	Bleeding (intestinal hemorrhage)	CNNs: (1) LeNet; (2) AlexNet; (3) GoogLeNet; and (4) VGG-Net	NA/NA	9672 images	2418 images	NA/NA	(1) 99.91/96.2; (2) 99.96/98.72; (3) 100/98.73; and (4) 99.96/98.72
Jia et al[26], 2017, Hong Kong, China	Bleeding (both active and inactive)	CNN	NA/240 × 240 pixels	1000 images	500 images	NA/NA	91.0/NA
Jia et al[27], 2016, Hong Kong, China	Bleeding (both active and inactive)	CNN (Inspired by AlexNet)	NA/240 × 240 pixels	8200 images	1800 images	NA/NA	99.2/NA
Aoki et al[28], 2019, Japan	Ulcer (erosion or ulceration)	CNN (SSD)	Pillcam SB2 or SB3 CE/300 × 300 pixels	5360 images from 115 patients	10440 images from 65 patients	90.8/0.958	88.2/90.9
Wang et al[29], 2019, China	Ulcer	CNN (RetinaNet)	Magnetic-guided CE by Ankon Technologies/480 × 480 pixels	37278 images from 1204 patient cases	9924 images from 300 patient cases	90.10/0.9469	89.71/90.48
Wang et al[30], 2019, China	Ulcer	CNN (based on ResNet 34)	Magnetic-guided CE by Ankon Technologies/480 × 480 pixels	80% of dataset from 1416 patients	20% of dataset from 1416 patients	92.05/0.9726	91.64/92.42
Alaskar et al[31], 2019, NA	Ulcer	CNN: (1) GoogLeNet; and (2) AlexNet	NA /(1) 224 × 224 pixels; and (2) 227 × 227 pixels	336 images	105 images	(1) 100/1; and (2) 100/1	(1) 100/100; and (2) 100/100
Fan et al[32], 2018, China	(1) Ulcer; and (2) Erosion	CNN (AlexNet)	NA/511 × 511 pixels	(1) 5500 images; and (2) 7410 images	(1) 2750 images; and (2) 5500 images	(1) 95.16/0.9891; and (2) 95.34/0.9863	(1) 96.80/94.79; and (2) 93.67/95.98
Zhou et al[6], 2017, USA	Celiac disease	CNN (GoogLeNet)	Pillcam SB2 CE/512 × 512 pixels	8800 images from 11 patients	8000 images from 10 patients	NA/NA	100/100
Klang et al[33], 2020, Israel	Crohn’s disease	CNN (Xception)	Pillcam SB2 CE/299 × 299 pixels	Experiment 1: 80% of 17640 images from 49 patients; Experiment 2: Images from 48 patients	Experiment 1: 20% of 17,640 images from 49 patients; Experiment 2: Images from 1 individual patient	Experiment 1: 95.4-96.7/0.989-0.994; Experiment 2: 73.7–98.2/0.940-0.999	Experiment 1: 92.5-97.1/96.0-98.1; Experiment 2: 69.5-100/56.8-100
Saito et al[34], 2020, Japan	Polyp (protruding lesion)	CNN (SSD)	Pillcam SB2 or SB3 CE/300 × 300 pixels	30584 images from 292 patients	17507 images from 93 patients	84.5/0.911	90.7/79.8
Yuan et al[7], 2017, Hong Kong, China	Polyp	Deep neural network	Pillcam SB CE/64 × 64 pixels	Unknown proportion of 4000 images from 35 patients	Unknown proportion of 4000 images from 35 patients	98/NA	98/99
He et al[35], 2018, Israel	Hookworm	CNN	Pillcam SB CE/227 × 227 pixels	10 out of 11 patients (436796 images from 11 patients)	1 individual patient (11-fold cross-validation)	88.5/NA	84.6/88.6

¹Abnormal classes include (1) inflammation; (2) ulcer; (3) polyps; (4) lymphangiectasia; (5) bleeding; (6) vascular disease; (7) protruding lesion; (8) lymphatic follicular hyperplasia; (9) diverticulum; and (10) parasite.

²Various lesions include gastritis, cancer, bleeding, ulcer, vascular anomalies, polypoid anomalies, and inflammation anomalies. AUC: Area under the receiver operating characteristic curve; CE: Capsule endoscopy; CNN: Convolutional neural networks; NA: Not available; SB: Small bowel; SSD: SingleShot Multi Box Detector.

Table 2 Deep learning applications in wireless capsule endoscopy for classifying non-disease objects

Ref.	Class/outcome variable	Deep network architecture	Device/image resolution	Training and internal validation dataset	Testing/external validation dataset	Accuracy (%)/AUC	Sensitivity (%)/specificity (%)
Seguí et al[38], 2016, Spain	Scenes (turbid, bubbles, clear blob, wrinkles, wall)	CNN	Pillcam SB2 CE/100 × 100 pixels	100000 images from 50 videos	20000 images from 50 videos	96/NA	NA/NA
Zou et al[5], 2015, NA	Organ locations (stomach, small intestine, and colon)	CNN (AlexNet)	NA/480 × 480 pixels	60000 images	15000 images	95.52/NA	NA/NA

AUC: Area under the receiver operating characteristic curve; CE: Capsule endoscopy; CNN: Convolutional neural networks; NA: Not available; SB: Small bowel.

Table 3 Deep learning applications in wireless capsule endoscopy for improving the reading efficiency of wireless capsule endoscopy

Ref.	Experiment type	Scope of WCE reading /device	Conventional reading	Deep learning assisted reading	P value
Aoki et al[39], 2019, Japan	Retrospective study using anonymized data	SB section only/Pillcam SB3	mean reading time (min): Trainee: 20.7; Expert: 12.2	mean reading time (min): Trainee: 5.2; Expert: 3.1	< 0.001
			Overall lesion detection rate: Trainee: 47%; Expert: 84%	Overall lesion detection rate: Trainee: 55%; Expert: 87%	NS
Ding et al[20], 2019, China	Retrospective study by randomly selected videos	Small bowel abnormalities/SB-CE by Ankon Technologies	mean reading time ± standard deviation (min): 96.6 ± 22.53	mean reading time ± standard deviation (min): 5.9 ± 2.23	< 0.001
			Overall lesion detection rate: 41.43%	Overall lesion detection rate: 47.00%	NA1

¹In per-patient analysis, deep learning assisted physician significantly outperformed conventional reading in detecting lymphangiectasia, lymphatic follicular, hyperplasia, inflammation, protruding lesion, and polyps. CE: Capsule endoscopy; NA: Not available; NS: Not significant; SB: Small bowel.