Real-time continuous image guidance for endoscopic retrograde cholangiopancreatography based on 3D/2D registration and respiratory compensation

Figure 1 Biliary phantom setup. A: Biliary model based on patient A; B: Biliary model based on patient B.

Figure 2 Surgical environment for endoscopic retrograde cholangiopancreatography phantom. A: Respiratory motion simulator; B: X-ray fluoroscopy guidance in navigation.

Figure 3 Phantom study of registration and fusion. A and E: Biliary phantom segmentation using computed tomography images; B and F: Biliary phantom segmentation using X-ray fluoroscopy; C and G: Biliary lumen centerline extraction; D and H: Three-dimensional biliary model registered with the best matching frame using X-ray fluoroscopy.

Figure 4 Automatic segmentation of structures based on preoperative computed tomography. A and B: Biliary and skin phantoms of different patients; C: Bone, liver, biliary tract, and gallbladder segmented by a three-dimensional UNet network based on preoperative computed tomography data.

Figure 5 Registration and fusion on the best matching frame of the patient. A: Three- dimensional/two-dimensional registration based on bone during endoscopic retrograde cholangiopancreatography; B: Overlay the biliary model on X-ray fluoroscopy through the transformation from bone registration.

Figure 6 Example of the estimated respiratory state over an 18. 8 s X-ray fluoroscopic image sequence.

Figure 7 Real-time navigation of image-guided endoscopic retrograde cholangiopancreatography. A: Beginning of inhalation; B: During inhalation; C: End of inhalation; D: During exhalation; E: End of exhalation.

Figure 8 Image-guided endoscopic retrograde cholangiopancreatography navigation system.

Figure 9 Additional examples of real-time image guidance navigation. A-D: Beginning of the inhalation; E-H: During the inhalation; I-L: End of the inhalation.

Figure 10  Comparison between two methods with respect to respiratory state estimation. A: Centroid of the X-ray image; B: Diaphragm of the X-ray image; C: Example of the estimated respiratory state over a 4.4 s X-ray image sequence.