Real-time continuous image guidance for endoscopic retrograde cholangiopancreatography based on 3D/2D registration and respiratory compensation

Abstract

BACKGROUND

It has been confirmed that three-dimensional (3D) imaging allows easier identification of bile duct anatomy and intraoperative guidance of endoscopic retrograde cholangiopancreatography (ERCP), which reduces the radiation dose and procedure time with improved safety. However, current 3D biliary imaging does not have good real-time fusion with intraoperative imaging, a process meant to overcome the influence of intraoperative respiratory motion and guide navigation. The present study explored the feasibility of real-time continuous image-guided ERCP.

AIM

To explore the feasibility of real-time continuous image-guided ERCP.

METHODS

We selected 2 3D-printed abdominal biliary tract models with different structures to simulate different patients. The ERCP environment was simulated for the biliary phantom experiment to create a navigation system, which was further tested in patients. In addition, based on the estimation of the patient’s respiratory motion, preoperative 3D biliary imaging from computed tomography of 18 patients with cholelithiasis was registered and fused in real-time with 2D fluoroscopic sequence generated by the C-arm unit during ERCP.

RESULTS

Continuous image-guided ERCP was applied in the biliary phantom with a registration error of 0.46 mm ± 0.13 mm and a tracking error of 0.64 mm ± 0.24 mm. After estimating the respiratory motion, 3D/2D registration accurately transformed preoperative 3D biliary images to each image in the X-ray image sequence in real-time in 18 patients, with an average fusion rate of 88%.

CONCLUSION

Continuous image-guided ERCP may be an effective approach to assist the operator and reduce the use of X-ray and contrast agents.

Key Words: Endoscopic retrograde cholangiopancreatography; Three-dimensional images; Registration; Cholelithiasis; Hilar cholangiocarcinoma

Core Tip: Three-dimensional imaging allows easier identification of bile duct anatomy and intraoperative guidance of endoscopic retrograde cholangiopancreatography (ERCP). Continuous image-guided ERCP may be an effective means to assist the operator and reduce the use of X-ray and contrast agents.

INTRODUCTION

Endoscopic retrograde cholangiopancreatography (ERCP) was first described by McCune et al[1] at George Washington University in 1968. Superselection of difficult bile ducts contributes to a higher incidence of adverse events, as well as increased radiation exposure for patients and endoscopists[2,3]. Recent studies have shown that even low doses of radiation may induce carcinogenic effects[4]. The radiation dose is one of the recommended quality indicators for ERCP[5,6]. Furthermore, injection of a contrast medium is considered a risk factor for post-ERCP pancreatitis and cholangitis[7]. Therefore, developing new technologies to enhance the efficacy and safety of ERCP is urgent.

Preoperative three-dimensional (3D) imaging is widely used for planning in interventional radiosurgery. Three-dimensional computed tomography (3D-CT) has been utilized in transcatheter arterial chemoembolization (TACE), and allows for better visualization of small vessels[8,9]. Transjugular intrahepatic portosystemic shunt (TIPS) combines preoperative 3D-CT images with images taken at the time of intervention, and has demonstrated a significant reduction in operation times and radiation doses[10]. Three-dimensional imaging is also beneficial in guiding therapeutic ERCP[11], as it allows easier identification of bile duct anatomy and guides intraoperative navigation, reducing necessary radiation dose and operation time with improved safety[11]. For example, biliary stent placement via ERCP in the setting of indeterminate biliary stricture, the X-ray exposure time was reduced from 8.1 min to 35.1 min. However, current 3D biliary imaging does not have good real-time fusion with intraoperative images.

Here, we explored the feasibility of real-time continuous image-guided ERCP, which overcomes the influence of intraoperative respiratory motion and provides an accurate navigation method. Preoperative 3D-CT bile duct sequences were registered and fused with real-time 2D images from the C-arm taken during ERCP; this strategy may improve the quality of ERCP.

MATERIALS AND METHODS

Study population

This was a single-centered prospective observational study of patients receiving ERCP at a tertiary care hospital between July 2021 and March 2022. Twenty patients with cholelithiasis undergoing therapeutic ERCP were recruited. Only patients with a preoperative 3D-CT sequence of the bile duct were eligible. Dynamic 2D images of the bile duct generated by the fluoroscopic C-arm were obtained. Exclusion criteria were: age < 18 y, poor general health, inability to tolerate basic or intravenous anesthesia, uncontrollable coagulation disorder, anticoagulation therapy that could not be discontinued, altered anatomy (e.g., Billroth II gastrectomy or Roux-en-Y anastomosis), pregnancy, and intraoperative placement of a biliopancreatic stent. Written informed consent was obtained from all patients. The study was approved by the Ethics Committee of the Chinese People’s Liberation Army (PLA) General Hospital (No. S2021-415-01) and was conducted per the Declaration of Helsinki.

Human biliary phantom

To verify the accuracy of our 3D/2D registration algorithm, we created 2 3D-printed biliary tract models with different structures to simulate different patients. Enhanced CT scan data of 2 patients with biliary dilatation were inputted into a 3D biomechanics research software tool (Visual 3D, Beijing, China). The operator manually segmented the contours of the biliary tract to acquire surface models. The acquired surface models were inputted into a Stratasys J750 3D printer (Markforged, Watertown, Massachusetts) and printed using resin (Figure 1). The printed phantom was life-sized and hollow, allowing sphincterotomes to enter. The phantom experiment was carried out in the PLA General Hospital operating room (Figure 2). Unlike in clinical experiments, model experiments could be repeated many times to simulate real intraoperative conditions to adjust the algorithm parameters.

Figure 1 Biliary phantom setup. A: Biliary model based on patient A; B: Biliary model based on patient B.

Figure 2 Surgical environment for endoscopic retrograde cholangiopancreatography phantom. A: Respiratory motion simulator; B: X-ray fluoroscopy guidance in navigation.

CT and fluoroscopic imaging

An iopromide contrast medium (Ultravist 370; Bayer Healthcare, Leverkusen, Germany) was diluted to visualize the biliary tract. All CT scans were performed in the Imaging Department of PLA General Hospital (uCT 510; United Imaging Healthcare, Shanghai, China) with a slice thickness of 1.25 mm and a tube voltage of 120 kV. Patients were asked to remain in the end-inspiratory state for CT scan. To acquire the best alignment transformation, patients were asked to maintain the same pre-operative state at the beginning of surgery. Then, fluoroscopy was performed and the image was defined as the best matching frame. On the day of the ERCP procedure, the acquired CT scan data were imported into Mimics (Materialise, Belgium) to reconstruct the 3D volume-render models of the biliary tract, liver, gallbladder, and bone. All patients received intraoperative deep sedation. ERCP was performed using a standard endoscope (Olympus Duodenoscope TJF260). When the guidewire was successfully superselected in the bile duct and the contrast agent was injected into the duct, the C-arm system performed 1 consecutive respiratory cycle of fluoroscopy around the patient, producing approximately 20 images which were transmitted in real time to our image navigation system.

CT co-registration and fusion with the fluoroscopic image

During the registration between CT and X-ray fluoroscopic images, the CT image was aligned with X-ray fluoroscopy through common features. For the biliary phantom, the registration fiducials were extracted from the CT and X-ray images, respectively, to get rigid transformation, by which the 3D biliary model segmented from CT was projected onto the X-ray fluoroscopic image to finish the overlap of biliary roadmapping. The biliary ducts were observed with X-ray images without contrast agents during model experimentation (Figure 3). After the 3D/2D registration was completed, the biliary ducts segmented from CT were projected onto the X-ray image, and the validity of the registration was judged according to the edge blending effect.

Figure 3 Phantom study of registration and fusion. A and E: Biliary phantom segmentation using computed tomography images; B and F: Biliary phantom segmentation using X-ray fluoroscopy; C and G: Biliary lumen centerline extraction; D and H: Three-dimensional biliary model registered with the best matching frame using X-ray fluoroscopy.

The bone, liver, gallbladder, and biliary tract were segmented by a 3D-UNet model based on pre-operative CT data (Figure 4). Afterward, the Medical Imaging Interaction Toolkit was used to extract 2D bone structures. Mimics software (Materialise, Leuven, Belgium) was used to extract 3D and 2D biliary centerlines for analysis. Finally, 3D/2D registration based on bone structures was performed to achieve effective fusion of the 3D biliary model and the best matching frame (Figure 3).

Figure 4 Automatic segmentation of structures based on preoperative computed tomography. A and B: Biliary and skin phantoms of different patients; C: Bone, liver, biliary tract, and gallbladder segmented by a three-dimensional UNet network based on preoperative computed tomography data.

The respiratory motion of the patient during the operation affects fusion accuracy. Some researchers have estimated the patient’s respiratory state by extracting diaphragm data in real time[12-15]. The diaphragm-based method not only requires manual annotation, but also cannot guarantee real-time performance, thus prolonging the operation time and increasing X-ray radiation exposure. The method based on X-ray image centroid extraction was used in the current study to estimate the respiratory state[9] and monitor the motion change between frames through X-ray image gray distribution. The respiratory motion curve was acquired and used to complete continuous compensation for the fluoroscopy sequence, and real-time biliary roadmapping projected onto X-ray fluoroscopy was completed. This 3D/2D image registration technology is available in the current intraoperative imaging system takes approximately 60 s. Our method has a fast calculation speed and is convenient to operate, features that aid in real-time navigation.

Data collection

Quantitative data included the registration error and the tracking error (TE). The registration error was calculated as the mean square error (MSE) between the projection of the 3D and 2D centerline. After finishing respiratory compensation and projecting, the mean MSE was set as the TE. Qualitative parameters included image quality and real-time fusion rates for both preoperative and intraoperative images.

Definitions

Image-guided ERCP: Image quality was defined as “good” if the 3D-CT and fluoroscopic images of the bile duct filled by a contrast medium were visible. The outcome of real-time fusion rates was assessed by the degree to which the bile duct in the fluoroscopic mode matched perfectly with the 3D-CT biliary imaging.

Statistical analysis

Data were assessed for normality, and appropriate descriptive statistics were calculated and expressed as mean ± standard deviation (SD) and frequency (percentages). Data were analyzed using SPSS version 22 (IBM Corp., Armonk, NY, United States).

RESULTS

Image-guided ERCP had a mean registration error of 0.46 mm ± 0.13 mm and a TE of 0.64 mm ± 0.24 mm. For patients, the biliary model was projected onto the X-ray based on 3D/2D registration (Figure 5), and the respiratory motion curve was estimated through a continuous X-ray fluoroscopy sequence (Figure 6). Continuous biliary roadmapping guidance was acquired (Figure 7). Our navigation system was performed in the operating room (Figure 8), and it provided image-based guidance in multiple patients (Figure 9). Image quality was good in all 18 cases and CT to X-ray fluoroscopic image registration and fusion were technically feasible, with an average fusion rate of 88% (Supplementary material).

Figure 5 Registration and fusion on the best matching frame of the patient. A: Three- dimensional/two-dimensional registration based on bone during endoscopic retrograde cholangiopancreatography; B: Overlay the biliary model on X-ray fluoroscopy through the transformation from bone registration.

Figure 6 Example of the estimated respiratory state over an 18.8 s X-ray fluoroscopic image sequence.

Figure 7 Real-time navigation of image-guided endoscopic retrograde cholangiopancreatography. A: Beginning of inhalation; B: During inhalation; C: End of inhalation; D: During exhalation; E: End of exhalation.

Figure 8 Image-guided endoscopic retrograde cholangiopancreatography navigation system.

Figure 9 Additional examples of real-time image guidance navigation. A-D: Beginning of the inhalation; E-H: During the inhalation; I-L: End of the inhalation.

DISCUSSION

In the current study, continuous image guidance for ERCP based on 3D/2D registration was evaluated, and we found that this strategy may be an effective means of real-time localization of the bile duct.

The prognosis of hilar cholangiocarcinoma (HCCA) is extremely poor, and surgery is the only radical treatment modality. However, most patients present with an unresectable tumor or cholangitis, high bilirubin, and insufficient volume of the residual liver; as such, biliary drainage by ERCP is often best available treatment. Previous studies have shown the benefit of 3D imaging in guiding difficult ERCP procedures[11]. Preoperative 3D-CT, magnetic resonance cholangiopancreatography, magnetic resonance virtual endoscopy, or intraoperative 3D-CT of the cone beam have all provided adequate guidance for biliary stenting in patients with HCCA, but do not achieve real-time fusion of 3D images with 2D fluoroscopic images or display guidewire or sphincterotomes in three dimensions[16-19]. Our technology can obtain 3D models of the biliary duct, tumor, and hilar vessels through preoperative magnetic resonance imaging/CT segmentation. The registration between preoperative 3D models and intraoperative 2D X-ray images was completed based on the centerlines of the biliary ducts, and 3D models were projected onto X-ray images for display. Meanwhile, based on the respiratory compensation method, a continuous fusion between 3D models and the X-ray image sequences was achieved, and real-time guidance was realized through graphics processing unit acceleration. We believe that our method may provide physicians with more visual information during surgery and effectively improve the operability of the intervention.

The present study reports real-time fusion of preoperative 3D-CT images with the 2D fluoroscopic images during ERCP. The radiation dose required for the registration between 3D-CT and fluoroscopic image can be considered negligible compared with the total radiation dose for difficult ERCP procedures. In addition, image registration was completed in an average time of 1 min, with a minimal overall increase in total operation time. Therefore, with the real-time visual aid during ERCP, it is likely that less radiation, contrast medium, and operative time may be required during the navigation.

Furthermore, our methods estimated the respiratory state better than the classical diaphragm extraction method[17]. Since the diaphragm is incomplete or invisible during ERCP in some patients, the applicability and robustness of our method are better based on image gray distribution. Furthermore, this method can achieve a more accurate estimation of the respiratory state of the biliary tract, thus ensuring better fusion of 3D-CT and 2D X-ray images. The main reason for this is that the respiratory motion of the diaphragm can reflect well upon the upper edge of the liver, but cannot directly represent the motion of the biliary tract. Experimental results also proved that our method is more accurate for biliary motion estimation of patients under free breathing. Compared with the TE of the diaphragm-based method, based on our cholangiography image fusion results, TE was relatively low (1.68 mm ± 0.34 mm vs 0.64 mm ± 0.24 mm), providing more accurate guidance. As shown in Figure 10, the respiratory state estimation based on our method was smoother and closer to the true breathing curve, which ensures the stability of image-guided navigation. Moreover, our method is simpler and faster to compute and process intraoperative X-ray fluoroscopic image sequences in real-time. Conversely, the diaphragm-based method requires real-time segmentation of the diaphragm whether manual or network methods are used, which represents a complex task and ultimately leads to an increase in operation time and radiation dose.

Figure 10  Comparison between two methods with respect to respiratory state estimation. A: Centroid of the X-ray image; B: Diaphragm of the X-ray image; C: Example of the estimated respiratory state over a 4.4 s X-ray image sequence.

There are further implications of our technology in the context of other minimally invasive procedures such as TACE, TIPS, and spine surgery[8-10,20]. Coronavirus disease-2019 (COVID-19) had devastating impacts on healthcare system operations. The pandemic disrupted normal workflow of spine surgeries, restricting and postponing elective procedures[20]. This disruption may have resulted in the prolonged impairment of patients who were forced to postpone their procedures. Minimally invasive surgery and perioperative telemedicine are inevitable trends in spine surgery after COVID-19. Spine surgery is usually completed under the guidance of X-ray images, and physicians need to conceive the 3D structure of the patient’s spine based on clinical experience. The spine is different from the soft liver and there is no respiratory interference; however, the similarity of the structure between the joints and the overall length makes surgery more difficult to perform only under the guidance of X-ray images. Meanwhile, the increase in operation time has also caused longer waiting periods for treatment. Our method can extract the 3D spine structure from preoperative CT and complete 3D/2D registration through the bone structure line, achieving the fusion of 3D information with intraoperative X-ray images. This technology could help physicians better localize lesions and enhance visual perception of depth. Fusion of 3D/2D imaging could also help physicians, especially those with insufficient clinical experience, to observe the patient’s spine structure more intuitively and improve surgical efficiency. In summary, the number of spinal surgeries increased following COVID-19 and has continued to rise; our technology may help the hospital to perform more operations and allow patients to receive timely and effective treatment.

COVID-19 has also been associated with an increased risk for ischemic and hemorrhagic strokes[21], conditions often treated with a vascular intervention procedure very similar to TACE that is achieved by embolizing the vessels connected to the aneurysm. Unlike liver vessels, intracranial vessels are not affected by respiratory motion and remain static with the use of X-ray images for guidance. However, intracranial vessels are generally thin and have multiple branching structures, and thus the risks associated with intracranial surgery are high. Surgeons require more clinical experience to ensure the accuracy of the surgery; otherwise, they may injure other blood vessels or tissues and cause secondary harm to the patient. Our technology can extract data regarding blood vessels connected to the lesion (e.g., aneurysm) before the surgery and guide the surgeon intraoperatively through high-precision registration and fusion. Given the risk of intracranial vascular intervention procedures, our technology could display the vessels around the catheter and render them in different colors to remind the surgeon to be cautious, thus benefiting the safety of the procedure. We believe that our technology may have potential clinical value in treating cerebrovascular diseases and could precisely treat ischemic and hemorrhagic strokes associated COVID-19.

CONCLUSION

Our image-guided ERCP strategy was fully validated in models that may be useful in cannulating specific biliary ducts in patients with complex biliopancreatic diseases such as HCCA. However, clinical studies are still needed to determine if this new approach could improve technical success while reducing radiation exposure and contrast agent administration during ERCP.

ARTICLE HIGHLIGHTS

Research background

Three-dimensional (3D) imaging is beneficial in guiding therapeutic endoscopic retrograde cholangiopancreatography (ERCP), reducing the radiation dose and procedure time while increasing overall procedure safety. However, current 3D biliary imaging does not have good real-time fusion with intraoperative images.

Research motivation

We explored the feasibility of real-time continuous image-guided ERCP, which was shown to overcome the influence of intraoperative respiratory motion and provides an accurate navigation method.

Research objectives

To explore the feasibility of real-time continuous image-guided ERCP.

Research methods

We simulated an ERCP environment using an experimental biliary phantom with the aim of designing a navigation system; this system was further tested in patients undergoing ERCP.

Research results

Continuous image-guided ERCP was applied in the biliary phantom with low registration and tracking errors. This 3D/2D registration accurately transformed preoperative 3D biliary images to each image in the X-ray sequence in real-time.

Research conclusions

Continuous image-guided ERCP may be an effective approach to assist the operator and reduce the use of X-ray and contrast agents.

Research perspectives

Clinical studies are needed to determine if this image-guided ERCP strategy could improve technical success while reducing radiation exposure and contrast agent administration during ERCP.