|
1
|
Wu X, Wang D, Xiang N, Pan M, Jia F, Yang J, Fang C. Augmented reality-assisted navigation system contributes to better intraoperative and short-time outcomes of laparoscopic pancreaticoduodenectomy: a retrospective cohort study. Int J Surg 2023; 109:2598-2607. [PMID: 37338535 PMCID: PMC10498855 DOI: 10.1097/js9.0000000000000536] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 05/26/2023] [Indexed: 06/21/2023]
Abstract
BACKGROUND Augmented reality (AR)-assisted navigation system are currently good techniques for hepatectomy; however, its application and efficacy for laparoscopic pancreatoduodenectomy have not been reported. This study sought to focus on and evaluate the advantages of laparoscopic pancreatoduodenectomy guided by the AR-assisted navigation system in intraoperative and short-time outcomes. METHODS Eighty-two patients who underwent laparoscopic pancreatoduodenectomy from January 2018 to May 2022 were enrolled and divided into the AR and non-AR groups. Clinical baseline features, operation time, intraoperative blood loss, blood transfusion rate, perioperative complications, and mortality were analyzed. RESULTS AR-guided laparoscopic pancreaticoduodenectomy was performed in the AR group ( n =41), whereas laparoscopic pancreatoduodenectomy was carried out routinely in the non-AR group ( n =41). There was no significant difference in baseline data between the two groups ( P >0.05); Although the operation time of the AR group was longer than that of the non-AR group (420.15±94.38 vs. 348.98±76.15, P <0.001), the AR group had a less intraoperative blood loss (219.51±167.03 vs. 312.20±195.51, P =0.023), lower blood transfusion rate (24.4 vs. 65.9%, P <0.001), lower occurrence rates of postoperative pancreatic fistula (12.2 vs. 46.3%, P =0.002) and bile leakage (0 vs. 14.6%, P =0.026), and shorter postoperative hospital stay (11.29±2.78 vs. 20.04±11.22, P <0.001) compared with the non-AR group. CONCLUSION AR-guided laparoscopic pancreatoduodenectomy has significant advantages in identifying important vascular structures, minimizing intraoperative damage, and reducing postoperative complications, suggesting that it is a safe, feasible method with a bright future in the clinical setting.
Collapse
Affiliation(s)
- Xiwen Wu
- Department of Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Institute of Digital Intelligence, Zhujiang Hospital, Southern Medical University
- Guangdong Digital Medical Clinical Engineering and Technology Research Center
- Pazhou Lab, Guangzhou
| | - Dehui Wang
- Department of Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Institute of Digital Intelligence, Zhujiang Hospital, Southern Medical University
- Guangdong Digital Medical Clinical Engineering and Technology Research Center
- Pazhou Lab, Guangzhou
| | - Nan Xiang
- Department of Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Institute of Digital Intelligence, Zhujiang Hospital, Southern Medical University
- Guangdong Digital Medical Clinical Engineering and Technology Research Center
- Pazhou Lab, Guangzhou
| | - Mingxin Pan
- Department of Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Institute of Digital Intelligence, Zhujiang Hospital, Southern Medical University
- Guangdong Digital Medical Clinical Engineering and Technology Research Center
- Pazhou Lab, Guangzhou
| | - Fucang Jia
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Jian Yang
- Department of Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Institute of Digital Intelligence, Zhujiang Hospital, Southern Medical University
- Guangdong Digital Medical Clinical Engineering and Technology Research Center
- Pazhou Lab, Guangzhou
| | - Chihua Fang
- Department of Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Institute of Digital Intelligence, Zhujiang Hospital, Southern Medical University
- Guangdong Digital Medical Clinical Engineering and Technology Research Center
- Pazhou Lab, Guangzhou
| |
Collapse
|
|
2
|
Abstract
INTRODUCTION During an operation, augmented reality (AR) enables surgeons to enrich their vision of the operating field by means of digital imagery, particularly as regards tumors and anatomical structures. While in some specialties, this type of technology is routinely ustilized, in liver surgery due to the complexity of modeling organ deformities in real time, its applications remain limited. At present, numerous teams are attempting to find a solution applicable to current practice, the objective being to overcome difficulties of intraoperative navigation in an opaque organ. OBJECTIVE To identify, itemize and analyze series reporting AR techniques tested in liver surgery, the objectives being to establish a state of the art and to provide indications of perspectives for the future. METHODS In compliance with the PRISMA guidelines and availing ourselves of the PubMed, Embase and Cochrane databases, we identified English-language articles published between January 2020 and January 2022 corresponding to the following keywords: augmented reality, hepatic surgery, liver and hepatectomy. RESULTS Initially, 102 titles, studies and summaries were preselected. Twenty-eight corresponding to the inclusion criteria were included, reporting on 183patients operated with the help of AR by laparotomy (n=31) or laparoscopy (n=152). Several techniques of acquisition and visualization were reported. Anatomical precision was the main assessment criterion in 19 articles, with values ranging from 3mm to 14mm, followed by time of acquisition and clinical feasibility. CONCLUSION While several AR technologies are presently being developed, due to insufficient anatomical precision their clinical applications have remained limited. That much said, numerous teams are currently working toward their optimization, and it is highly likely that in the short term, the application of AR in liver surgery will have become more frequent and effective. As for its clinical impact, notably in oncology, it remains to be assessed.
Collapse
Affiliation(s)
- B Acidi
- Department of Surgery, AP-HP hôpital Paul-Brousse, Hepato-Biliary Center, 12, avenue Paul-Vaillant Couturier, 94804 Villejuif cedex, France; Augmented Operating Room Innovation Chair (BOPA), France; Inria « Mimesis », Strasbourg, France
| | - M Ghallab
- Department of Surgery, AP-HP hôpital Paul-Brousse, Hepato-Biliary Center, 12, avenue Paul-Vaillant Couturier, 94804 Villejuif cedex, France; Augmented Operating Room Innovation Chair (BOPA), France
| | - S Cotin
- Augmented Operating Room Innovation Chair (BOPA), France; Inria « Mimesis », Strasbourg, France
| | - E Vibert
- Department of Surgery, AP-HP hôpital Paul-Brousse, Hepato-Biliary Center, 12, avenue Paul-Vaillant Couturier, 94804 Villejuif cedex, France; Augmented Operating Room Innovation Chair (BOPA), France; DHU Hepatinov, 94800 Villejuif, France; Inserm, Paris-Saclay University, UMRS 1193, Pathogenesis and treatment of liver diseases; FHU Hepatinov, 94800 Villejuif, France
| | - N Golse
- Department of Surgery, AP-HP hôpital Paul-Brousse, Hepato-Biliary Center, 12, avenue Paul-Vaillant Couturier, 94804 Villejuif cedex, France; Augmented Operating Room Innovation Chair (BOPA), France; Inria « Mimesis », Strasbourg, France; DHU Hepatinov, 94800 Villejuif, France; Inserm, Paris-Saclay University, UMRS 1193, Pathogenesis and treatment of liver diseases; FHU Hepatinov, 94800 Villejuif, France.
| |
Collapse
|
|
3
|
Minimally invasive and invasive liver surgery based on augmented reality training: a review of the literature. J Robot Surg 2022; 17:753-763. [DOI: 10.1007/s11701-022-01499-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 11/14/2022] [Indexed: 11/29/2022]
|
|
4
|
Rudiman R. Minimally invasive gastrointestinal surgery: From past to the future. Ann Med Surg (Lond) 2021; 71:102922. [PMID: 34703585 PMCID: PMC8521242 DOI: 10.1016/j.amsu.2021.102922] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Revised: 10/02/2021] [Accepted: 10/03/2021] [Indexed: 12/21/2022] Open
Abstract
The improvement of the science and art of surgery began over 150 years ago. Surgical core tasks, “cutting and sewing” with hand and direct contact with the organs, have remained the same. However, in the 21st century, there has been a shifting paradigm in the methodology of surgery. The joint union between innovators, engineers, industry, and patient demands resulted in minimally invasive surgery (MIS). This method has influenced the techniques in every aspect of abdominal surgery, such as surgeons are not required to direct contact or see the structures on which they operate. Advances in the endoscope, imaging, and improved instrumentations convert the essential open surgery into the endoscopic method. Furthermore, computers and robotics show a promising future to facilitate complex procedures, enhance accuracy in microscale operations, and develop a simulation to improve the ability to face sophisticated approaches. MIS has been replacing open surgery due to improved survival, fewer complications, and rapid recoveries in recent years. Minimally invasive surgery's further research in diagnostic and therapeutic modalities is under investigation to achieve genuinely “noninvasive” surgery. Thus, MIS has gained interest in recent days and has been improving with promising outcomes.
Minimally invasive surgery has interfered with multiple aspects of the surgical approach. Advancement in the endoscope, imaging, and other instrumentations shifting the current methodological conventional surgery. The benefit over risk is the promising primary outcome to achieve an exceptional quality of life.
Collapse
Affiliation(s)
- Reno Rudiman
- Digestive Surgeon, Division of Digestive Surgery, Department of General Surgery, School of Medicine, Padjadjaran University, Hasan Sadikin General Hospital, Bandung, Indonesia
| |
Collapse
|
|
5
|
Wang Y, Cao D, Chen SL, Li YM, Zheng YW, Ohkohchi N. Current trends in three-dimensional visualization and real-time navigation as well as robot-assisted technologies in hepatobiliary surgery. World J Gastrointest Surg 2021; 13:904-922. [PMID: 34621469 PMCID: PMC8462083 DOI: 10.4240/wjgs.v13.i9.904] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/14/2021] [Revised: 04/19/2021] [Accepted: 08/02/2021] [Indexed: 02/06/2023] Open
Abstract
With the continuous development of digital medicine, minimally invasive precision and safety have become the primary development trends in hepatobiliary surgery. Due to the specificity and complexity of hepatobiliary surgery, traditional preoperative imaging techniques such as computed tomography and magnetic resonance imaging cannot meet the need for identification of fine anatomical regions. Imaging-based three-dimensional (3D) reconstruction, virtual simulation of surgery and 3D printing optimize the surgical plan through preoperative assessment, improving the controllability and safety of intraoperative operations, and in difficult-to-reach areas of the posterior and superior liver, assistive robots reproduce the surgeon’s natural movements with stable cameras, reducing natural vibrations. Electromagnetic navigation in abdominal surgery solves the problem of conventional surgery still relying on direct visual observation or preoperative image assessment. We summarize and compare these recent trends in digital medical solutions for the future development and refinement of digital medicine in hepatobiliary surgery.
Collapse
Affiliation(s)
- Yun Wang
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
| | - Di Cao
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
| | - Si-Lin Chen
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
| | - Yu-Mei Li
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
| | - Yun-Wen Zheng
- Institute of Regenerative Medicine, and Affiliated Hospital of Jiangsu University, Jiangsu University, Zhenjiang 212001, Jiangsu Province, China
- Department of Gastrointestinal and Hepato-Biliary-Pancreatic Surgery, Faculty of Medicine, University of Tsukuba, Tsukuba 305-8575, Ibaraki, Japan
- Guangdong Provincial Key Laboratory of Large Animal Models for Biomedicine, and School of Biotechnology and Heath Sciences, Wuyi University, Jiangmen 529020, Guangdong Province, China
- School of Medicine, Yokohama City University, Yokohama 234-0006, Kanagawa, Japan
| | - Nobuhiro Ohkohchi
- Department of Gastrointestinal and Hepato-Biliary-Pancreatic Surgery, Faculty of Medicine, University of Tsukuba, Tsukuba 305-8575, Ibaraki, Japan
| |
Collapse
|
|
6
|
Wahba R, Thomas MN, Bunck AC, Bruns CJ, Stippel DL. Clinical use of augmented reality, mixed reality, three-dimensional-navigation and artificial intelligence in liver surgery. Artif Intell Gastroenterol 2021; 2:94-104. [DOI: 10.35712/aig.v2.i4.94] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Revised: 07/10/2021] [Accepted: 08/27/2021] [Indexed: 02/06/2023] Open
Abstract
A precise knowledge of intra-parenchymal vascular and biliary architecture and the location of lesions in relation to the complex anatomy is indispensable to perform liver surgery. Therefore, virtual three-dimensional (3D)-reconstruction models from computed tomography/magnetic resonance imaging scans of the liver might be helpful for visualization. Augmented reality, mixed reality and 3D-navigation could transfer such 3D-image data directly into the operation theater to support the surgeon. This review examines the literature about the clinical and intraoperative use of these image guidance techniques in liver surgery and provides the reader with the opportunity to learn about these techniques. Augmented reality and mixed reality have been shown to be feasible for the use in open and minimally invasive liver surgery. 3D-navigation facilitated targeting of intraparenchymal lesions. The existing data is limited to small cohorts and description about technical details e.g., accordance between the virtual 3D-model and the real liver anatomy. Randomized controlled trials regarding clinical data or oncological outcome are not available. Up to now there is no intraoperative application of artificial intelligence in liver surgery. The usability of all these sophisticated image guidance tools has still not reached the grade of immersion which would be necessary for a widespread use in the daily surgical routine. Although there are many challenges, augmented reality, mixed reality, 3D-navigation and artificial intelligence are emerging fields in hepato-biliary surgery.
Collapse
Affiliation(s)
- Roger Wahba
- Department of General, Visceral, Cancer and Transplantation Surgery, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| | - Michael N Thomas
- Department of General, Visceral, Cancer and Transplantation Surgery, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| | - Alexander C Bunck
- Department of Diagnostic and Interventional Radiology, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| | - Christiane J Bruns
- Department of General, Visceral, Cancer and Transplantation Surgery, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| | - Dirk L Stippel
- Department of General, Visceral, Cancer and Transplantation Surgery, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| |
Collapse
|
|
7
|
Kanehira M, Okamoto T, Abe K, Yasuda J, Onda S, Futagawa Y, Ikegami T, Suzuki N, Hattori A. Development of recognised position-guided navigation system. Int J Med Robot 2021; 17:e2322. [PMID: 34405536 DOI: 10.1002/rcs.2322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Revised: 08/16/2021] [Accepted: 08/16/2021] [Indexed: 11/11/2022]
Abstract
BACKGROUND Previously, we developed an image-guided navigation system (IG-NS) incorporating augmented reality technology. Nevertheless, the system could still only aid the operator by presenting imagery and was short of achieving the goal of developing a real navigation system. Therefore, we developed a recognised position-guided navigation system (RP-NS) and herein reported the functionality and usefulness of this system in a phantom model for clinical applications. METHODS We developed RP-NS which was reconstructed by adding the positional recognition and instruction functions with the cautions by displaying the images on the monitor with a voice to the IG-NS. We evaluated accuracy of positional recognition and instruction functions using phantom model. By utilising the chronological recording of the tip position of the surgical apparatus, the surgical precision of the operators was assessed. Finally, the feasibility of improvements in surgical precision using this system was evaluated. RESULTS The RP-NS indicated an accuracy of the position recognition functions with an error of 2.7 mm. The surgeons could perform partial hepatectomies within mean value of 7.5% error as compared with calculated volume according to the instruction. Improvements in surgical precision using this system were obtained on the surgeons with different levels. CONCLUSIONS The RP-NS was highly effective as a navigation system owing to precise positional recognition and adequate instruction functions. Therefore, these results indicate that the use of this system may complement differences in proficiency, and numerically evaluate surgical skills and analyse tendencies of surgeons.
Collapse
Affiliation(s)
- Masaru Kanehira
- Department of Surgery, The Jikei University Daisan Hospital, Komae, Japan
| | - Tomoyoshi Okamoto
- Department of Surgery, The Jikei University Daisan Hospital, Komae, Japan
| | - Kyohei Abe
- Department of Surgery, The Jikei University Daisan Hospital, Komae, Japan
| | - Jungo Yasuda
- Department of Surgery, The Jikei University School of Medicine, Minato-ku, Japan
| | - Shinji Onda
- Department of Surgery, The Jikei University School of Medicine, Minato-ku, Japan
| | - Yasuro Futagawa
- Department of Surgery, The Jikei University Daisan Hospital, Komae, Japan
| | - Toru Ikegami
- Department of Surgery, The Jikei University School of Medicine, Minato-ku, Japan
| | - Naoki Suzuki
- Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine, Minato-ku, Japan
| | - Asaki Hattori
- Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine, Minato-ku, Japan
| |
Collapse
|
|
8
|
Semenkov AV, Subbot VS. [Systematic review of current trends in preoperative planning of surgery for liver tumors]. Khirurgiia (Mosk) 2021:84-97. [PMID: 34363450 DOI: 10.17116/hirurgia202108184] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The purpose of the study was a systematic review of current trends in preoperative planning of surgery for liver tumors. These data will be valuable to determine the advantages and disadvantages of 3D modeling, augmented reality technology and 3D printing in preoperative planning of surgery for focal liver lesions.
Collapse
Affiliation(s)
- A V Semenkov
- Sklifosovsky Institute for Emergency Care, Moscow, Russia.,Sechenov First Moscow State Medical University, Moscow, Russia
| | - V S Subbot
- Sklifosovsky Institute for Emergency Care, Moscow, Russia.,Sechenov First Moscow State Medical University, Moscow, Russia
| |
Collapse
|
|
9
|
A narrative review on endopancreatic interventions: an innovative access to the pancreas. JOURNAL OF PANCREATOLOGY 2021. [DOI: 10.1097/jp9.0000000000000069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
|
|
10
|
Golse N, Petit A, Lewin M, Vibert E, Cotin S. Augmented Reality during Open Liver Surgery Using a Markerless Non-rigid Registration System. J Gastrointest Surg 2021; 25:662-671. [PMID: 32040812 DOI: 10.1007/s11605-020-04519-4] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/06/2019] [Accepted: 01/10/2020] [Indexed: 01/31/2023]
Abstract
INTRODUCTION Intraoperative navigation during liver resection remains difficult and requires high radiologic skills because liver anatomy is complex and patient-specific. Augmented reality (AR) during open liver surgery could be helpful to guide hepatectomies and optimize resection margins but faces many challenges when large parenchymal deformations take place. We aimed to experiment a new vision-based AR to assess its clinical feasibility and anatomical accuracy. PATIENTS AND METHODS Based on preoperative CT scan 3-D segmentations, we applied a non-rigid registration method, integrating a physics-based elastic model of the liver, computed in real time using an efficient finite element method. To fit the actual deformations, the model was driven by data provided by a single RGB-D camera. Five livers were considered in this experiment. In vivo AR was performed during hepatectomy (n = 4), with manual handling of the livers resulting in large realistic deformations. Ex vivo experiment (n = 1) consisted in repeated CT scans of explanted whole organ carrying internal metallic landmarks, in fixed deformations, and allowed us to analyze our estimated deformations and quantify spatial errors. RESULTS In vivo AR tests were successfully achieved in all patients with a fast and agile setup installation (< 10 min) and real-time overlay of the virtual anatomy onto the surgical field displayed on an external screen. In addition, an ex vivo quantification demonstrated a 7.9 mm root mean square error for the registration of internal landmarks. CONCLUSION These first experiments of a markerless AR provided promising results, requiring very little equipment and setup time, yet providing real-time AR with satisfactory 3D accuracy. These results must be confirmed in a larger prospective study to definitively assess the impact of such minimally invasive technology on pathological margins and oncological outcomes.
Collapse
Affiliation(s)
- Nicolas Golse
- Department of Surgery, Paul-Brousse Hospital, Assistance Publique Hôpitaux de Paris, Centre Hépato-Biliaire, 12 Avenue Paul Vaillant Couturier, 94804, Villejuif Cedex, France. .,DHU Hepatinov, 94800, Villejuif, France. .,INSERM, Unit 1193, 94800, Villejuif, France. .,Univ Paris-Sud, UMR-S 1193, 94800, Villejuif, France. .,Inria, Strasbourg, France.
| | | | - Maïté Lewin
- Department of Radiology, Paul-Brousse Hospital, Assistance Publique Hôpitaux de Paris, Centre Hépato-Biliaire, 94800, Villejuif, France
| | - Eric Vibert
- Department of Surgery, Paul-Brousse Hospital, Assistance Publique Hôpitaux de Paris, Centre Hépato-Biliaire, 12 Avenue Paul Vaillant Couturier, 94804, Villejuif Cedex, France.,DHU Hepatinov, 94800, Villejuif, France.,INSERM, Unit 1193, 94800, Villejuif, France.,Univ Paris-Sud, UMR-S 1193, 94800, Villejuif, France
| | | |
Collapse
|
|
11
|
Bari H, Wadhwani S, Dasari BVM. Role of artificial intelligence in hepatobiliary and pancreatic surgery. World J Gastrointest Surg 2021; 13:7-18. [PMID: 33552391 PMCID: PMC7830072 DOI: 10.4240/wjgs.v13.i1.7] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/28/2020] [Revised: 12/08/2020] [Accepted: 12/17/2020] [Indexed: 02/06/2023] Open
Abstract
Over the past decade, enhanced preoperative imaging and visualization, improved delineation of the complex anatomical structures of the liver and pancreas, and intra-operative technological advances have helped deliver the liver and pancreatic surgery with increased safety and better postoperative outcomes. Artificial intelligence (AI) has a major role to play in 3D visualization, virtual simulation, augmented reality that helps in the training of surgeons and the future delivery of conventional, laparoscopic, and robotic hepatobiliary and pancreatic (HPB) surgery; artificial neural networks and machine learning has the potential to revolutionize individualized patient care during the preoperative imaging, and postoperative surveillance. In this paper, we reviewed the existing evidence and outlined the potential for applying AI in the perioperative care of patients undergoing HPB surgery.
Collapse
Affiliation(s)
- Hassaan Bari
- Department of HPB and Liver Transplantation Surgery, Queen Elizabeth Hospital, Birmingham B15 2TH, United Kingdom
| | - Sharan Wadhwani
- Department of Radiology, Queen Elizabeth Hospital, Birmingham B15 2TH, United Kingdom
| | - Bobby V M Dasari
- Department of HPB and Liver Transplantation Surgery, Queen Elizabeth Hospital, Birmingham B15 2TH, United Kingdom
| |
Collapse
|
|
12
|
Müller PC, Haslebacher C, Steinemann DC, Müller-Stich BP, Hackert T, Peterhans M, Eigl B. Image-guided minimally invasive endopancreatic surgery using a computer-assisted navigation system. Surg Endosc 2020; 35:1610-1617. [PMID: 32253555 DOI: 10.1007/s00464-020-07540-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2020] [Accepted: 03/31/2020] [Indexed: 12/16/2022]
Abstract
BACKGROUND Minimally invasive endopancreatic surgery (EPS), performing a pancreatic resection from inside the pancreatic duct, has been proposed as an experimental alternative to duodenum-preserving pancreatic head resection in benign diseases such as chronic pancreatitis, but is complicated by difficult spatial orientation when trying to reach structures of interest. This study assessed the feasibility and potential benefits of image-guided EPS using a computer-assisted navigation system in artificial pancreas silicon model. METHODS A surgical navigation system displayed a 3D reconstruction of the original computed tomography (CT) scan and the endoscope in relation to a selected target structure. In a first step, different surface landmark (LM)-based and intraparenchymal LM-based approaches for image-to-physical space registration were evaluated. The accuracy of registration was measured as fiducial registration error (FRE). Subsequently, intrapancreatic lesions (n = 8) that were visible on preoperative imaging, but not on the endoscopic view, were targeted with a computer-assisted, image-guided endopancreatic resection technique in pancreas silicon models. After each experiment, a CT scan was obtained for measurement of the shortest distance from the resection cavity to the centre of the lesion. RESULTS Intraparenchymal LM registration [FRE 2.24 mm (1.40-2.85)] was more accurate than surface LM registration [FRE 3.46 mm (2.25-4.85); p = 0.035], but not more accurate than combined registration of intraparenchymal and surface LM [FRE 2.46 mm (1.60-3.35); p = 0.052]. Using image-guided EPS, six of seven lesions were successfully targeted. The median distance from the resection cavity to the centre of the lesion on CT was 1.52 mm (0-2.4). In one pancreas, a lesion could not be resected due to the fragility of the pancreas model. CONCLUSION Image-guided minimally invasive EPS using a computer-assisted navigation system enabled successful targeting of pancreatic lesions that were invisible on the endoscopic image, but detectable on preoperative imaging. In the clinical setting, this tool could facilitate complex minimally invasive and robotic pancreatic procedures.
Collapse
Affiliation(s)
- Philip C Müller
- Department of Visceral and Transplant Surgery, University Hospital Zurich, Rämistrasse 100, 8091, Zurich, Switzerland.
| | | | - Daniel C Steinemann
- Department of Surgery, Clarunis, University Hospital Basel, Basel, Switzerland
| | - Beat P Müller-Stich
- Department of General, Visceral and Transplantation Surgery, University Hospital Heidelberg, Heidelberg, Germany
| | - Thilo Hackert
- Department of General, Visceral and Transplantation Surgery, University Hospital Heidelberg, Heidelberg, Germany
| | | | - Benjamin Eigl
- CAScination AG, Bern, Switzerland.,ARTORG Center for Computer-Aided Surgery, University of Bern, Bern, Switzerland
| |
Collapse
|
|
13
|
Yasuda J, Okamoto T, Onda S, Fujioka S, Yanaga K, Suzuki N, Hattori A. Application of image-guided navigation system for laparoscopic hepatobiliary surgery. Asian J Endosc Surg 2020; 13:39-45. [PMID: 30945434 DOI: 10.1111/ases.12696] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/01/2018] [Revised: 01/10/2019] [Accepted: 01/16/2019] [Indexed: 01/16/2023]
Abstract
BACKGROUND To achieve safety of the operation, preoperative simulation became a routine practice for hepatobiliary and pancreatic (HBP) surgery. The use of intraoperative ultrasonography (IOUS) is essential in HBP surgery. There is a limitation in the use of IOUS in laparoscopic surgery (LS), for which a new intraoperative system is expected. We have developed an image-guided navigation system (IG-NS) for open HBP surgery since 2006, and we have applied our system to LS. The aim of this study is to evaluate the results of clinical application of IG-NS in LS. MATERIALS AND METHODS Eight patients underwent LS using IG-NS; LS consisted of cholecystectomy and hepatectomy in four patients each. After registration, the 3D models were superimposed on the surgical field. We performed LS while observing the navigation image. Moreover, we developed a support system for operations. RESULTS The average registration error was 8.8 mm for LS. Repeated registration was effective for organ deformation and improved the precision of IG-NS. By using various countermeasures, identification of the tumor's position and the setting of the resection line became easy. CONCLUSION As IG-NS provided real-time detailed and intuitive information, this intraoperative assist system may be an effective tool in LS.
Collapse
Affiliation(s)
- Jungo Yasuda
- Department of Surgery, The Jikei University School of Medicine, Tokyo, Japan
| | | | - Shinji Onda
- Department of Surgery, The Jikei University School of Medicine, Tokyo, Japan
| | - Shuuichi Fujioka
- Department of Surgery, The Jikei University School of Medicine, Tokyo, Japan
| | - Katsuhiko Yanaga
- Department of Surgery, The Jikei University School of Medicine, Tokyo, Japan
| | - Naoki Suzuki
- Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine, Tokyo, Japan
| | - Asaki Hattori
- Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine, Tokyo, Japan
| |
Collapse
|
|
14
|
Quero G, Lapergola A, Soler L, Shahbaz M, Hostettler A, Collins T, Marescaux J, Mutter D, Diana M, Pessaux P. Virtual and Augmented Reality in Oncologic Liver Surgery. Surg Oncol Clin N Am 2018; 28:31-44. [PMID: 30414680 DOI: 10.1016/j.soc.2018.08.002] [Citation(s) in RCA: 50] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Virtual reality (VR) and augmented reality (AR) in complex surgery are evolving technologies enabling improved preoperative planning and intraoperative navigation. The basis of these technologies is a computer-based generation of a patient-specific 3-dimensional model from Digital Imaging and Communications in Medicine (DICOM) data. This article provides a state-of-the- art overview on the clinical use of this technology with a specific focus on hepatic surgery. Although VR and AR are still in an evolving stage with only some clinical application today, these technologies have the potential to become a key factor in improving preoperative and intraoperative decision making.
Collapse
Affiliation(s)
- Giuseppe Quero
- IHU-Strasbourg, Institute of Image-Guided Surgery, 1 Place de l'Hôpital, Strasbourg 67091, France
| | - Alfonso Lapergola
- IRCAD, Research Institute Against Cancer of the Digestive System, 1 Place de l'Hôpital, Strasbourg 67091, France
| | - Luc Soler
- IRCAD, Research Institute Against Cancer of the Digestive System, 1 Place de l'Hôpital, Strasbourg 67091, France
| | - Muhammad Shahbaz
- IHU-Strasbourg, Institute of Image-Guided Surgery, 1 Place de l'Hôpital, Strasbourg 67091, France
| | - Alexandre Hostettler
- IRCAD, Research Institute Against Cancer of the Digestive System, 1 Place de l'Hôpital, Strasbourg 67091, France
| | - Toby Collins
- IRCAD, Research Institute Against Cancer of the Digestive System, 1 Place de l'Hôpital, Strasbourg 67091, France
| | - Jacques Marescaux
- IHU-Strasbourg, Institute of Image-Guided Surgery, 1 Place de l'Hôpital, Strasbourg 67091, France; IRCAD, Research Institute Against Cancer of the Digestive System, 1 Place de l'Hôpital, Strasbourg 67091, France
| | - Didier Mutter
- Department of General, Digestive and Endocrine Surgery, University Hospital of Strasbourg, 1 Place de l'Hôpital, Strasbourg 67091, France
| | - Michele Diana
- IHU-Strasbourg, Institute of Image-Guided Surgery, 1 Place de l'Hôpital, Strasbourg 67091, France; IRCAD, Research Institute Against Cancer of the Digestive System, 1 Place de l'Hôpital, Strasbourg 67091, France; Department of General, Digestive and Endocrine Surgery, University Hospital of Strasbourg, 1 Place de l'Hôpital, Strasbourg 67091, France
| | - Patrick Pessaux
- IHU-Strasbourg, Institute of Image-Guided Surgery, 1 Place de l'Hôpital, Strasbourg 67091, France; IRCAD, Research Institute Against Cancer of the Digestive System, 1 Place de l'Hôpital, Strasbourg 67091, France; Department of General, Digestive and Endocrine Surgery, University Hospital of Strasbourg, 1 Place de l'Hôpital, Strasbourg 67091, France.
| |
Collapse
|
|
15
|
Fida B, Cutolo F, di Franco G, Ferrari M, Ferrari V. Augmented reality in open surgery. Updates Surg 2018; 70:389-400. [PMID: 30006832 DOI: 10.1007/s13304-018-0567-8] [Citation(s) in RCA: 52] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2018] [Accepted: 07/08/2018] [Indexed: 12/17/2022]
Abstract
Augmented reality (AR) has been successfully providing surgeons an extensive visual information of surgical anatomy to assist them throughout the procedure. AR allows surgeons to view surgical field through the superimposed 3D virtual model of anatomical details. However, open surgery presents new challenges. This study provides a comprehensive overview of the available literature regarding the use of AR in open surgery, both in clinical and simulated settings. In this way, we aim to analyze the current trends and solutions to help developers and end/users discuss and understand benefits and shortcomings of these systems in open surgery. We performed a PubMed search of the available literature updated to January 2018 using the terms (1) "augmented reality" AND "open surgery", (2) "augmented reality" AND "surgery" NOT "laparoscopic" NOT "laparoscope" NOT "robotic", (3) "mixed reality" AND "open surgery", (4) "mixed reality" AND "surgery" NOT "laparoscopic" NOT "laparoscope" NOT "robotic". The aspects evaluated were the following: real data source, virtual data source, visualization processing modality, tracking modality, registration technique, and AR display type. The initial search yielded 502 studies. After removing the duplicates and by reading abstracts, a total of 13 relevant studies were chosen. In 1 out of 13 studies, in vitro experiments were performed, while the rest of the studies were carried out in a clinical setting including pancreatic, hepatobiliary, and urogenital surgeries. AR system in open surgery appears as a versatile and reliable tool in the operating room. However, some technological limitations need to be addressed before implementing it into the routine practice.
Collapse
Affiliation(s)
- Benish Fida
- Department of Information Engineering, University of Pisa, Pisa, Italy.,Department of Translational Research and New Technologies in Medicine and Surgery, EndoCAS Center, University of Pisa, Pisa, Italy
| | - Fabrizio Cutolo
- Department of Information Engineering, University of Pisa, Pisa, Italy. .,Department of Translational Research and New Technologies in Medicine and Surgery, EndoCAS Center, University of Pisa, Pisa, Italy.
| | - Gregorio di Franco
- General Surgery Unit, Department of Surgery, Translational and New Technologies, University of Pisa, Pisa, Italy
| | - Mauro Ferrari
- Department of Translational Research and New Technologies in Medicine and Surgery, EndoCAS Center, University of Pisa, Pisa, Italy.,Vascular Surgery Unit, Cisanello University Hospital AOUP, Pisa, Italy
| | - Vincenzo Ferrari
- Department of Information Engineering, University of Pisa, Pisa, Italy.,Department of Translational Research and New Technologies in Medicine and Surgery, EndoCAS Center, University of Pisa, Pisa, Italy
| |
Collapse
|
|
16
|
Yasuda J, Okamoto T, Onda S, Futagawa Y, Yanaga K, Suzuki N, Hattori A. Novel navigation system by augmented reality technology using a tablet PC for hepatobiliary and pancreatic surgery. Int J Med Robot 2018; 14:e1921. [PMID: 29749069 DOI: 10.1002/rcs.1921] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2017] [Revised: 04/06/2018] [Accepted: 04/08/2018] [Indexed: 12/14/2022]
Abstract
BACKGROUND We previously developed an image-guided navigation system (IG-NS) using augmented reality technology for hepatobiliary and pancreatic (HBP) surgery. This system superimposed a 3D model onto a stereoscope-captured surgical field (i.e., the scope method). Unfortunately, this method requires an expensive stereoscope, surgeons have to shift their eyesight away from the surgical field, and the method has poor controllability. Therefore, an IG-NS using a tablet PC (i.e., the tablet method) was developed. The aim of the current study is to evaluate the efficiency of this novel method. METHODS We studied 9 patients, for whom a 3D model was created from computed tomography images. After registration was performed, the 3D model was superimposed onto the surgical field, which was captured by the tablet PC's camera. RESULTS The IG-NS could be applied with very little time lag. The visibility and controllability of the tablet method were superior to those of the scope method. It was especially useful in surgery for multiple metastatic liver carcinoma due to easy localization of the position of the carcinomas and vessels. CONCLUSIONS We successfully developed the tablet method and tested it in a clinical setting. This system may contribute to surgical efficacy and improve the educational effects.
Collapse
Affiliation(s)
- Jungo Yasuda
- Department of Surgery, The Jikei University School of Medicine, Tokyo, Japan
| | - Tomoyoshi Okamoto
- Department of Surgery, The Jikei University Daisan Hospital, Tokyo, Japan
| | - Shinji Onda
- Department of Surgery, The Jikei University School of Medicine, Tokyo, Japan
| | - Yasuro Futagawa
- Department of Surgery, The Jikei University Daisan Hospital, Tokyo, Japan
| | - Katsuhiko Yanaga
- Department of Surgery, The Jikei University School of Medicine, Tokyo, Japan
| | - Naoki Suzuki
- Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine, Tokyo, Japan
| | - Asaki Hattori
- Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine, Tokyo, Japan
| |
Collapse
|
|
17
|
Augmented reality technology for preoperative planning and intraoperative navigation during hepatobiliary surgery: A review of current methods. Hepatobiliary Pancreat Dis Int 2018; 17:101-112. [PMID: 29567047 DOI: 10.1016/j.hbpd.2018.02.002] [Citation(s) in RCA: 67] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/31/2017] [Accepted: 11/16/2017] [Indexed: 02/05/2023]
Abstract
BACKGROUND Augmented reality (AR) technology is used to reconstruct three-dimensional (3D) images of hepatic and biliary structures from computed tomography and magnetic resonance imaging data, and to superimpose the virtual images onto a view of the surgical field. In liver surgery, these superimposed virtual images help the surgeon to visualize intrahepatic structures and therefore, to operate precisely and to improve clinical outcomes. DATA SOURCES The keywords "augmented reality", "liver", "laparoscopic" and "hepatectomy" were used for searching publications in the PubMed database. The primary source of literatures was from peer-reviewed journals up to December 2016. Additional articles were identified by manual search of references found in the key articles. RESULTS In general, AR technology mainly includes 3D reconstruction, display, registration as well as tracking techniques and has recently been adopted gradually for liver surgeries including laparoscopy and laparotomy with video-based AR assisted laparoscopic resection as the main technical application. By applying AR technology, blood vessels and tumor structures in the liver can be displayed during surgery, which permits precise navigation during complex surgical procedures. Liver transformation and registration errors during surgery were the main factors that limit the application of AR technology. CONCLUSIONS With recent advances, AR technologies have the potential to improve hepatobiliary surgical procedures. However, additional clinical studies will be required to evaluate AR as a tool for reducing postoperative morbidity and mortality and for the improvement of long-term clinical outcomes. Future research is needed in the fusion of multiple imaging modalities, improving biomechanical liver modeling, and enhancing image data processing and tracking technologies to increase the accuracy of current AR methods.
Collapse
|
|
18
|
Tang R, Ma L, Xiang C, Wang X, Li A, Liao H, Dong J. Augmented reality navigation in open surgery for hilar cholangiocarcinoma resection with hemihepatectomy using video-based in situ three-dimensional anatomical modeling: A case report. Medicine (Baltimore) 2017; 96:e8083. [PMID: 28906410 PMCID: PMC5604679 DOI: 10.1097/md.0000000000008083] [Citation(s) in RCA: 29] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/22/2022] Open
Abstract
RATIONALE Patients who undergo hilar cholangiocarcinoma (HCAC) resection with concomitant hepatectomy have a high risk of postoperative morbidity and mortality due to surgical trauma to the hepatic and biliary vasculature. PATIENT CONCERNS A 58-year-old Chinese man with yellowing skin and sclera, abdominal distension, pruritus, and anorexia for approximately 3 weeks. DIAGNOSES Magnetic resonance cholangiopancreatography and enhanced computed tomography (CT) scanning revealed a mass over the biliary tree at the porta hepatis, which diagnosed to be s a hilar cholangiocarcinoma. INTERVENTION Three-dimensional (3D) images of the patient's hepatic and biliary structures were reconstructed preoperatively from CT data, and the 3D images were used for preoperative planning and augmented reality (AR)-assisted intraoperative navigation during open HCAC resection with hemihepatectomy. A 3D-printed model of the patient's biliary structures was also used intraoperatively as a visual reference. OUTCOMES No serious postoperative complications occurred, and the patient was tumor-free at the 9-month follow-up examination based on CT results. LESSONS AR-assisted preoperative planning and intraoperative navigation might be beneficial in other patients with HCAC patients to reduce postoperative complications and ensure disease-free survival. In our postoperative analysis, we also found that, when the3D images were superimposed 3D-printed model using a see-through integral video graphy display device, our senses of depth perception and motion parallax were improved, compared with that which we had experienced intraoperatively using the videobased AR display system.
Collapse
Affiliation(s)
- Rui Tang
- Department of Hepatopancreatobiliary Surgery, Beijing Tsinghua Changgung Hospital
| | - Longfei Ma
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Canhong Xiang
- Department of Hepatopancreatobiliary Surgery, Beijing Tsinghua Changgung Hospital
| | - Xuedong Wang
- Department of Hepatopancreatobiliary Surgery, Beijing Tsinghua Changgung Hospital
| | - Ang Li
- Department of Hepatopancreatobiliary Surgery, Beijing Tsinghua Changgung Hospital
| | - Hongen Liao
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Jiahong Dong
- Department of Hepatopancreatobiliary Surgery, Beijing Tsinghua Changgung Hospital
| |
Collapse
|
|
19
|
Recent Development of Augmented Reality in Surgery: A Review. JOURNAL OF HEALTHCARE ENGINEERING 2017; 2017:4574172. [PMID: 29065604 PMCID: PMC5585624 DOI: 10.1155/2017/4574172] [Citation(s) in RCA: 160] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/12/2017] [Accepted: 07/03/2017] [Indexed: 12/11/2022]
Abstract
Introduction The development augmented reality devices allow physicians to incorporate data visualization into diagnostic and treatment procedures to improve work efficiency, safety, and cost and to enhance surgical training. However, the awareness of possibilities of augmented reality is generally low. This review evaluates whether augmented reality can presently improve the results of surgical procedures. Methods We performed a review of available literature dating from 2010 to November 2016 by searching PubMed and Scopus using the terms “augmented reality” and “surgery.” Results. The initial search yielded 808 studies. After removing duplicates and including only journal articles, a total of 417 studies were identified. By reading of abstracts, 91 relevant studies were chosen to be included. 11 references were gathered by cross-referencing. A total of 102 studies were included in this review. Conclusions The present literature suggest an increasing interest of surgeons regarding employing augmented reality into surgery leading to improved safety and efficacy of surgical procedures. Many studies showed that the performance of newly devised augmented reality systems is comparable to traditional techniques. However, several problems need to be addressed before augmented reality is implemented into the routine practice.
Collapse
|
|
20
|
Bond L, Schulz B, VanMeter T, Martin R. Intra-operative navigation of a 3-dimensional needle localization system for precision of irreversible electroporation needles in locally advanced pancreatic cancer. Eur J Surg Oncol 2017; 43:337-343. [DOI: 10.1016/j.ejso.2016.09.009] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2016] [Revised: 08/29/2016] [Accepted: 09/06/2016] [Indexed: 12/18/2022] Open
|
|
21
|
Ntourakis D, Memeo R, Soler L, Marescaux J, Mutter D, Pessaux P. Augmented Reality Guidance for the Resection of Missing Colorectal Liver Metastases: An Initial Experience. World J Surg 2016; 40:419-26. [PMID: 26316112 DOI: 10.1007/s00268-015-3229-8] [Citation(s) in RCA: 58] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
BACKGROUND Modern chemotherapy achieves the shrinking of colorectal cancer liver metastases (CRLM) to such extent that they may disappear from radiological imaging. Disappearing CRLM rarely represents a complete pathological remission and have an important risk of recurrence. Augmented reality (AR) consists in the fusion of real-time patient images with a computer-generated 3D virtual patient model created from pre-operative medical imaging. The aim of this prospective pilot study is to investigate the potential of AR navigation as a tool to help locate and surgically resect missing CRLM. METHODS A 3D virtual anatomical model was created from thoracoabdominal CT-scans using customary software (VR RENDER(®), IRCAD). The virtual model was superimposed to the operative field using an Exoscope (VITOM(®), Karl Storz, Tüttlingen, Germany). Virtual and real images were manually registered in real-time using a video mixer, based on external anatomical landmarks with an estimated accuracy of 5 mm. This modality was tested in three patients, with four missing CRLM that had sizes from 12 to 24 mm, undergoing laparotomy after receiving pre-operative oxaliplatin-based chemotherapy. RESULTS AR display and fine registration was performed within 6 min. AR helped detect all four missing CRLM, and guided their resection. In all cases the planned security margin of 1 cm was clear and resections were confirmed to be R0 by pathology. There was no postoperative major morbidity or mortality. No local recurrence occurred in the follow-up period of 6-22 months. CONCLUSIONS This initial experience suggests that AR may be a helpful navigation tool for the resection of missing CRLM.
Collapse
Affiliation(s)
- Dimitrios Ntourakis
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France.
| | - Ricardo Memeo
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France
| | - Luc Soler
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France
| | - Jacques Marescaux
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France
| | - Didier Mutter
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France
| | - Patrick Pessaux
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France.
| |
Collapse
|
|
22
|
Ntourakis D, Memeo R, Soler L, Marescaux J, Mutter D, Pessaux P. Augmented Reality Guidance for the Resection of Missing Colorectal Liver Metastases: An Initial Experience. World J Surg 2016. [PMID: 26316112 DOI: 10.1007/-s00268-015-3229-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
BACKGROUND Modern chemotherapy achieves the shrinking of colorectal cancer liver metastases (CRLM) to such extent that they may disappear from radiological imaging. Disappearing CRLM rarely represents a complete pathological remission and have an important risk of recurrence. Augmented reality (AR) consists in the fusion of real-time patient images with a computer-generated 3D virtual patient model created from pre-operative medical imaging. The aim of this prospective pilot study is to investigate the potential of AR navigation as a tool to help locate and surgically resect missing CRLM. METHODS A 3D virtual anatomical model was created from thoracoabdominal CT-scans using customary software (VR RENDER(®), IRCAD). The virtual model was superimposed to the operative field using an Exoscope (VITOM(®), Karl Storz, Tüttlingen, Germany). Virtual and real images were manually registered in real-time using a video mixer, based on external anatomical landmarks with an estimated accuracy of 5 mm. This modality was tested in three patients, with four missing CRLM that had sizes from 12 to 24 mm, undergoing laparotomy after receiving pre-operative oxaliplatin-based chemotherapy. RESULTS AR display and fine registration was performed within 6 min. AR helped detect all four missing CRLM, and guided their resection. In all cases the planned security margin of 1 cm was clear and resections were confirmed to be R0 by pathology. There was no postoperative major morbidity or mortality. No local recurrence occurred in the follow-up period of 6-22 months. CONCLUSIONS This initial experience suggests that AR may be a helpful navigation tool for the resection of missing CRLM.
Collapse
Affiliation(s)
- Dimitrios Ntourakis
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France.
| | - Ricardo Memeo
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France
| | - Luc Soler
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France
| | - Jacques Marescaux
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France
| | - Didier Mutter
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France
| | - Patrick Pessaux
- IRCAD-IHU, University of Strasbourg, 1 place de l'Hôpital, 67091, Strasbourg, France.
| |
Collapse
|
|
23
|
Okamoto T, Onda S, Yasuda J, Yanaga K, Suzuki N, Hattori A. Navigation surgery using an augmented reality for pancreatectomy. Dig Surg 2015; 32:117-23. [PMID: 25766302 DOI: 10.1159/000371860] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/30/2014] [Accepted: 12/31/2014] [Indexed: 12/18/2022]
Abstract
AIM The aim of this study was to evaluate the utility of navigation surgery using augmented reality technology (AR-based NS) for pancreatectomy. METHODS The 3D reconstructed images from CT were created by segmentation. The initial registration was performed by using the optical location sensor. The reconstructed images were superimposed onto the real organs in the monitor display. Of the 19 patients who had undergone hepatobiliary and pancreatic surgery using AR-based NS, the accuracy, visualization ability, and utility of our system were assessed in five cases with pancreatectomy. RESULTS The position of each organ in the surface-rendering image corresponded almost to that of the actual organ. Reference to the display image allowed for safe dissection while preserving the adjacent vessels or organs. The locations of the lesions and resection line on the targeted organ were overlaid on the operating field. The initial mean registration error was improved to approximately 5 mm by our refinements. However, several problems such as registration accuracy, portability and cost still remain. CONCLUSION AR-based NS contributed to accurate and effective surgical resection in pancreatectomy. The pancreas appears to be a suitable organ for further investigations. This technology is promising to improve surgical quality, training, and education.
Collapse
Affiliation(s)
- Tomoyoshi Okamoto
- Department of Surgery, The Jikei University Daisan Hospital, Tokyo, Japan
| | | | | | | | | | | |
Collapse
|
|
24
|
Okamoto T, Onda S, Yanaga K, Suzuki N, Hattori A. Clinical application of navigation surgery using augmented reality in the abdominal field. Surg Today 2014; 45:397-406. [PMID: 24898629 DOI: 10.1007/s00595-014-0946-9] [Citation(s) in RCA: 48] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2013] [Accepted: 01/23/2014] [Indexed: 12/20/2022]
Abstract
This article presents general principles and recent advancements in the clinical application of augmented reality-based navigation surgery (AR based NS) for abdominal procedures and includes a description of our clinical trial and subsequent outcomes. Moreover, current problems and future aspects are discussed. The development of AR-based NS in the abdomen is delayed compared with another field because of the problem of intraoperative organ deformations or the existence of established modalities. Although there are a few reports on the clinical use of AR-based NS for digestive surgery, sophisticated technologies in urology have often been reported. However, the rapid widespread use of video- or robot assisted surgeries requires this technology. We have worked to develop a system of AR-based NS for hepatobiliary and pancreatic surgery. Then we developed a short rigid scope that enables surgeons to obtain 3D view. We recently focused on pancreatic surgery, because intraoperative organ shifting is minimal. The position of each organ in overlaid image almost corresponded with that of the actual organ with about 5 mm of mean registration errors. Intraoperative information generated from this system provided us with useful navigation. However, AR-based NS has several problems to overcome such as organ deformity, evaluation of utility, portability or cost.
Collapse
Affiliation(s)
- Tomoyoshi Okamoto
- Department of Surgery, The Jikei University Daisan Hospital, 4-11-1 Izumihoncho, Komae-shi, Tokyo, Japan,
| | | | | | | | | |
Collapse
|
|
25
|
Onda S, Okamoto T, Kanehira M, Suzuki F, Ito R, Fujioka S, Suzuki N, Hattori A, Yanaga K. Identification of inferior pancreaticoduodenal artery during pancreaticoduodenectomy using augmented reality-based navigation system. JOURNAL OF HEPATO-BILIARY-PANCREATIC SCIENCES 2013; 21:281-7. [PMID: 23970384 DOI: 10.1002/jhbp.25] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
BACKGROUND In pancreaticoduodenectomy (PD), early ligation of the inferior pancreaticoduodenal artery (IPDA) before efferent veins has been advocated to decrease blood loss by congestion of the pancreatic head to be resected. In this study, we herein report the utility of early identification of the IPDA using an augmented reality (AR)-based navigation system (NS). METHODS Seven nonconsecutive patients underwent PD using AR-based NS. After paired-point matching registration, the reconstructed image obtained by preoperative computed tomography (CT) was fused with a real-time operative field image and displayed on 3D monitors. The vascular reconstructed images, including the superior mesenteric artery, jejunal artery, and IPDA were visualized to facilitate image-guided surgical procedures. We compared operating time and intraoperative blood loss of six patients who successfully underwent identification of IPDA using AR-based NS (group A) with nine patients who underwent early ligation of IPDA without using AR (group B) and 18 patients who underwent a conventional PD (group C). RESULTS The IPDA or the jejunal artery was rapidly identified and ligated in six patients. The mean operating time and intraoperative blood loss in group A was 415 min and 901 ml, respectively. There was no significant difference in operating time and intraoperative blood loss among the groups. CONCLUSIONS The AR-based NS provided precise anatomical information, which allowed the surgeons to rapidly identify and perform early ligation of IPDA in PD.
Collapse
Affiliation(s)
- Shinji Onda
- Division of Digestive Surgery, The Jikei University Graduate School of Medicine, 3-25-8 Nishi-Shimbashi, Minato-ku, Tokyo 105-8461, Japan.
| | | | | | | | | | | | | | | | | |
Collapse
|