|
1
|
Galli R, Uckermann O. Toward cancer detection by label-free microscopic imaging in oncological surgery: Techniques, instrumentation and applications. Micron 2025; 191:103800. [PMID: 39923310 DOI: 10.1016/j.micron.2025.103800] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2024] [Revised: 01/31/2025] [Accepted: 02/03/2025] [Indexed: 02/11/2025]
Abstract
This review examines the clinical application of label-free microscopy and spectroscopy, which are based on optical signals emitted by tissue components. Over the past three decades, a variety of techniques have been investigated with the aim of developing an in situ histopathology method that can rapidly and accurately identify tumor margins during surgical procedures. These techniques can be divided into two groups. One group encompasses techniques exploiting linear optical signals, and includes infrared and Raman microspectroscopy, and autofluorescence microscopy. The second group includes techniques based on nonlinear optical signals, including harmonic generation, coherent Raman scattering, and multiphoton autofluorescence microscopy. Some of these methods provide comparable information, while others are complementary. However, all of them have distinct advantages and disadvantages due to their inherent nature. The first part of the review provides an explanation of the underlying physics of the excitation mechanisms and a description of the instrumentation. It also covers endomicroscopy and data analysis, which are important for understanding the current limitations in implementing label-free techniques in clinical settings. The second part of the review describes the application of label-free microscopy imaging to improve oncological surgery with focus on brain tumors and selected gastrointestinal cancers, and provides a critical assessment of the current state of translation of these methods into clinical practice. Finally, the potential of confocal laser endomicroscopy for the acquisition of autofluorescence is discussed in the context of immediate clinical applications.
Collapse
Affiliation(s)
- Roberta Galli
- Medical Physics and Biomedical Engineering, Faculty of Medicine, TU Dresden, Fetscherstr. 74, Dresden 01307, Germany.
| | - Ortrud Uckermann
- Department of Neurosurgery, University Hospital Carl Gustav Carus, TU Dresden, Fetscherstr. 74, Dresden 01307, Germany
| |
Collapse
|
|
2
|
Romeo M, Dallio M, Napolitano C, Basile C, Di Nardo F, Vaia P, Iodice P, Federico A. Clinical Applications of Artificial Intelligence (AI) in Human Cancer: Is It Time to Update the Diagnostic and Predictive Models in Managing Hepatocellular Carcinoma (HCC)? Diagnostics (Basel) 2025; 15:252. [PMID: 39941182 PMCID: PMC11817573 DOI: 10.3390/diagnostics15030252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2024] [Revised: 01/20/2025] [Accepted: 01/21/2025] [Indexed: 02/16/2025] Open
Abstract
In recent years, novel findings have progressively and promisingly supported the potential role of Artificial intelligence (AI) in transforming the management of various neoplasms, including hepatocellular carcinoma (HCC). HCC represents the most common primary liver cancer. Alarmingly, the HCC incidence is dramatically increasing worldwide due to the simultaneous "pandemic" spreading of metabolic dysfunction-associated steatotic liver disease (MASLD). MASLD currently constitutes the leading cause of chronic hepatic damage (steatosis and steatohepatitis), fibrosis, and liver cirrhosis, configuring a scenario where an HCC onset has been reported even in the early disease stage. On the other hand, HCC represents a serious plague, significantly burdening the outcomes of chronic hepatitis B (HBV) and hepatitis C (HCV) virus-infected patients. Despite the recent progress in the management of this cancer, the overall prognosis for advanced-stage HCC patients continues to be poor, suggesting the absolute need to develop personalized healthcare strategies further. In this "cold war", machine learning techniques and neural networks are emerging as weapons, able to identify the patterns and biomarkers that would have normally escaped human observation. Using advanced algorithms, AI can analyze large volumes of clinical data and medical images (including routinely obtained ultrasound data) with an elevated accuracy, facilitating early diagnosis, improving the performance of predictive models, and supporting the multidisciplinary (oncologist, gastroenterologist, surgeon, radiologist) team in opting for the best "tailored" individual treatment. Additionally, AI can significantly contribute to enhancing the effectiveness of metabolomics-radiomics-based models, promoting the identification of specific HCC-pathogenetic molecules as new targets for realizing novel therapeutic regimens. In the era of precision medicine, integrating AI into routine clinical practice appears as a promising frontier, opening new avenues for liver cancer research and treatment.
Collapse
Affiliation(s)
- Mario Romeo
- Hepatogastroenterology Division, Department of Precision Medicine, University of Campania Luigi Vanvitelli, 80138 Naples, Italy; (M.R.); (C.N.); (C.B.); (F.D.N.); (P.V.); (A.F.)
| | - Marcello Dallio
- Hepatogastroenterology Division, Department of Precision Medicine, University of Campania Luigi Vanvitelli, 80138 Naples, Italy; (M.R.); (C.N.); (C.B.); (F.D.N.); (P.V.); (A.F.)
| | - Carmine Napolitano
- Hepatogastroenterology Division, Department of Precision Medicine, University of Campania Luigi Vanvitelli, 80138 Naples, Italy; (M.R.); (C.N.); (C.B.); (F.D.N.); (P.V.); (A.F.)
| | - Claudio Basile
- Hepatogastroenterology Division, Department of Precision Medicine, University of Campania Luigi Vanvitelli, 80138 Naples, Italy; (M.R.); (C.N.); (C.B.); (F.D.N.); (P.V.); (A.F.)
| | - Fiammetta Di Nardo
- Hepatogastroenterology Division, Department of Precision Medicine, University of Campania Luigi Vanvitelli, 80138 Naples, Italy; (M.R.); (C.N.); (C.B.); (F.D.N.); (P.V.); (A.F.)
| | - Paolo Vaia
- Hepatogastroenterology Division, Department of Precision Medicine, University of Campania Luigi Vanvitelli, 80138 Naples, Italy; (M.R.); (C.N.); (C.B.); (F.D.N.); (P.V.); (A.F.)
| | | | - Alessandro Federico
- Hepatogastroenterology Division, Department of Precision Medicine, University of Campania Luigi Vanvitelli, 80138 Naples, Italy; (M.R.); (C.N.); (C.B.); (F.D.N.); (P.V.); (A.F.)
| |
Collapse
|
|
3
|
Wu C, Chen Q, Wang H, Guan Y, Mian Z, Huang C, Ruan C, Song Q, Jiang H, Pan J, Li X. A review of deep learning approaches for multimodal image segmentation of liver cancer. J Appl Clin Med Phys 2024; 25:e14540. [PMID: 39374312 PMCID: PMC11633801 DOI: 10.1002/acm2.14540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2024] [Revised: 05/30/2024] [Accepted: 08/13/2024] [Indexed: 10/09/2024] Open
Abstract
This review examines the recent developments in deep learning (DL) techniques applied to multimodal fusion image segmentation for liver cancer. Hepatocellular carcinoma is a highly dangerous malignant tumor that requires accurate image segmentation for effective treatment and disease monitoring. Multimodal image fusion has the potential to offer more comprehensive information and more precise segmentation, and DL techniques have achieved remarkable progress in this domain. This paper starts with an introduction to liver cancer, then explains the preprocessing and fusion methods for multimodal images, then explores the application of DL methods in this area. Various DL architectures such as convolutional neural networks (CNN) and U-Net are discussed and their benefits in multimodal image fusion segmentation. Furthermore, various evaluation metrics and datasets currently used to measure the performance of segmentation models are reviewed. While reviewing the progress, the challenges of current research, such as data imbalance, model generalization, and model interpretability, are emphasized and future research directions are suggested. The application of DL in multimodal image segmentation for liver cancer is transforming the field of medical imaging and is expected to further enhance the accuracy and efficiency of clinical decision making. This review provides useful insights and guidance for medical practitioners.
Collapse
Affiliation(s)
- Chaopeng Wu
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
| | - Qiyao Chen
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
| | - Haoyu Wang
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
| | - Yu Guan
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
| | - Zhangyang Mian
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
| | - Cong Huang
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
| | - Changli Ruan
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
| | - Qibin Song
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
| | - Hao Jiang
- School of Electronic InformationWuhan UniversityWuhanHubeiChina
| | - Jinghui Pan
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
- School of Electronic InformationWuhan UniversityWuhanHubeiChina
| | - Xiangpan Li
- Department of Radiation OncologyRenmin HospitalWuhan UniversityWuhanHubeiChina
| |
Collapse
|
|
4
|
Wang S, Pan J, Zhang X, Li Y, Liu W, Lin R, Wang X, Kang D, Li Z, Huang F, Chen L, Chen J. Towards next-generation diagnostic pathology: AI-empowered label-free multiphoton microscopy. LIGHT, SCIENCE & APPLICATIONS 2024; 13:254. [PMID: 39277586 PMCID: PMC11401902 DOI: 10.1038/s41377-024-01597-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/26/2024] [Revised: 08/04/2024] [Accepted: 08/21/2024] [Indexed: 09/17/2024]
Abstract
Diagnostic pathology, historically dependent on visual scrutiny by experts, is essential for disease detection. Advances in digital pathology and developments in computer vision technology have led to the application of artificial intelligence (AI) in this field. Despite these advancements, the variability in pathologists' subjective interpretations of diagnostic criteria can lead to inconsistent outcomes. To meet the need for precision in cancer therapies, there is an increasing demand for accurate pathological diagnoses. Consequently, traditional diagnostic pathology is evolving towards "next-generation diagnostic pathology", prioritizing on the development of a multi-dimensional, intelligent diagnostic approach. Using nonlinear optical effects arising from the interaction of light with biological tissues, multiphoton microscopy (MPM) enables high-resolution label-free imaging of multiple intrinsic components across various human pathological tissues. AI-empowered MPM further improves the accuracy and efficiency of diagnosis, holding promise for providing auxiliary pathology diagnostic methods based on multiphoton diagnostic criteria. In this review, we systematically outline the applications of MPM in pathological diagnosis across various human diseases, and summarize common multiphoton diagnostic features. Moreover, we examine the significant role of AI in enhancing multiphoton pathological diagnosis, including aspects such as image preprocessing, refined differential diagnosis, and the prognostication of outcomes. We also discuss the challenges and perspectives faced by the integration of MPM and AI, encompassing equipment, datasets, analytical models, and integration into the existing clinical pathways. Finally, the review explores the synergy between AI and label-free MPM to forge novel diagnostic frameworks, aiming to accelerate the adoption and implementation of intelligent multiphoton pathology systems in clinical settings.
Collapse
Affiliation(s)
- Shu Wang
- School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, 350108, China
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China
| | - Junlin Pan
- School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, 350108, China
| | - Xiao Zhang
- College of Computer and Data Science, Fuzhou University, Fuzhou, 350108, China
| | - Yueying Li
- School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, 350108, China
| | - Wenxi Liu
- College of Computer and Data Science, Fuzhou University, Fuzhou, 350108, China
| | - Ruolan Lin
- Department of Radiology, Fujian Medical University Union Hospital, Fuzhou, 350001, China
| | - Xingfu Wang
- Department of Pathology, The First Affiliated Hospital of Fujian Medical University, Fuzhou, 350005, China
| | - Deyong Kang
- Department of Pathology, Fujian Medical University Union Hospital, Fuzhou, 350001, China
| | - Zhijun Li
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China
| | - Feng Huang
- School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, 350108, China.
| | - Liangyi Chen
- New Cornerstone Laboratory, State Key Laboratory of Membrane Biology, Beijing Key Laboratory of Cardiometabolic Molecular Medicine, Institute of Molecular Medicine, National Biomedical Imaging Center, School of Future Technology, Peking University, Beijing, 100091, China.
| | - Jianxin Chen
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China.
| |
Collapse
|
|
5
|
Liu B, Liu Y, Liu W, Luo T, Chen W, Lin C, Lin L, Zhuo S, Sun Y. Label-free imaging diagnosis and collagen-optical evaluation of endometrioid adenocarcinoma with multiphoton microscopy. JOURNAL OF BIOPHOTONICS 2024; 17:e202400177. [PMID: 38887864 DOI: 10.1002/jbio.202400177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/27/2024] [Revised: 05/17/2024] [Accepted: 05/21/2024] [Indexed: 06/20/2024]
Abstract
The assessment of tumor grade and pathological stage plays a pivotal role in determining the treatment strategy and predicting the prognosis of endometrial cancer. In this study, we employed multiphoton microscopy (MPM) to establish distinctive optical pathological signatures specific to endometrioid adenocarcinoma (EAC), while also assessing the diagnostic sensitivity, specificity, and accuracy of MPM for this particular malignancy. The MPM technique exhibits robust capability in discriminating between benign hyperplasia and various grades of cancer tissue, with statistically significant differences observed in nucleocytoplasmic ratio and second harmonic generation/two-photon excited fluorescence intensity. Moreover, by utilizing semi-automated image analysis, we identified notable disparities in six collagen signatures between benign and malignant endometrial stroma. Our study demonstrates that MPM can differentiate between benign endometrial hyperplasia and EAC without labels, while also quantitatively assessing changes in the tumor microenvironment by analyzing collagen signatures in the endometrial stromal tissue.
Collapse
Affiliation(s)
- Bin Liu
- Department of Gynecology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
| | - Yan Liu
- Fujian Provincial Key Laboratory of Brain Aging and Neurodegenerative Diseases, School of Basic Medical Sciences, Fujian Medical University, Fuzhou, China
| | - Wenju Liu
- Department of Gastric Surgery, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
| | - Tianyi Luo
- School of Science, Jimei University, Xiamen, Fujian, China
| | - Wei Chen
- Department of Gynecology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
| | - Cuibo Lin
- Department of Gynecology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
| | - Ling Lin
- Department of Gynecology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
| | - Shuangmu Zhuo
- School of Science, Jimei University, Xiamen, Fujian, China
| | - Yang Sun
- Department of Gynecology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
| |
Collapse
|
|
6
|
Myslicka M, Kawala-Sterniuk A, Bryniarska A, Sudol A, Podpora M, Gasz R, Martinek R, Kahankova Vilimkova R, Vilimek D, Pelc M, Mikolajewski D. Review of the application of the most current sophisticated image processing methods for the skin cancer diagnostics purposes. Arch Dermatol Res 2024; 316:99. [PMID: 38446274 DOI: 10.1007/s00403-024-02828-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 12/28/2023] [Accepted: 01/25/2024] [Indexed: 03/07/2024]
Abstract
This paper presents the most current and innovative solutions applying modern digital image processing methods for the purpose of skin cancer diagnostics. Skin cancer is one of the most common types of cancers. It is said that in the USA only, one in five people will develop skin cancer and this trend is constantly increasing. Implementation of new, non-invasive methods plays a crucial role in both identification and prevention of skin cancer occurrence. Early diagnosis and treatment are needed in order to decrease the number of deaths due to this disease. This paper also contains some information regarding the most common skin cancer types, mortality and epidemiological data for Poland, Europe, Canada and the USA. It also covers the most efficient and modern image recognition methods based on the artificial intelligence applied currently for diagnostics purposes. In this work, both professional, sophisticated as well as inexpensive solutions were presented. This paper is a review paper and covers the period of 2017 and 2022 when it comes to solutions and statistics. The authors decided to focus on the latest data, mostly due to the rapid technology development and increased number of new methods, which positively affects diagnosis and prognosis.
Collapse
Affiliation(s)
- Maria Myslicka
- Faculty of Medicine, Wroclaw Medical University, J. Mikulicza-Radeckiego 5, 50-345, Wroclaw, Poland.
| | - Aleksandra Kawala-Sterniuk
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland.
| | - Anna Bryniarska
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
| | - Adam Sudol
- Faculty of Natural Sciences and Technology, University of Opole, Dmowskiego 7-9, 45-368, Opole, Poland
| | - Michal Podpora
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
| | - Rafal Gasz
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
| | - Radek Martinek
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, Ostrava, 70800, Czech Republic
| | - Radana Kahankova Vilimkova
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, Ostrava, 70800, Czech Republic
| | - Dominik Vilimek
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, Ostrava, 70800, Czech Republic
| | - Mariusz Pelc
- Institute of Computer Science, University of Opole, Oleska 48, 45-052, Opole, Poland
- School of Computing and Mathematical Sciences, University of Greenwich, Old Royal Naval College, Park Row, SE10 9LS, London, UK
| | - Dariusz Mikolajewski
- Institute of Computer Science, Kazimierz Wielki University in Bydgoszcz, ul. Kopernika 1, 85-074, Bydgoszcz, Poland
- Neuropsychological Research Unit, 2nd Clinic of the Psychiatry and Psychiatric Rehabilitation, Medical University in Lublin, Gluska 1, 20-439, Lublin, Poland
| |
Collapse
|
|
7
|
Grignaffini F, Barbuto F, Troiano M, Piazzo L, Simeoni P, Mangini F, De Stefanis C, Onetti Muda A, Frezza F, Alisi A. The Use of Artificial Intelligence in the Liver Histopathology Field: A Systematic Review. Diagnostics (Basel) 2024; 14:388. [PMID: 38396427 PMCID: PMC10887838 DOI: 10.3390/diagnostics14040388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Revised: 02/02/2024] [Accepted: 02/06/2024] [Indexed: 02/25/2024] Open
Abstract
Digital pathology (DP) has begun to play a key role in the evaluation of liver specimens. Recent studies have shown that a workflow that combines DP and artificial intelligence (AI) applied to histopathology has potential value in supporting the diagnosis, treatment evaluation, and prognosis prediction of liver diseases. Here, we provide a systematic review of the use of this workflow in the field of hepatology. Based on the PRISMA 2020 criteria, a search of the PubMed, SCOPUS, and Embase electronic databases was conducted, applying inclusion/exclusion filters. The articles were evaluated by two independent reviewers, who extracted the specifications and objectives of each study, the AI tools used, and the results obtained. From the 266 initial records identified, 25 eligible studies were selected, mainly conducted on human liver tissues. Most of the studies were performed using whole-slide imaging systems for imaging acquisition and applying different machine learning and deep learning methods for image pre-processing, segmentation, feature extractions, and classification. Of note, most of the studies selected demonstrated good performance as classifiers of liver histological images compared to pathologist annotations. Promising results to date bode well for the not-too-distant inclusion of these techniques in clinical practice.
Collapse
Affiliation(s)
- Flavia Grignaffini
- Department of Information Engineering, Electronics and Telecommunications (DIET), “La Sapienza”, University of Rome, 00184 Rome, Italy; (F.G.); (F.B.); (L.P.); (F.M.); (F.F.)
| | - Francesco Barbuto
- Department of Information Engineering, Electronics and Telecommunications (DIET), “La Sapienza”, University of Rome, 00184 Rome, Italy; (F.G.); (F.B.); (L.P.); (F.M.); (F.F.)
| | - Maurizio Troiano
- Research Unit of Genetics of Complex Phenotypes, Bambino Gesù Children’s Hospital, IRCCS, 00165 Rome, Italy; (M.T.); (C.D.S.)
| | - Lorenzo Piazzo
- Department of Information Engineering, Electronics and Telecommunications (DIET), “La Sapienza”, University of Rome, 00184 Rome, Italy; (F.G.); (F.B.); (L.P.); (F.M.); (F.F.)
| | - Patrizio Simeoni
- National Transport Authority (NTA), D02 WT20 Dublin, Ireland;
- Faculty of Lifelong Learning, South East Technological University (SETU), R93 V960 Carlow, Ireland
| | - Fabio Mangini
- Department of Information Engineering, Electronics and Telecommunications (DIET), “La Sapienza”, University of Rome, 00184 Rome, Italy; (F.G.); (F.B.); (L.P.); (F.M.); (F.F.)
| | - Cristiano De Stefanis
- Research Unit of Genetics of Complex Phenotypes, Bambino Gesù Children’s Hospital, IRCCS, 00165 Rome, Italy; (M.T.); (C.D.S.)
| | | | - Fabrizio Frezza
- Department of Information Engineering, Electronics and Telecommunications (DIET), “La Sapienza”, University of Rome, 00184 Rome, Italy; (F.G.); (F.B.); (L.P.); (F.M.); (F.F.)
| | - Anna Alisi
- Research Unit of Genetics of Complex Phenotypes, Bambino Gesù Children’s Hospital, IRCCS, 00165 Rome, Italy; (M.T.); (C.D.S.)
| |
Collapse
|
|
8
|
Huang X, Fu F, Guo W, Kang D, Han X, Zheng L, Zhan Z, Wang C, Zhang Q, Wang S, Xu S, Ma J, Qiu L, Chen J, Li L. Prognostic significance of collagen signatures at breast tumor boundary obtained by combining multiphoton imaging and imaging analysis. Cell Oncol (Dordr) 2024; 47:69-80. [PMID: 37606817 DOI: 10.1007/s13402-023-00851-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/28/2023] [Indexed: 08/23/2023] Open
Abstract
PURPOSE Collagen features in breast tumor microenvironment is closely associated with the prognosis of patients. We aim to explore the prognostic significance of collagen features at breast tumor border by combining multiphoton imaging and imaging analysis. METHODS We used multiphoton microscopy (MPM) to label-freely image human breast tumor samples and then constructed an automatic classification model based on deep learning to identify collagen signatures from multiphoton images. We recognized three kinds of collagen signatures at tumor boundary (CSTB I-III) in a small-scale, and furthermore obtained a CSTB score for each patient based on the combined CSTB I-III by using the ridge regression analysis. The prognostic performance of CSTB score is assessed by the area under the receiver operating characteristic curve (AUC), Cox proportional hazard regression analysis, as well as Kaplan-Meier survival analysis. RESULTS As an independent prognostic factor, statistical results reveal that the prognostic performance of CSTB score is better than that of the clinical model combining three independent prognostic indicators, molecular subtype, tumor size, and lymph nodal metastasis (AUC, Training dataset: 0.773 vs. 0.749; External validation: 0.753 vs. 0.724; HR, Training dataset: 4.18 vs. 3.92; External validation: 4.98 vs. 4.16), and as an auxiliary indicator, it can greatly improve the accuracy of prognostic prediction. And furthermore, a nomogram combining the CSTB score with the clinical model is established for prognosis prediction and clinical decision making. CONCLUSION This standardized and automated imaging prognosticator may convince pathologists to adopt it as a prognostic factor, thereby customizing more effective treatment plans for patients.
Collapse
Affiliation(s)
- Xingxin Huang
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China
| | - Fangmeng Fu
- Department of Breast Surgery, Fujian Medical University Union Hospital, Fuzhou, 350001, China
| | - Wenhui Guo
- Department of Breast Surgery, Fujian Medical University Union Hospital, Fuzhou, 350001, China
| | - Deyong Kang
- Department of Pathology, Fujian Medical University Union Hospital, Fuzhou, 350001, China
| | - Xiahui Han
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China
| | - Liqin Zheng
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China
| | - Zhenlin Zhan
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China
| | - Chuan Wang
- Department of Breast Surgery, Fujian Medical University Union Hospital, Fuzhou, 350001, China
| | - Qingyuan Zhang
- Department of Medical Oncology, Harbin Medical University Cancer Hospital, Harbin, 150081, China
| | - Shu Wang
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China
- College of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, 350108, China
| | - Shunwu Xu
- School of Electronic and Mechanical Engineering, Fujian Polytechnic Normal University, Fuqing, 350300, China
| | - Jianli Ma
- Department of Radiation Oncology, Harbin Medical University Cancer Hospital, Harbin, 150081, China.
| | - Lida Qiu
- College of Physics and Electronic Information Engineering, Minjiang University, Fuzhou, 350108, China.
| | - Jianxin Chen
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China.
| | - Lianhuang Li
- Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350007, China.
| |
Collapse
|
|
9
|
Feng S, Wang J, Wang L, Qiu Q, Chen D, Su H, Li X, Xiao Y, Lin C. Current Status and Analysis of Machine Learning in Hepatocellular Carcinoma. J Clin Transl Hepatol 2023; 11:1184-1191. [PMID: 37577233 PMCID: PMC10412715 DOI: 10.14218/jcth.2022.00077s] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 12/11/2022] [Accepted: 02/21/2023] [Indexed: 07/03/2023] Open
Abstract
Hepatocellular carcinoma (HCC) is a common tumor. Although the diagnosis and treatment of HCC have made great progress, the overall prognosis remains poor. As the core component of artificial intelligence, machine learning (ML) has developed rapidly in the past decade. In particular, ML has become widely used in the medical field, and it has helped in the diagnosis and treatment of cancer. Different algorithms of ML have different roles in diagnosis, treatment, and prognosis. This article reviews recent research, explains the application of different ML models in HCC, and provides suggestions for follow-up research.
Collapse
Affiliation(s)
- Sijia Feng
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Jianhua Wang
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Liheng Wang
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Qixuan Qiu
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Dongdong Chen
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Huo Su
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Xiaoli Li
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Yao Xiao
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Chiayen Lin
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| |
Collapse
|
|
10
|
Stanciu SG, König K, Song YM, Wolf L, Charitidis CA, Bianchini P, Goetz M. Toward next-generation endoscopes integrating biomimetic video systems, nonlinear optical microscopy, and deep learning. BIOPHYSICS REVIEWS 2023; 4:021307. [PMID: 38510341 PMCID: PMC10903409 DOI: 10.1063/5.0133027] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Accepted: 05/26/2023] [Indexed: 03/22/2024]
Abstract
According to the World Health Organization, the proportion of the world's population over 60 years will approximately double by 2050. This progressive increase in the elderly population will lead to a dramatic growth of age-related diseases, resulting in tremendous pressure on the sustainability of healthcare systems globally. In this context, finding more efficient ways to address cancers, a set of diseases whose incidence is correlated with age, is of utmost importance. Prevention of cancers to decrease morbidity relies on the identification of precursor lesions before the onset of the disease, or at least diagnosis at an early stage. In this article, after briefly discussing some of the most prominent endoscopic approaches for gastric cancer diagnostics, we review relevant progress in three emerging technologies that have significant potential to play pivotal roles in next-generation endoscopy systems: biomimetic vision (with special focus on compound eye cameras), non-linear optical microscopies, and Deep Learning. Such systems are urgently needed to enhance the three major steps required for the successful diagnostics of gastrointestinal cancers: detection, characterization, and confirmation of suspicious lesions. In the final part, we discuss challenges that lie en route to translating these technologies to next-generation endoscopes that could enhance gastrointestinal imaging, and depict a possible configuration of a system capable of (i) biomimetic endoscopic vision enabling easier detection of lesions, (ii) label-free in vivo tissue characterization, and (iii) intelligently automated gastrointestinal cancer diagnostic.
Collapse
Affiliation(s)
- Stefan G. Stanciu
- Center for Microscopy-Microanalysis and Information Processing, University Politehnica of Bucharest, Bucharest, Romania
| | | | | | - Lior Wolf
- School of Computer Science, Tel Aviv University, Tel-Aviv, Israel
| | - Costas A. Charitidis
- Research Lab of Advanced, Composite, Nano-Materials and Nanotechnology, School of Chemical Engineering, National Technical University of Athens, Athens, Greece
| | - Paolo Bianchini
- Nanoscopy and NIC@IIT, Italian Institute of Technology, Genoa, Italy
| | - Martin Goetz
- Medizinische Klinik IV-Gastroenterologie/Onkologie, Kliniken Böblingen, Klinikverbund Südwest, Böblingen, Germany
| |
Collapse
|
|
11
|
Lei L, Du LX, He YL, Yuan JP, Wang P, Ye BL, Wang C, Hou Z. Dictionary learning LASSO for feature selection with application to hepatocellular carcinoma grading using contrast enhanced magnetic resonance imaging. Front Oncol 2023; 13:1123493. [PMID: 37091168 PMCID: PMC10118007 DOI: 10.3389/fonc.2023.1123493] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Accepted: 03/17/2023] [Indexed: 04/09/2023] Open
Abstract
IntroductionThe successful use of machine learning (ML) for medical diagnostic purposes has prompted myriad applications in cancer image analysis. Particularly for hepatocellular carcinoma (HCC) grading, there has been a surge of interest in ML-based selection of the discriminative features from high-dimensional magnetic resonance imaging (MRI) radiomics data. As one of the most commonly used ML-based selection methods, the least absolute shrinkage and selection operator (LASSO) has high discriminative power of the essential feature based on linear representation between input features and output labels. However, most LASSO methods directly explore the original training data rather than effectively exploiting the most informative features of radiomics data for HCC grading. To overcome this limitation, this study marks the first attempt to propose a feature selection method based on LASSO with dictionary learning, where a dictionary is learned from the training features, using the Fisher ratio to maximize the discriminative information in the feature.MethodsThis study proposes a LASSO method with dictionary learning to ensure the accuracy and discrimination of feature selection. Specifically, based on the Fisher ratio score, each radiomic feature is classified into two groups: the high-information and the low-information group. Then, a dictionary is learned through an optimal mapping matrix to enhance the high-information part and suppress the low discriminative information for the task of HCC grading. Finally, we select the most discrimination features according to the LASSO coefficients based on the learned dictionary.Results and discussionThe experimental results based on two classifiers (KNN and SVM) showed that the proposed method yielded accuracy gains, compared favorably with another 5 state-of-the-practice feature selection methods.
Collapse
Affiliation(s)
- Lei Lei
- College of Information Science and Engineering, Jiaxing University, Jiaxing, China
- *Correspondence: Lei Lei, ; Ying-Long He,
| | - Li-Xin Du
- Medical Imaging Department, Shenzhen Longhua District Central Hospital, Shenzhen, China
| | - Ying-Long He
- School of Mechanical Engineering Sciences, University of Surrey, Guildford, United Kingdom
- *Correspondence: Lei Lei, ; Ying-Long He,
| | - Jian-Peng Yuan
- Department of Radiology, The Seventh Affiliated Hospital, Sun Yat-sen University, Shenzhen, China
| | - Pan Wang
- Medical Imaging Department, Shenzhen Longhua District Central Hospital, Shenzhen, China
| | - Bao-Lin Ye
- College of Information Science and Engineering, Jiaxing University, Jiaxing, China
| | - Cong Wang
- Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China
| | - ZuJun Hou
- Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China
| |
Collapse
|
|
12
|
Dievernich A, Stegmaier J, Achenbach P, Warkentin S, Braunschweig T, Neumann UP, Klinge U. A Deep-Learning-Computed Cancer Score for the Identification of Human Hepatocellular Carcinoma Area Based on a Six-Colour Multiplex Immunofluorescence Panel. Cells 2023; 12:cells12071074. [PMID: 37048147 PMCID: PMC10093209 DOI: 10.3390/cells12071074] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 03/25/2023] [Accepted: 04/01/2023] [Indexed: 04/05/2023] Open
Abstract
Liver cancer is one of the most frequently diagnosed and fatal cancers worldwide, with hepatocellular carcinoma (HCC) being the most common primary liver cancer. Hundreds of studies involving thousands of patients have now been analysed across different cancer types, including HCC, regarding the effects of immune infiltrates on the prognosis of cancer patients. However, for these analyses, an unambiguous delineation of the cancer area is paramount, which is difficult due to the strong heterogeneity and considerable inter-operator variability induced by qualitative visual assessment and manual assignment. Nowadays, however, multiplex analyses allow the simultaneous evaluation of multiple protein markers, which, in conjunction with recent machine learning approaches, may offer great potential for the objective, enhanced identification of cancer areas with further in situ analysis of prognostic immune parameters. In this study, we, therefore, used an exemplary five-marker multiplex immunofluorescence panel of commonly studied markers for prognosis (CD3 T, CD4 T helper, CD8 cytotoxic T, FoxP3 regulatory T, and PD-L1) and DAPI to assess which analytical approach is best suited to combine morphological and immunohistochemical data into a cancer score to identify the cancer area that best matches an independent pathologist’s assignment. For each cell, a total of 68 individual cell features were determined, which were used as input for 4 different approaches for computing a cancer score: a correlation-based selection of individual cell features, a MANOVA-based selection of features, a multilayer perceptron, and a convolutional neural network (a U-net). Accuracy was used to evaluate performance. With a mean accuracy of 75%, the U-net was best capable of identifying the cancer area. Although individual cell features showed a strong heterogeneity between patients, the spatial representations obtained with the computed cancer scores delineate HCC well from non-cancer liver tissues. Future analyses with larger sample sizes will help to improve the model and enable direct, in-depth investigations of prognostic parameters, ultimately enabling precision medicine.
Collapse
Affiliation(s)
- Axel Dievernich
- Department of General, Visceral and Transplant Surgery, University Hospital RWTH Aachen, 52074 Aachen, Germany
- Forschungs-und Entwicklungsgesellschaft FEG Textiltechnik, 52070 Aachen, Germany
| | - Johannes Stegmaier
- Institute of Imaging and Computer Vision, RWTH Aachen University, 52074 Aachen, Germany
| | - Pascal Achenbach
- Department of Neurology, University Hospital RWTH Aachen, 52074 Aachen, Germany
- Institute of Neuropathology, University Hospital RWTH Aachen, 52074 Aachen, Germany
| | - Svetlana Warkentin
- Institute of Pathology, University Hospital RWTH Aachen, 52074 Aachen, Germany
| | - Till Braunschweig
- Institute of Pathology, University Hospital RWTH Aachen, 52074 Aachen, Germany
| | - Ulf Peter Neumann
- Department of General, Visceral and Transplant Surgery, University Hospital RWTH Aachen, 52074 Aachen, Germany
- Department of Surgery, Maastricht University Medical Center+, 6229 HX Maastricht, The Netherlands
| | - Uwe Klinge
- Department of General, Visceral and Transplant Surgery, University Hospital RWTH Aachen, 52074 Aachen, Germany
| |
Collapse
|
|
13
|
Galli R, Siciliano T, Aust D, Korn S, Kirsche K, Baretton GB, Weitz J, Koch E, Riediger C. Label-free multiphoton microscopy enables histopathological assessment of colorectal liver metastases and supports automated classification of neoplastic tissue. Sci Rep 2023; 13:4274. [PMID: 36922643 PMCID: PMC10017791 DOI: 10.1038/s41598-023-31401-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Accepted: 03/10/2023] [Indexed: 03/18/2023] Open
Abstract
As the state of resection margins is an important prognostic factor after extirpation of colorectal liver metastases, surgeons aim to obtain negative margins, sometimes elaborated by resections of the positive resection plane after intraoperative frozen sections. However, this is time consuming and results sometimes remain unclear during surgery. Label-free multimodal multiphoton microscopy (MPM) is an optical technique that retrieves morpho-chemical information avoiding all staining and that can potentially be performed in real-time. Here, we investigated colorectal liver metastases and hepatic tissue using a combination of three endogenous nonlinear signals, namely: coherent anti-Stokes Raman scattering (CARS) to visualize lipids, two-photon excited fluorescence (TPEF) to visualize cellular patterns, and second harmonic generation (SHG) to visualize collagen fibers. We acquired and analyzed over forty thousand MPM images of metastatic and normal liver tissue of 106 patients. The morphological information with biochemical specificity produced by MPM allowed discriminating normal liver from metastatic tissue and discerning the tumor borders on cryosections as well as formalin-fixed bulk tissue. Furthermore, automated tissue type classification with a correct rate close to 95% was possible using a simple approach based on discriminant analysis of texture parameters. Therefore, MPM has the potential to increase the precision of resection margins in hepatic surgery of metastases without prolonging surgical intervention.
Collapse
Affiliation(s)
- Roberta Galli
- Department of Medical Physics and Biomedical Engineering, Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Fetscherstr. 74, 01307, Dresden, Germany.
| | - Tiziana Siciliano
- Center for Regenerative Therapies (CRTD), Technische Universität Dresden, Fetscherstr. 105, 01307, Dresden, Germany
| | - Daniela Aust
- Institute of Pathology, University Hospital Carl Gustav Carus, Medical Faculty, Technische Universität Dresden, Fetscherstr. 74, 01307, Dresden, Germany.,National Center for Tumor Diseases (NCT/UCC), Partner Site Dresden: German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120, Heidelberg, Germany
| | - Sandra Korn
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus, Technische Universität Dresden, Fetscherstr. 74, 01307, Dresden, Germany
| | - Katrin Kirsche
- Neurosurgery, University Hospital Carl Gustav Carus, Technische Universität Dresden, Fetscherstr. 74, 01307, Dresden, Germany
| | - Gustavo B Baretton
- Institute of Pathology, University Hospital Carl Gustav Carus, Medical Faculty, Technische Universität Dresden, Fetscherstr. 74, 01307, Dresden, Germany.,National Center for Tumor Diseases (NCT/UCC), Partner Site Dresden: German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120, Heidelberg, Germany
| | - Jürgen Weitz
- National Center for Tumor Diseases (NCT/UCC), Partner Site Dresden: German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120, Heidelberg, Germany.,Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus, Technische Universität Dresden, Fetscherstr. 74, 01307, Dresden, Germany
| | - Edmund Koch
- Clinical Sensoring and Monitoring, Department of Anesthesiology and Intensive Care Medicine, Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Fetscherstr. 74, 01307, Dresden, Germany
| | - Carina Riediger
- National Center for Tumor Diseases (NCT/UCC), Partner Site Dresden: German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120, Heidelberg, Germany.,Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus, Technische Universität Dresden, Fetscherstr. 74, 01307, Dresden, Germany
| |
Collapse
|
|
14
|
Cinar U, Cetin Atalay R, Cetin YY. Human Hepatocellular Carcinoma Classification from H&E Stained Histopathology Images with 3D Convolutional Neural Networks and Focal Loss Function. J Imaging 2023; 9:jimaging9020025. [PMID: 36826944 PMCID: PMC9959324 DOI: 10.3390/jimaging9020025] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Revised: 01/13/2023] [Accepted: 01/18/2023] [Indexed: 01/26/2023] Open
Abstract
This paper proposes a new Hepatocellular Carcinoma (HCC) classification method utilizing a hyperspectral imaging system (HSI) integrated with a light microscope. Using our custom imaging system, we have captured 270 bands of hyperspectral images of healthy and cancer tissue samples with HCC diagnosis from a liver microarray slide. Convolutional Neural Networks with 3D convolutions (3D-CNN) have been used to build an accurate classification model. With the help of 3D convolutions, spectral and spatial features within the hyperspectral cube are incorporated to train a strong classifier. Unlike 2D convolutions, 3D convolutions take the spectral dimension into account while automatically collecting distinctive features during the CNN training stage. As a result, we have avoided manual feature engineering on hyperspectral data and proposed a compact method for HSI medical applications. Moreover, the focal loss function, utilized as a CNN cost function, enables our model to tackle the class imbalance problem residing in the dataset effectively. The focal loss function emphasizes the hard examples to learn and prevents overfitting due to the lack of inter-class balancing. Our empirical results demonstrate the superiority of hyperspectral data over RGB data for liver cancer tissue classification. We have observed that increased spectral dimension results in higher classification accuracy. Both spectral and spatial features are essential in training an accurate learner for cancer tissue classification.
Collapse
|
|
15
|
Classification of multi-differentiated liver cancer pathological images based on deep learning attention mechanism. BMC Med Inform Decis Mak 2022; 22:176. [PMID: 35787805 PMCID: PMC9254605 DOI: 10.1186/s12911-022-01919-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2022] [Accepted: 06/23/2022] [Indexed: 12/24/2022] Open
Abstract
PURPOSE Liver cancer is one of the most common malignant tumors in the world, ranking fifth in malignant tumors. The degree of differentiation can reflect the degree of malignancy. The degree of malignancy of liver cancer can be divided into three types: poorly differentiated, moderately differentiated, and well differentiated. Diagnosis and treatment of different levels of differentiation are crucial to the survival rate and survival time of patients. As the gold standard for liver cancer diagnosis, histopathological images can accurately distinguish liver cancers of different levels of differentiation. Therefore, the study of intelligent classification of histopathological images is of great significance to patients with liver cancer. At present, the classification of histopathological images of liver cancer with different degrees of differentiation has disadvantages such as time-consuming, labor-intensive, and large manual investment. In this context, the importance of intelligent classification of histopathological images is obvious. METHODS Based on the development of a complete data acquisition scheme, this paper applies the SENet deep learning model to the intelligent classification of all types of differentiated liver cancer histopathological images for the first time, and compares it with the four deep learning models of VGG16, ResNet50, ResNet_CBAM, and SKNet. The evaluation indexes adopted in this paper include confusion matrix, Precision, recall, F1 Score, etc. These evaluation indexes can be used to evaluate the model in a very comprehensive and accurate way. RESULTS Five different deep learning classification models are applied to collect the data set and evaluate model. The experimental results show that the SENet model has achieved the best classification effect with an accuracy of 95.27%. The model also has good reliability and generalization ability. The experiment proves that the SENet deep learning model has a good application prospect in the intelligent classification of histopathological images. CONCLUSIONS This study also proves that deep learning has great application value in solving the time-consuming and laborious problems existing in traditional manual film reading, and it has certain practical significance for the intelligent classification research of other cancer histopathological images.
Collapse
|
|
16
|
Dong X, Li M, Zhou P, Deng X, Li S, Zhao X, Wu Y, Qin J, Guo W. Fusing pre-trained convolutional neural networks features for multi-differentiated subtypes of liver cancer on histopathological images. BMC Med Inform Decis Mak 2022; 22:122. [PMID: 35509058 PMCID: PMC9066403 DOI: 10.1186/s12911-022-01798-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Accepted: 02/21/2022] [Indexed: 11/10/2022] Open
Abstract
Liver cancer is a malignant tumor with high morbidity and mortality, which has a tremendous negative impact on human survival. However, it is a challenging task to recognize tens of thousands of histopathological images of liver cancer by naked eye, which poses numerous challenges to inexperienced clinicians. In addition, factors such as long time-consuming, tedious work and huge number of images impose a great burden on clinical diagnosis. Therefore, our study combines convolutional neural networks with histopathology images and adopts a feature fusion approach to help clinicians efficiently discriminate the differentiation types of primary hepatocellular carcinoma histopathology images, thus improving their diagnostic efficiency and relieving their work pressure. In this study, for the first time, 73 patients with different differentiation types of primary liver cancer tumors were classified. We performed an adequate classification evaluation of liver cancer differentiation types using four pre-trained deep convolutional neural networks and nine different machine learning (ML) classifiers on a dataset of liver cancer histopathology images with multiple differentiation types. And the test set accuracy, validation set accuracy, running time with different strategies, precision, recall and F1 value were used for adequate comparative evaluation. Proved by experimental results, fusion networks (FuNet) structure is a good choice, which covers both channel attention and spatial attention, and suppresses channel interference with less information. Meanwhile, it can clarify the importance of each spatial location by learning the weights of different locations in space, then apply it to the study of classification of multi-differentiated types of liver cancer. In addition, in most cases, the Stacking-based integrated learning classifier outperforms other ML classifiers in the classification task of multi-differentiation types of liver cancer with the FuNet fusion strategy after dimensionality reduction of the fused features by principle component analysis (PCA) features, and a satisfactory result of 72.46% is achieved in the test set, which has certain practicality.
Collapse
Affiliation(s)
- Xiaogang Dong
- Department of Hepatopancreatobiliary Surgery, Cancer Affiliated Hospital of Xinjiang Medical University, Ürümqi, Xinjiang, China
| | - Min Li
- Key Laboratory of Signal Detection and Processing, Xinjiang University, Ürümqi, 830046, China.,College of Information Science and Engineering, Xinjiang University, Ürümqi, 830046, China
| | - Panyun Zhou
- College of Software, Xinjiang University, Ürümqi, 830046, China
| | - Xin Deng
- College of Software, Xinjiang University, Ürümqi, 830046, China
| | - Siyu Li
- College of Software, Xinjiang University, Ürümqi, 830046, China
| | - Xingyue Zhao
- College of Software, Xinjiang University, Ürümqi, 830046, China
| | - Yi Wu
- College of Software, Xinjiang University, Ürümqi, 830046, China
| | - Jiwei Qin
- College of Information Science and Engineering, Xinjiang University, Ürümqi, 830046, China.
| | - Wenjia Guo
- Cancer Institute, Affiliated Cancer Hospital of Xinjiang Medical University, Ürümqi, 830011, China. .,Key Laboratory of Oncology of Xinjiang Uyghur Autonomous Region, Ürümqi, 830011, China.
| |
Collapse
|
|
17
|
Chen WM, Fu M, Zhang CJ, Xing QQ, Zhou F, Lin MJ, Dong X, Huang J, Lin S, Hong MZ, Zheng QZ, Pan JS. Deep Learning-Based Universal Expert-Level Recognizing Pathological Images of Hepatocellular Carcinoma and Beyond. Front Med (Lausanne) 2022; 9:853261. [PMID: 35530044 PMCID: PMC9072864 DOI: 10.3389/fmed.2022.853261] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 03/30/2022] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND AND AIMS We aim to develop a diagnostic tool for pathological-image classification using transfer learning that can be applied to diverse tumor types. METHODS Microscopic images of liver tissue with and without hepatocellular carcinoma (HCC) were used to train and validate the classification framework based on a convolutional neural network. To evaluate the universal classification performance of the artificial intelligence (AI) framework, histological images from colorectal tissue and the breast were collected. Images for the training and validation sets were obtained from the Xiamen Hospital of Traditional Chinese Medicine, and those for the test set were collected from Zhongshan Hospital Xiamen University. The accuracy, sensitivity, and specificity values for the proposed framework were reported and compared with those of human image interpretation. RESULTS In the human-machine comparisons, the sensitivity, and specificity for the AI algorithm were 98.0, and 99.0%, whereas for the human experts, the sensitivity ranged between 86.0 and 97.0%, while the specificity ranged between 91.0 and 100%. Based on transfer learning, the accuracies of the AI framework in classifying colorectal carcinoma and breast invasive ductal carcinoma were 96.8 and 96.0%, respectively. CONCLUSION The performance of the proposed AI framework in classifying histological images with HCC was comparable to the classification performance achieved by human experts, indicating that extending the proposed AI's application to diagnoses and treatment recommendations is a promising area for future investigation.
Collapse
Affiliation(s)
- Wei-Ming Chen
- Liver Research Center, The First Affiliated Hospital of Fujian Medical University, Fuzhou, China
- School of Medicine, Xiamen University, Xiamen, China
| | - Min Fu
- School of Aerospace Engineering, Xiamen University, Xiamen, China
| | - Cheng-Ju Zhang
- Department of Anesthesiology, Zhongshan Hospital Xiamen University, Xiamen, China
| | - Qing-Qing Xing
- Liver Research Center, The First Affiliated Hospital of Fujian Medical University, Fuzhou, China
| | - Fei Zhou
- Department of Gastroenterology, Zhongshan Hospital Xiamen University, Xiamen, China
| | - Meng-Jie Lin
- Department of Pathology, Zhongshan Hospital Xiamen University, Xiamen, China
| | - Xuan Dong
- Liver Research Center, The First Affiliated Hospital of Fujian Medical University, Fuzhou, China
- School of Medicine, Xiamen University, Xiamen, China
| | - Jiaofeng Huang
- Liver Research Center, The First Affiliated Hospital of Fujian Medical University, Fuzhou, China
| | - Su Lin
- Liver Research Center, The First Affiliated Hospital of Fujian Medical University, Fuzhou, China
| | - Mei-Zhu Hong
- Department of Traditional Chinese Medicine, Mengchao Hepatobiliary Hospital of Fujian Medical University, Fuzhou, China
| | - Qi-Zhong Zheng
- Department of Pathology, Xiamen Hospital of Traditional Chinese Medicine, Xiamen, China
| | - Jin-Shui Pan
- Liver Research Center, The First Affiliated Hospital of Fujian Medical University, Fuzhou, China
| |
Collapse
|
|
18
|
Christou CD, Tsoulfas G. Role of three-dimensional printing and artificial intelligence in the management of hepatocellular carcinoma: Challenges and opportunities. World J Gastrointest Oncol 2022; 14:765-793. [PMID: 35582107 PMCID: PMC9048537 DOI: 10.4251/wjgo.v14.i4.765] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Revised: 08/24/2021] [Accepted: 03/27/2022] [Indexed: 02/06/2023] Open
Abstract
Hepatocellular carcinoma (HCC) constitutes the fifth most frequent malignancy worldwide and the third most frequent cause of cancer-related deaths. Currently, treatment selection is based on the stage of the disease. Emerging fields such as three-dimensional (3D) printing, 3D bioprinting, artificial intelligence (AI), and machine learning (ML) could lead to evidence-based, individualized management of HCC. In this review, we comprehensively report the current applications of 3D printing, 3D bioprinting, and AI/ML-based models in HCC management; we outline the significant challenges to the broad use of these novel technologies in the clinical setting with the goal of identifying means to overcome them, and finally, we discuss the opportunities that arise from these applications. Notably, regarding 3D printing and bioprinting-related challenges, we elaborate on cost and cost-effectiveness, cell sourcing, cell viability, safety, accessibility, regulation, and legal and ethical concerns. Similarly, regarding AI/ML-related challenges, we elaborate on intellectual property, liability, intrinsic biases, data protection, cybersecurity, ethical challenges, and transparency. Our findings show that AI and 3D printing applications in HCC management and healthcare, in general, are steadily expanding; thus, these technologies will be integrated into the clinical setting sooner or later. Therefore, we believe that physicians need to become familiar with these technologies and prepare to engage with them constructively.
Collapse
Affiliation(s)
- Chrysanthos D Christou
- Department of Transplantation Surgery, Hippokration General Hospital, School of Medicine, Aristotle University of Thessaloniki, Thessaloniki 54622, Greece
| | - Georgios Tsoulfas
- Department of Transplantation Surgery, Hippokration General Hospital, School of Medicine, Aristotle University of Thessaloniki, Thessaloniki 54622, Greece
| |
Collapse
|
|
19
|
Balsano C, Alisi A, Brunetto MR, Invernizzi P, Burra P, Piscaglia F. The application of artificial intelligence in hepatology: A systematic review. Dig Liver Dis 2022; 54:299-308. [PMID: 34266794 DOI: 10.1016/j.dld.2021.06.011] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Revised: 06/04/2021] [Accepted: 06/07/2021] [Indexed: 02/06/2023]
Abstract
The integration of human and artificial intelligence (AI) in medicine has only recently begun but it has already become obvious that intelligent systems can dramatically improve the management of liver diseases. Big data made it possible to envisage transformative developments of the use of AI for diagnosing, predicting prognosis and treating liver diseases, but there is still a lot of work to do. If we want to achieve the 21st century digital revolution, there is an urgent need for specific national and international rules, and to adhere to bioethical parameters when collecting data. Avoiding misleading results is essential for the effective use of AI. A crucial question is whether it is possible to sustain, technically and morally, the process of integration between man and machine. We present a systematic review on the applications of AI to hepatology, highlighting the current challenges and crucial issues related to the use of such technologies.
Collapse
Affiliation(s)
- Clara Balsano
- Dept. of Life, Health and Environmental Sciences MESVA, University of L'Aquila, Piazza S. Salvatore Tommasi 1, 67100, Coppito, L'Aquila. Italy; Francesco Balsano Foundation, Via Giovanni Battista Martini 6, 00198, Rome, Italy.
| | - Anna Alisi
- Research Unit of Molecular Genetics of Complex Phenotypes, Bambino Gesù Children's Hospital, IRCCS, Rome, Italy
| | - Maurizia R Brunetto
- Hepatology Unit and Laboratory of Molecular Genetics and Pathology of Hepatitis Viruses, University Hospital of Pisa, Pisa, Italy
| | - Pietro Invernizzi
- Division of Gastroenterology and Center of Autoimmune Liver Diseases, Department of Medicine and Surgery, San Gerardo Hospital, University of Milano, Bicocca, Italy
| | - Patrizia Burra
- Multivisceral Transplant Unit, Department of Surgery, Oncology, Gastroenterology, Padua University Hospital, Padua, Italy
| | - Fabio Piscaglia
- Division of Internal Medicine, IRCCS Azienda Ospedaliero Universitaria di Bologna, Bologna, Italy
| |
Collapse
|
|
20
|
Ahn JC, Qureshi TA, Singal AG, Li D, Yang JD. Deep learning in hepatocellular carcinoma: Current status and future perspectives. World J Hepatol 2021; 13:2039-2051. [PMID: 35070007 PMCID: PMC8727204 DOI: 10.4254/wjh.v13.i12.2039] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Revised: 07/19/2021] [Accepted: 11/15/2021] [Indexed: 02/06/2023] Open
Abstract
Hepatocellular carcinoma (HCC) is among the leading causes of cancer incidence and death. Despite decades of research and development of new treatment options, the overall outcomes of patients with HCC continue to remain poor. There are areas of unmet need in risk prediction, early diagnosis, accurate prognostication, and individualized treatments for patients with HCC. Recent years have seen an explosive growth in the application of artificial intelligence (AI) technology in medical research, with the field of HCC being no exception. Among the various AI-based machine learning algorithms, deep learning algorithms are considered state-of-the-art techniques for handling and processing complex multimodal data ranging from routine clinical variables to high-resolution medical images. This article will provide a comprehensive review of the recently published studies that have applied deep learning for risk prediction, diagnosis, prognostication, and treatment planning for patients with HCC.
Collapse
Affiliation(s)
- Joseph C Ahn
- Gastroenterology and Hepatology, Mayo Clinic, Rochester, MN 55904, United States
| | - Touseef Ahmad Qureshi
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States
| | - Amit G Singal
- Internal Medicine, University of Texas Southwestern Medical Center, Dallas, TX 75390, United States
| | - Debiao Li
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States
| | - Ju-Dong Yang
- Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States
| |
Collapse
|
|
21
|
Menegotto AB, Becker CDL, Cazella SC. Computer-aided diagnosis of hepatocellular carcinoma fusing imaging and structured health data. Health Inf Sci Syst 2021; 9:20. [PMID: 33968399 PMCID: PMC8096870 DOI: 10.1007/s13755-021-00151-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2020] [Accepted: 04/20/2021] [Indexed: 12/21/2022] Open
Abstract
INTRODUCTION Hepatocellular carcinoma is the prevalent primary liver cancer, a silent disease that killed 782,000 worldwide in 2018. Multimodal deep learning is the application of deep learning techniques, fusing more than one data modality as the model's input. PURPOSE A computer-aided diagnosis system for hepatocellular carcinoma developed with multimodal deep learning approaches could use multiple data modalities as recommended by clinical guidelines, and enhance the robustness and the value of the second-opinion given to physicians. This article describes the process of creation and evaluation of an algorithm for computer-aided diagnosis of hepatocellular carcinoma developed with multimodal deep learning techniques fusing preprocessed computed-tomography images with structured data from patient Electronic Health Records. RESULTS The classification performance achieved by the proposed algorithm in the test dataset was: accuracy = 86.9%, precision = 89.6%, recall = 86.9% and F-Score = 86.7%. These classification performance metrics are closer to the state-of-the-art in this area and were achieved with data modalities which are cheaper than traditional Magnetic Resonance Imaging approaches, enabling the use of the proposed algorithm by low and mid-sized healthcare institutions. CONCLUSION The classification performance achieved with the multimodal deep learning algorithm is higher than human specialists diagnostic performance using only CT for diagnosis. Even though the results are promising, the multimodal deep learning architecture used for hepatocellular carcinoma prediction needs more training and test processes using different datasets before the use of the proposed algorithm by physicians in real healthcare routines. The additional training aims to confirm the classification performance achieved and enhance the model's robustness.
Collapse
Affiliation(s)
- Alan Baronio Menegotto
- Universidade Federal de Ciências da Saúde de Porto Alegre, Rua Sarmento Leite, 245-Porto Alegre, Rio Grande do Sul, Brazil
| | - Carla Diniz Lopes Becker
- Universidade Federal de Ciências da Saúde de Porto Alegre, Rua Sarmento Leite, 245-Porto Alegre, Rio Grande do Sul, Brazil
| | - Silvio Cesar Cazella
- Universidade Federal de Ciências da Saúde de Porto Alegre, Rua Sarmento Leite, 245-Porto Alegre, Rio Grande do Sul, Brazil
| |
Collapse
|
|
22
|
Solovyev R, Kalinin AA, Gabruseva T. 3D convolutional neural networks for stalled brain capillary detection. Comput Biol Med 2021; 141:105089. [PMID: 34920160 DOI: 10.1016/j.compbiomed.2021.105089] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2021] [Revised: 11/26/2021] [Accepted: 11/26/2021] [Indexed: 01/08/2023]
Abstract
Adequate blood supply is critical for normal brain function. Brain vasculature dysfunctions, including stalled blood flow in cerebral capillaries, are associated with cognitive decline and pathogenesis in Alzheimer's disease. Recent advances in imaging technology enabled generation of high-quality 3D images that can be used to visualize stalled blood vessels. However, localization of stalled vessels in 3D images is often required as the first step for downstream analysis. When performed manually, this process is tedious, time-consuming, and error-prone. Here, we describe a deep learning-based approach for automatic detection of stalled capillaries in brain images based on 3D convolutional neural networks. Our approach includes custom 3D data augmentations and a weights transfer method that re-uses weights from 2D models pre-trained on natural images for initialization of 3D networks. We used an ensemble of several 3D models to produce the winning submission to the "Clog Loss: Advance Alzheimer's Research with Stall Catchers" machine learning competition that challenged the participants with classifying blood vessels in 3D image stacks as stalled or flowing. In this setting, our approach outperformed other methods and demonstrated state-of-the-art results, achieving 85% Matthews correlation coefficient, 85% sensitivity, and 99.3% specificity. The source code for our solution is publicly available.
Collapse
Affiliation(s)
- Roman Solovyev
- Institute for Design Problems in Microelectronics of Russian Academy of Sciences, 3, Sovetskaya Street, Moscow, 124 365, Russian Federation.
| | - Alexandr A Kalinin
- Shenzhen Research Institute of Big Data, Shenzhen, 518 172, Guangdong, China; Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI 48 109, USA
| | | |
Collapse
|
|
23
|
Hong D, Zheng YY, Xin Y, Sun L, Yang H, Lin MY, Liu C, Li BN, Zhang ZW, Zhuang J, Qian MY, Wang SS. Genetic syndromes screening by facial recognition technology: VGG-16 screening model construction and evaluation. Orphanet J Rare Dis 2021; 16:344. [PMID: 34344442 PMCID: PMC8336249 DOI: 10.1186/s13023-021-01979-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2021] [Accepted: 07/25/2021] [Indexed: 12/24/2022] Open
Abstract
Background Many genetic syndromes (GSs) have distinct facial dysmorphism, and facial gestalts can be used as a diagnostic tool for recognizing a syndrome. Facial recognition technology has advanced in recent years, and the screening of GSs by facial recognition technology has become feasible. This study constructed an automatic facial recognition model for the identification of children with GSs. Results A total of 456 frontal facial photos were collected from 228 children with GSs and 228 healthy children in Guangdong Provincial People's Hospital from Jun 2016 to Jan 2021. Only one frontal facial image was selected for each participant. The VGG-16 network (named after its proposal lab, Visual Geometry Group from Oxford University) was pretrained by transfer learning methods, and a facial recognition model based on the VGG-16 architecture was constructed. The performance of the VGG-16 model was evaluated by five-fold cross-validation. Comparison of VGG-16 model to five physicians were also performed. The VGG-16 model achieved the highest accuracy of 0.8860 ± 0.0211, specificity of 0.9124 ± 0.0308, recall of 0.8597 ± 0.0190, F1-score of 0.8829 ± 0.0215 and an area under the receiver operating characteristic curve of 0.9443 ± 0.0276 (95% confidence interval: 0.9210–0.9620) for GS screening, which was significantly higher than that achieved by human experts. Conclusions This study highlighted the feasibility of facial recognition technology for GSs identification. The VGG-16 recognition model can play a prominent role in GSs screening in clinical practice.
Collapse
Affiliation(s)
- Dian Hong
- Department of Paediatric Cardiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangdong Cardiovascular Institute, Guangdong Provincial Key Laboratory of South China Structural Heart Disease, Guangzhou, 510000, China
| | - Ying-Yi Zheng
- Cardiac Center, Guangdong Women and Children Hospital, Guangzhou, China
| | - Ying Xin
- Department of Paediatric Cardiology, Shenzhen Children's Hospital, Shenzhen, China
| | - Ling Sun
- Department of Paediatric Cardiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangdong Cardiovascular Institute, Guangdong Provincial Key Laboratory of South China Structural Heart Disease, Guangzhou, 510000, China
| | - Hang Yang
- Department of Paediatric Cardiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangdong Cardiovascular Institute, Guangdong Provincial Key Laboratory of South China Structural Heart Disease, Guangzhou, 510000, China
| | - Min-Yin Lin
- Department of Paediatric Cardiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangdong Cardiovascular Institute, Guangdong Provincial Key Laboratory of South China Structural Heart Disease, Guangzhou, 510000, China
| | - Cong Liu
- Department of Paediatric Cardiology, Shenzhen Children's Hospital, Shenzhen, China
| | - Bo-Ning Li
- Department of Paediatric Cardiology, Shenzhen Children's Hospital, Shenzhen, China
| | - Zhi-Wei Zhang
- Department of Paediatric Cardiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangdong Cardiovascular Institute, Guangdong Provincial Key Laboratory of South China Structural Heart Disease, Guangzhou, 510000, China
| | - Jian Zhuang
- Department of Cardiac Surgery, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangdong Cardiovascular Institute, Guangdong Provincial Key Laboratory of South China Structural Heart Disease, Guangzhou, China
| | - Ming-Yang Qian
- Department of Paediatric Cardiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangdong Cardiovascular Institute, Guangdong Provincial Key Laboratory of South China Structural Heart Disease, Guangzhou, 510000, China.
| | - Shu-Shui Wang
- Department of Paediatric Cardiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangdong Cardiovascular Institute, Guangdong Provincial Key Laboratory of South China Structural Heart Disease, Guangzhou, 510000, China.
| |
Collapse
|
|
24
|
Terradillos E, Saratxaga CL, Mattana S, Cicchi R, Pavone FS, Andraka N, Glover BJ, Arbide N, Velasco J, Etxezarraga MC, Picon A. Analysis on the Characterization of Multiphoton Microscopy Images for Malignant Neoplastic Colon Lesion Detection under Deep Learning Methods. J Pathol Inform 2021; 12:27. [PMID: 34447607 PMCID: PMC8359734 DOI: 10.4103/jpi.jpi_113_20] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Revised: 04/29/2021] [Accepted: 06/21/2021] [Indexed: 12/22/2022] Open
Abstract
BACKGROUND Colorectal cancer has a high incidence rate worldwide, with over 1.8 million new cases and 880,792 deaths in 2018. Fortunately, its early detection significantly increases the survival rate, reaching a cure rate of 90% when diagnosed at a localized stage. Colonoscopy is the gold standard technique for detection and removal of colorectal lesions with potential to evolve into cancer. When polyps are found in a patient, the current procedure is their complete removal. However, in this process, gastroenterologists cannot assure complete resection and clean margins which are given by the histopathology analysis of the removed tissue, which is performed at laboratory. AIMS In this paper, we demonstrate the capabilities of multiphoton microscopy (MPM) technology to provide imaging biomarkers that can be extracted by deep learning techniques to identify malignant neoplastic colon lesions and distinguish them from healthy, hyperplastic, or benign neoplastic tissue, without the need for histopathological staining. MATERIALS AND METHODS To this end, we present a novel MPM public dataset containing 14,712 images obtained from 42 patients and grouped into 2 classes. A convolutional neural network is trained on this dataset and a spatially coherent predictions scheme is applied for performance improvement. RESULTS We obtained a sensitivity of 0.8228 ± 0.1575 and a specificity of 0.9114 ± 0.0814 on detecting malignant neoplastic lesions. We also validated this approach to estimate the self-confidence of the network on its own predictions, obtaining a mean sensitivity of 0.8697 and a mean specificity of 0.9524 with the 18.67% of the images classified as uncertain. CONCLUSIONS This work lays the foundations for performing in vivo optical colon biopsies by combining this novel imaging technology together with deep learning algorithms, hence avoiding unnecessary polyp resection and allowing in situ diagnosis assessment.
Collapse
Affiliation(s)
| | | | - Sara Mattana
- European Laboratory for Non-Linear Spectroscopy, Sesto Fiorentino, Italy
| | - Riccardo Cicchi
- European Laboratory for Non-Linear Spectroscopy, Sesto Fiorentino, Italy
| | | | - Nagore Andraka
- Basque Foundation for Health Innovation and Research, Barakaldo, Spain
| | | | - Nagore Arbide
- Department of Pathological Anatomy, Osakidetza Basque Health Service, Basurto University Hospital, Bilbao, Spain
| | - Jacques Velasco
- Department of Pathological Anatomy, Osakidetza Basque Health Service, Basurto University Hospital, Bilbao, Spain
| | - Mª Carmen Etxezarraga
- Department of Pathological Anatomy, Osakidetza Basque Health Service, Basurto University Hospital, Bilbao, Spain
| | - Artzai Picon
- University of the Basque Country UPV/EHU, Bilbao, Spain
| |
Collapse
|
|
25
|
Jain D, Torres R, Celli R, Koelmel J, Charkoftaki G, Vasiliou V. Evolution of the liver biopsy and its future. Transl Gastroenterol Hepatol 2021; 6:20. [PMID: 33824924 PMCID: PMC7829074 DOI: 10.21037/tgh.2020.04.01] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/12/2020] [Accepted: 03/19/2020] [Indexed: 12/12/2022] Open
Abstract
Liver biopsies are commonly used to evaluate a wide variety of medical disorders, including neoplasms and post-transplant complications. However, its use is being impacted by improved clinical diagnosis of disorders, and non-invasive methods for evaluating liver tissue and as a result the indications of a liver biopsy have undergone major changes in the last decade. The evolution of highly effective treatments for some of the common indications for liver biopsy in the last decade (e.g., viral hepatitis B and C) has led to a decline in the number of liver biopsies in recent years. At the same time, the emergence of better technologies for histologic evaluation, tissue content analysis and genomics are among the many new and exciting developments in the field that hold great promise for the future and are going to shape the indications for a liver biopsy in the future. Recent advances in slide scanners now allow creation of "digital/virtual" slides that have image of the entire tissue section present in a slide [whole slide imaging (WSI)]. WSI can now be done very rapidly and at very high resolution, allowing its use in routine clinical practice. In addition, a variety of technologies have been developed in recent years that use different light sources and/or microscopes allowing visualization of tissues in a completely different way. One such technique that is applicable to liver specimens combines multiphoton microscopy (MPM) with advanced clearing and fluorescent stains known as Clearing Histology with MultiPhoton Microscopy (CHiMP). Although it has not yet been extensively validated, the technique has the potential to decrease inefficiency, reduce artifacts, and increase data while being readily integrable into clinical workflows. Another technology that can provide rapid and in-depth characterization of thousands of molecules in a tissue sample, including liver tissues, is matrix assisted laser desorption/ionization (MALDI) mass spectrometry. MALDI has already been applied in a clinical research setting with promising diagnostic and prognostic capabilities, as well as being able to elucidate mechanisms of liver diseases that may be targeted for the development of new therapies. The logical next step in huge data sets obtained from such advanced analysis of liver tissues is the application of machine learning (ML) algorithms and application of artificial intelligence (AI), for automated generation of diagnoses and prognoses. This review discusses the evolving role of liver biopsies in clinical practice over the decades, and describes newer technologies that are likely to have a significant impact on how they will be used in the future.
Collapse
Affiliation(s)
- Dhanpat Jain
- Department of Anatomic Pathology, Yale University School of Medicine, New Haven, CT, USA
| | - Richard Torres
- Department of Laboratory Medicine, Yale University School of Medicine, New Haven, CT, USA
| | - Romulo Celli
- Department of Anatomic Pathology, Yale University School of Medicine, New Haven, CT, USA
| | - Jeremy Koelmel
- Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA
| | - Georgia Charkoftaki
- Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA
| | - Vasilis Vasiliou
- Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA
| |
Collapse
|
|
26
|
Zherebtsov E, Zajnulina M, Kandurova K, Potapova E, Dremin V, Mamoshin A, Sokolovski S, Dunaev A, Rafailov EU. Machine Learning Aided Photonic Diagnostic System for Minimally Invasive Optically Guided Surgery in the Hepatoduodenal Area. Diagnostics (Basel) 2020; 10:E873. [PMID: 33121013 PMCID: PMC7693603 DOI: 10.3390/diagnostics10110873] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Revised: 10/19/2020] [Accepted: 10/24/2020] [Indexed: 12/29/2022] Open
Abstract
Abdominal cancer is a widely prevalent group of tumours with a high level of mortality if diagnosed at a late stage. Although the cancer death rates have in general declined over the past few decades, the mortality from tumours in the hepatoduodenal area has significantly increased in recent years. The broader use of minimal access surgery (MAS) for diagnostics and treatment can significantly improve the survival rate and quality of life of patients after surgery. This work aims to develop and characterise an appropriate technical implementation for tissue endogenous fluorescence (TEF) and assess the efficiency of machine learning methods for the real-time diagnosis of tumours in the hepatoduodenal area. In this paper, we present the results of the machine learning approach applied to the optically guided MAS. We have elaborated tissue fluorescence approach with a fibre-optic probe to record the TEF and blood perfusion parameters during MAS in patients with cancers in the hepatoduodenal area. The measurements from the laser Doppler flowmetry (LDF) channel were used as a sensor of the tissue vitality to reduce variability in TEF data. Also, we evaluated how the blood perfusion oscillations are changed in the tumour tissue. The evaluated amplitudes of the cardiac (0.6-1.6 Hz) and respiratory (0.2-0.6 Hz) oscillations was significantly higher in intact tissues (p < 0.001) compared to the cancerous ones, while the myogenic (0.2-0.06 Hz) oscillation did not demonstrate any statistically significant difference. Our results demonstrate that a fibre-optic TEF probe accompanied with ML algorithms such as k-Nearest Neighbours or AdaBoost is highly promising for the real-time in situ differentiation between cancerous and healthy tissues by detecting the information about the tissue type that is encoded in the fluorescence spectrum. Also, we show that the detection can be supplemented and enhanced by parallel collection and classification of blood perfusion oscillations.
Collapse
Affiliation(s)
- Evgeny Zherebtsov
- Research and Development Center of Biomedical Photonics, Orel State University, 302026 Orel, Russia; (K.K.); (E.P.); (V.D.); (A.M.); (A.D.)
- Faculty of Information Technology and Electrical Engineering, University of Oulu, Optoelectronics and Measurement Techniques Unit, 90570 Oulu, Finland
| | - Marina Zajnulina
- Aston Institute of Photonic Technologies, Aston University, Birmingham B4 7ET, UK; (M.Z.); (S.S.); (E.U.R.)
| | - Ksenia Kandurova
- Research and Development Center of Biomedical Photonics, Orel State University, 302026 Orel, Russia; (K.K.); (E.P.); (V.D.); (A.M.); (A.D.)
| | - Elena Potapova
- Research and Development Center of Biomedical Photonics, Orel State University, 302026 Orel, Russia; (K.K.); (E.P.); (V.D.); (A.M.); (A.D.)
| | - Viktor Dremin
- Research and Development Center of Biomedical Photonics, Orel State University, 302026 Orel, Russia; (K.K.); (E.P.); (V.D.); (A.M.); (A.D.)
- Aston Institute of Photonic Technologies, Aston University, Birmingham B4 7ET, UK; (M.Z.); (S.S.); (E.U.R.)
| | - Andrian Mamoshin
- Research and Development Center of Biomedical Photonics, Orel State University, 302026 Orel, Russia; (K.K.); (E.P.); (V.D.); (A.M.); (A.D.)
- Department of X-ray Surgical Methods of Diagnosis and Treatment, Orel Regional Clinical Hospital, 302028 Orel, Russia
| | - Sergei Sokolovski
- Aston Institute of Photonic Technologies, Aston University, Birmingham B4 7ET, UK; (M.Z.); (S.S.); (E.U.R.)
| | - Andrey Dunaev
- Research and Development Center of Biomedical Photonics, Orel State University, 302026 Orel, Russia; (K.K.); (E.P.); (V.D.); (A.M.); (A.D.)
| | - Edik U. Rafailov
- Aston Institute of Photonic Technologies, Aston University, Birmingham B4 7ET, UK; (M.Z.); (S.S.); (E.U.R.)
| |
Collapse
|
|
27
|
Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning. NPJ Precis Oncol 2020; 4:14. [PMID: 32550270 PMCID: PMC7280520 DOI: 10.1038/s41698-020-0120-3] [Citation(s) in RCA: 107] [Impact Index Per Article: 21.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2020] [Accepted: 05/07/2020] [Indexed: 12/24/2022] Open
Abstract
Hepatocellular carcinoma (HCC) is the most common subtype of liver cancer, and assessing its histopathological grade requires visual inspection by an experienced pathologist. In this study, the histopathological H&E images from the Genomic Data Commons Databases were used to train a neural network (inception V3) for automatic classification. According to the evaluation of our model by the Matthews correlation coefficient, the performance level was close to the ability of a 5-year experience pathologist, with 96.0% accuracy for benign and malignant classification, and 89.6% accuracy for well, moderate, and poor tumor differentiation. Furthermore, the model was trained to predict the ten most common and prognostic mutated genes in HCC. We found that four of them, including CTNNB1, FMN2, TP53, and ZFX4, could be predicted from histopathology images, with external AUCs from 0.71 to 0.89. The findings demonstrated that convolutional neural networks could be used to assist pathologists in the classification and detection of gene mutation in liver cancer.
Collapse
|
|
28
|
Pradhan P, Guo S, Ryabchykov O, Popp J, Bocklitz TW. Deep learning a boon for biophotonics? JOURNAL OF BIOPHOTONICS 2020; 13:e201960186. [PMID: 32167235 DOI: 10.1002/jbio.201960186] [Citation(s) in RCA: 51] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/15/2019] [Revised: 02/22/2020] [Accepted: 03/10/2020] [Indexed: 06/10/2023]
Abstract
This review covers original articles using deep learning in the biophotonic field published in the last years. In these years deep learning, which is a subset of machine learning mostly based on artificial neural network geometries, was applied to a number of biophotonic tasks and has achieved state-of-the-art performances. Therefore, deep learning in the biophotonic field is rapidly growing and it will be utilized in the next years to obtain real-time biophotonic decision-making systems and to analyze biophotonic data in general. In this contribution, we discuss the possibilities of deep learning in the biophotonic field including image classification, segmentation, registration, pseudostaining and resolution enhancement. Additionally, we discuss the potential use of deep learning for spectroscopic data including spectral data preprocessing and spectral classification. We conclude this review by addressing the potential applications and challenges of using deep learning for biophotonic data.
Collapse
Affiliation(s)
- Pranita Pradhan
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| | - Shuxia Guo
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| | - Oleg Ryabchykov
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| | - Juergen Popp
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| | - Thomas W Bocklitz
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| |
Collapse
|
|
29
|
Cai S, Tian Y, Lui H, Zeng H, Wu Y, Chen G. Dense-UNet: a novel multiphoton in vivo cellular image segmentation model based on a convolutional neural network. Quant Imaging Med Surg 2020; 10:1275-1285. [PMID: 32550136 DOI: 10.21037/qims-19-1090] [Citation(s) in RCA: 74] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
Background Multiphoton microscopy (MPM) offers a feasible approach for the biopsy in clinical medicine, but it has not been used in clinical applications due to the lack of efficient image processing methods, especially the automatic segmentation technology. Segmentation technology is still one of the most challenging assignments of the MPM imaging technique. Methods The MPM imaging segmentation model based on deep learning is one of the most effective methods to address this problem. In this paper, the practicability of using a convolutional neural network (CNN) model to segment the MPM image of skin cells in vivo was explored. A set of MPM in vivo skin cells images with a resolution of 128×128 was successfully segmented under the Python environment with TensorFlow. A novel deep-learning segmentation model named Dense-UNet was proposed. The Dense-UNet, which is based on U-net structure, employed the dense concatenation to deepen the depth of the network architecture and achieve feature reuse. This model included four expansion modules (each module consisted of four down-sampling layers) to extract features. Results Sixty training images were taken from the dorsal forearm using a femtosecond Ti:Sa laser running at 735 nm. The resolution of the images is 128×128 pixels. Experimental results confirmed that the accuracy of Dense-UNet (92.54%) was higher than that of U-Net (88.59%), with a significantly lower loss value of 0.1681. The 90.60% Dice coefficient value of Dense-UNet outperformed U-Net by 11.07%. The F1-Score of Dense-UNet, U-Net, and Seg-Net was 93.35%, 90.02%, and 85.04%, respectively. Conclusions The deepened down-sampling path improved the ability of the model to capture cellular fined-detailed boundary features, while the symmetrical up-sampling path provided a more accurate location based on the test result. These results were the first time that the segmentation of MPM in vivo images had been adopted by introducing a deep CNN to bridge this gap in Dense-UNet technology. Dense-UNet has reached ultramodern performance for MPM images, especially for in vivo images with low resolution. This implementation supplies an automatic segmentation model based on deep learning for high-precision segmentation of MPM images in vivo.
Collapse
Affiliation(s)
- Sijing Cai
- Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, China.,School of Information Science and Engineering, Fujian University of Technology, Fuzhou, China
| | - Yunxian Tian
- Imaging Unit, Integrative Oncology Department, BC Cancer Research Centre, Vancouver, BC, Canada.,Photomedicine Institute, Department of Dermatology and Skin Science, University of British Columbia and Vancouver Coastal Health Research Institute, Vancouver, BC, Canada
| | - Harvey Lui
- Imaging Unit, Integrative Oncology Department, BC Cancer Research Centre, Vancouver, BC, Canada.,Photomedicine Institute, Department of Dermatology and Skin Science, University of British Columbia and Vancouver Coastal Health Research Institute, Vancouver, BC, Canada
| | - Haishan Zeng
- Imaging Unit, Integrative Oncology Department, BC Cancer Research Centre, Vancouver, BC, Canada.,Photomedicine Institute, Department of Dermatology and Skin Science, University of British Columbia and Vancouver Coastal Health Research Institute, Vancouver, BC, Canada
| | - Yi Wu
- Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, China.,Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou, China
| | - Guannan Chen
- Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, China.,Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou, China
| |
Collapse
|
|
30
|
Huttunen MJ, Hristu R, Dumitru A, Floroiu I, Costache M, Stanciu SG. Multiphoton microscopy of the dermoepidermal junction and automated identification of dysplastic tissues with deep learning. BIOMEDICAL OPTICS EXPRESS 2020; 11:186-199. [PMID: 32010509 PMCID: PMC6968761 DOI: 10.1364/boe.11.000186] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/02/2019] [Revised: 10/24/2019] [Accepted: 11/05/2019] [Indexed: 05/05/2023]
Abstract
Histopathological image analysis performed by a trained expert is currently regarded as the gold-standard for the diagnostics of many pathologies, including cancers. However, such approaches are laborious, time consuming and contain a risk for bias or human error. There is thus a clear need for faster, less intrusive and more accurate diagnostic solutions, requiring also minimal human intervention. Multiphoton microscopy (MPM) can alleviate some of the drawbacks specific to traditional histopathology by exploiting various endogenous optical signals to provide virtual biopsies that reflect the architecture and composition of tissues, both in-vivo or ex-vivo. Here we show that MPM imaging of the dermoepidermal junction (DEJ) in unstained fixed tissues provides useful cues for a histopathologist to identify the onset of non-melanoma skin cancers. Furthermore, we show that MPM images collected on the DEJ, besides being easy to interpret by a trained specialist, can be automatically classified into healthy and dysplastic classes with high precision using a Deep Learning method and existing pre-trained convolutional neural networks. Our results suggest that deep learning enhanced MPM for in-vivo skin cancer screening could facilitate timely diagnosis and intervention, enabling thus more optimal therapeutic approaches.
Collapse
Affiliation(s)
- Mikko J. Huttunen
- Photonics Laboratory, Physics Unit, Tampere University, Tampere, Finland
- These authors contributed equally to this work
| | - Radu Hristu
- Center for Microscopy-Microanalysis and Information Processing, Politehnica University of Bucharest, Bucharest, Romania
- These authors contributed equally to this work
| | - Adrian Dumitru
- Department of Pathology, Carol Davila University of Medicine and Pharmacy, Bucharest, Romania
- These authors contributed equally to this work
| | - Iustin Floroiu
- Center for Microscopy-Microanalysis and Information Processing, Politehnica University of Bucharest, Bucharest, Romania
- Faculty of Medical Engineering, Politehnica University of Bucharest, Bucharest, Romania
| | - Mariana Costache
- Department of Pathology, Carol Davila University of Medicine and Pharmacy, Bucharest, Romania
| | - Stefan G. Stanciu
- Center for Microscopy-Microanalysis and Information Processing, Politehnica University of Bucharest, Bucharest, Romania
| |
Collapse
|
|
31
|
Deep-UV excitation fluorescence microscopy for detection of lymph node metastasis using deep neural network. Sci Rep 2019; 9:16912. [PMID: 31729459 PMCID: PMC6858352 DOI: 10.1038/s41598-019-53405-w] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2019] [Accepted: 10/31/2019] [Indexed: 02/07/2023] Open
Abstract
Deep-UV (DUV) excitation fluorescence microscopy has potential to provide rapid diagnosis with simple technique comparing to conventional histopathology based on hematoxylin and eosin (H&E) staining. We established a fluorescent staining protocol for DUV excitation fluorescence imaging that has enabled clear discrimination of nucleoplasm, nucleolus, and cytoplasm. Fluorescence images of metastasis-positive/-negative lymph nodes of gastric cancer patients were used for patch-based training with a deep neural network (DNN) based on Inception-v3 architecture. The performance on small patches of the fluorescence images was comparable with that of H&E images. Gradient-weighted class activation mapping analysis revealed the areas where the trained model identified metastatic lesions in the images containing cancer cells. We extended the method to large-size image analysis enabling accurate detection of metastatic lesions. We discuss usefulness of DUV excitation fluorescence imaging with the aid of DNN analysis, which is promising for assisting pathologists in assessment of lymph node metastasis.
Collapse
|
|
32
|
Lin H, Fan T, Sui J, Wang G, Chen J, Zhuo S, Zhang H. Recent advances in multiphoton microscopy combined with nanomaterials in the field of disease evolution and clinical applications to liver cancer. NANOSCALE 2019; 11:19619-19635. [PMID: 31599299 DOI: 10.1039/c9nr04902a] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Multiphoton microscopy (MPM) is expected to become a powerful clinical tool, with its unique advantages of being label-free, high resolution, deep imaging depth, low light photobleaching and low phototoxicity. Nanomaterials, with excellent physical and chemical properties, are biocompatible and easy to prepare and functionalize. The addition of nanomaterials exactly compensates for some defects of MPM, such as the weak endogenous signal strength, limited imaging materials, insufficient imaging depth and lack of therapeutic effects. Therefore, combining MPM with nanomaterials is a promising biomedical imaging method. Here, we mainly review the principle of MPM and its application in liver cancer, especially in disease evolution and clinical applications, including monitoring tumor progression, diagnosing tumor occurrence, detecting tumor metastasis, and evaluating cancer therapy response. Then, we introduce the latest advances in the combination of MPM with nanomaterials, including the MPM imaging of gold nanoparticles (AuNPs) and carbon dots (CDs). Finally, we also propose the main challenges and future research directions of MPM technology in HCC.
Collapse
Affiliation(s)
- Hongxin Lin
- Fujian Normal University, Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fuzhou, 350007, China.
| | - Taojian Fan
- Shenzhen Engineering Laboratory of Phosphorene and Optoelectronics and Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province, Shenzhen University, Shenzhen, 518060, China.
| | - Jian Sui
- Department of Gastrointestinal surgery, Fujian Provincial Hospital, Fuzhou, 350000, China
| | - Guangxing Wang
- Fujian Normal University, Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fuzhou, 350007, China.
| | - Jianxin Chen
- Fujian Normal University, Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fuzhou, 350007, China.
| | - Shuangmu Zhuo
- Fujian Normal University, Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fuzhou, 350007, China.
| | - Han Zhang
- Shenzhen Engineering Laboratory of Phosphorene and Optoelectronics and Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province, Shenzhen University, Shenzhen, 518060, China.
| |
Collapse
|
|
33
|
König TT, Goedeke J, Muensterer OJ. Multiphoton microscopy in surgical oncology- a systematic review and guide for clinical translatability. Surg Oncol 2019; 31:119-131. [PMID: 31654957 DOI: 10.1016/j.suronc.2019.10.011] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2019] [Revised: 10/02/2019] [Accepted: 10/13/2019] [Indexed: 02/07/2023]
Abstract
BACKGROUND Multiphoton microscopy (MPM) facilitates three-dimensional, high-resolution functional imaging of unlabeled tissues in vivo and ex vivo. This systematic review discusses the diagnostic value, advantages and challenges in the practical use of MPM in surgical oncology. METHOD AND FINDINGS A Medline search was conducted in April 2019. Fifty-three original research papers investigating MPM compared to standard histology in human patients with solid tumors were identified. A qualitative synopsis and meta-analysis of 14 blinded studies was performed. Risk of bias and applicability were evaluated. MPM can image fresh, frozen or fixed tissues up to a depth 1000 μm in the z-plane. Best results including functional imaging and virtual histochemistry are obtained by in vivo imaging or scanning fresh tissue immediately after excision. Two-photon excited fluorescence by natural fluorophores of the cytoplasm and second harmonic generation signals by fluorophores of the extracellular matrix can be scanned simultaneously, providing high resolution optical histochemistry comparable to standard histology. Functional parameters like fluorescence lifetime imaging or optical redox ratio provide additional objective information. A major concern is inability to visualize the nucleus. However, in a subpopulation analysis of 440 specimens, MPM yielded a sensitivity of 94%, specificity of 96% and accuracy of 95% for the detection of malignant tissue. CONCLUSION MPM is a promising emerging technique in surgical oncology. Ex vivo imaging has high sensitivity, specificity and accuracy for the detection of tumor cells. For broad clinical application in vivo, technical challenges need to be resolved.
Collapse
Affiliation(s)
| | - Jan Goedeke
- Universitätsmedizin Mainz, Department of Pediatric Surgery, Mainz, Germany
| | | |
Collapse
|
|
34
|
Wang R, He Y, Yao C, Wang S, Xue Y, Zhang Z, Wang J, Liu X. Classification and Segmentation of Hyperspectral Data of Hepatocellular Carcinoma Samples Using 1-D Convolutional Neural Network. Cytometry A 2019; 97:31-38. [PMID: 31403260 DOI: 10.1002/cyto.a.23871] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2019] [Revised: 07/16/2019] [Accepted: 07/19/2019] [Indexed: 12/24/2022]
Abstract
Pathological diagnosis plays an important role in the diagnosis and treatment of hepatocellular carcinoma (HCC). The traditional method of pathological diagnosis of most cancers requires freezing, slicing, hematoxylin and eosin staining, and manual analysis, limiting the speed of the diagnosis process. In this study, we designed a one-dimensional convolutional neural network to classify the hyperspectral data of HCC sample slices acquired by our hyperspectral imaging system. A weighted loss function was employed to promote the performance of the model. The proposed method allows us to accelerate the diagnosis process of identifying tumor tissues. Our deep learning model achieved good performance on our data set with sensitivity, specificity, and area under receiver operating characteristic curve of 0.871, 0.888, and 0.950, respectively. Meanwhile, our deep learning model outperformed the other machine learning methods assessed on our data set. The proposed method is a potential tool for the label-free and real-time pathologic diagnosis. © 2019 International Society for Advancement of Cytometry.
Collapse
Affiliation(s)
- Rendong Wang
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an 710049, China
| | - Yida He
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an 710049, China
| | - Cuiping Yao
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an 710049, China
| | - Sijia Wang
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an 710049, China
| | - Yuan Xue
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an 710049, China
| | - Zhenxi Zhang
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an 710049, China
| | - Jing Wang
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an 710049, China
| | - Xiaolong Liu
- The United Innovation of Mengchao Hepatobiliary Technology Key Laboratory of Fujian Province, Mengchao Hepatobiliary Hospital of Fujian Medical University, Fuzhou, 350025, People's Republic of China
| |
Collapse
|