|
1
|
Han K, Li C, Xiao A, Tian Y, Tian J, Hu Z. TSPE: Reconstruction of multi-morphological tumors of NIR-II fluorescence molecular tomography based on positional encoding. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2025; 261:108554. [PMID: 39889498 DOI: 10.1016/j.cmpb.2024.108554] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/22/2024] [Revised: 11/25/2024] [Accepted: 12/04/2024] [Indexed: 02/03/2025]
Abstract
BACKGROUND AND OBJECTIVE Fluorescence molecular tomography (FMT) is a noninvasive and highly sensitive imaging modality, which can display 3D visualization of tumors by reconstructing fluorescence probes' distribution. However, existing methods mostly ignore positional information, which includes spatial structure information crucial for the reconstruction of light sources. This limits the reconstruction accuracy of light sources with multiple morphologies. Therefore, to our best knowledge, we for the first time integrated positional encoding into the FMT task, enabling the incorporation of high-frequency spatial structure information. METHODS We proposed a three-stage network embedded with a positional encoding module (TSPE) to perform high reconstruction accuracy of tumors with multiple morphologies. Additionally, our study focused on NIR-II which had less severe scattering problems and higher imaging accuracy than NIR-I. RESULTS The simulation experiments demonstrated that TSPE achieved high reconstruction accuracy in NIR-II FMT, with the barycenter error (BCE) for single-tumor reconstruction reaching 0.18 mm, representing a 14 % reduction compared to other methods. TSPE more accurately distinguished adjacent multi-morphological tumors with a minimal edge-to-edge distance (EED) of 0.3 mm. In vivo experiments also showed that TSPE could achieve more accurate reconstruction of tumors compared with other methods. CONCLUSIONS The proposed method can achieve the best reconstruction performance. It has potential to promote the development of NIR-II FMT and its preclinical application.
Collapse
Affiliation(s)
- Keyi Han
- CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Chunzhao Li
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China; China National Clinical Research Center for Neurological Diseases, Beijing Tiantan Hospital, Beijing 100070, China
| | - Anqi Xiao
- CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Yaqi Tian
- Department of Radiology, Beijing Friendship Hospital, Capital Medical University, Beijing, 100050, China
| | - Jie Tian
- CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China; Beijing Advanced Innovation Center for Big Data-based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing 100191, China; Enginering Research Center of Molecular and Neuro Imaging of Ministry of Education, School of Life Science and Technology, Xidian University, Xi'an 710071, China; National Key Laboratory of Kidney Diseases, Beijing, 100853, China.
| | - Zhenhua Hu
- CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China; National Key Laboratory of Kidney Diseases, Beijing, 100853, China.
| |
Collapse
|
|
2
|
Li S, Wang B, Yu J, Kang D, He X, Guo H, He X. 3D-deep optical learning: a multimodal and multitask reconstruction framework for optical molecular tomography. OPTICS EXPRESS 2023; 31:23768-23789. [PMID: 37475220 DOI: 10.1364/oe.490139] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Accepted: 06/16/2023] [Indexed: 07/22/2023]
Abstract
Optical molecular tomography (OMT) is an emerging imaging technique. To date, the poor universality of reconstruction algorithms based on deep learning for various imaged objects and optical probes limits the development and application of OMT. In this study, based on a new mapping representation, a multimodal and multitask reconstruction framework-3D deep optical learning (3DOL), was presented to overcome the limitations of OMT in universality by decomposing it into two tasks, optical field recovery and luminous source reconstruction. Specifically, slices of the original anatomy (provided by computed tomography) and boundary optical measurement of imaged objects serve as inputs of a recurrent convolutional neural network encoded parallel to extract multimodal features, and 2D information from a few axial planes within the samples is explicitly incorporated, which enables 3DOL to recognize different imaged objects. Subsequently, the optical field is recovered under the constraint of the object geometry, and then the luminous source is segmented by a learnable Laplace operator from the recovered optical field, which obtains stable and high-quality reconstruction results with extremely few parameters. This strategy enable 3DOL to better understand the relationship between the boundary optical measurement, optical field, and luminous source to improve 3DOL's ability to work in a wide range of spectra. The results of numerical simulations, physical phantoms, and in vivo experiments demonstrate that 3DOL is a compatible deep-learning approach to tomographic imaging diverse objects. Moreover, the fully trained 3DOL under specific wavelengths can be generalized to other spectra in the 620-900 nm NIR-I window.
Collapse
|
|
3
|
Cao X, Li W, Chen Y, Du M, Zhang G, Zhang J, Li K, Su L. K-CapsNet: K-Nearest Neighbor Based Convolution Capsule Network for Cerenkov Luminescence Tomography Reconstruction. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38082846 DOI: 10.1109/embc40787.2023.10341089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Cerenkov luminescence tomography (CLT) has received significant attention as a promising imaging modality that can display the three-dimensional (3D) distribution of radioactive probes. However, the reconstruction of CLT suffers from severe ill-posed problem. It is difficult for traditional model-based method to obtain satisfactory result. Recently, deep learning-based method have shown great potential for accurate and efficient CLT reconstruction. In this study, a KNN-based convolution capsule network, named K-CapsNet, is proposed for cerenkov luminescence tomography. In K-CapsNet, the surface photon intensity is encoded in capsule form. The KNN-based convolution and K-means clustering are proposed for efficient encoding. Numerical simulation experiments have been carried out to verify the performance of K-CapsNet, and the results show that it performs superior in source localization and morphological restoration compared with existing methods.
Collapse
|
|
4
|
Cao X, Du M, Chen Y, Zhang G, Zhang J, Li W, Li K, Zhao F. FISTA-NET: Deep Algorithm Unrolling for Cerenkov luminescence tomography. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083164 DOI: 10.1109/embc40787.2023.10340506] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Cerenkov luminescence tomography (CLT) is a highly sensitive and promising imaging technique that can be used to reconstruct the three-dimensional distribution of radioactive probes in living animals. However, the accuracy of CLT reconstruction is limited by the simplified radiative transfer equation and ill-conditioned inverse problem. To address this issue, we propose a model-based deep learning network that combines the neural network with a model-based approach to enhance the performance of CLT reconstruction. The Fast Iterative Shrinkage Thresholding Algorithm (FISTA), a traditional model-based approach, is expanded into a deep network (known as FISTA-NET). Each layer in the network represents an iteration of the algorithm steps, and connecting these layers can form a deep neural network. In addition, different from the traditional FISTA, the key parameters in FISTA, such as gradient step size and threshold value, can be learned through training data without manual production. To evaluate the performance of FISTA-NET, numerical simulation experiments were conducted, which demonstrate its excellent positioning and shape recovery abilities.Clinical Relevance-This indicates that FISTA-NET strategy can significantly improve the quality of CLT reconstruction, which is further beneficial to the assessment of disease activity and treatment effect based on CLT.
Collapse
|
|
5
|
Feng J, Zhang H, Geng M, Chen H, Jia K, Sun Z, Li Z, Cao X, Pogue BW. X-ray Cherenkov-luminescence tomography reconstruction with a three-component deep learning algorithm: Swin transformer, convolutional neural network, and locality module. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:026004. [PMID: 36818584 PMCID: PMC9932523 DOI: 10.1117/1.jbo.28.2.026004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Accepted: 01/19/2023] [Indexed: 06/18/2023]
Abstract
SIGNIFICANCE X-ray Cherenkov-luminescence tomography (XCLT) produces fast emission data from megavoltage (MV) x-ray scanning, in which the excitation location of molecules within tissue is reconstructed. However standard filtered backprojection (FBP) algorithms for XCLT sinogram reconstruction can suffer from insufficient data due to dose limitations, so there are limits in the reconstruction quality with some artifacts. We report a deep learning algorithm for XCLT with high image quality and improved quantitative accuracy. AIM To directly reconstruct the distribution of emission quantum yield for x-ray Cherenkov-luminescence tomography, we proposed a three-component deep learning algorithm that includes a Swin transformer, convolution neural network, and locality module model. APPROACH A data-to-image model x-ray Cherenkov-luminescence tomography is developed based on a Swin transformer, which is used to extract pixel-level prior information from the sinogram domain. Meanwhile, a convolutional neural network structure is deployed to transform the extracted pixel information from the sinogram domain to the image domain. Finally, a locality module is designed between the encoder and decoder connection structures for delivering features. Its performance was validated with simulation, physical phantom, and in vivo experiments. RESULTS This approach can better deal with the limits to data than conventional FBP methods. The method was validated with numerical and physical phantom experiments, with results showing that it improved the reconstruction performance mean square error ( > 94.1 % ), peak signal-to-noise ratio ( > 41.7 % ), and Pearson correlation ( > 19 % ) compared with the FBP algorithm. The Swin-CNN also achieved a 32.1% improvement in PSNR over the deep learning method AUTOMAP. CONCLUSIONS This study shows that the three-component deep learning algorithm provides an effective reconstruction method for x-ray Cherenkov-luminescence tomography.
Collapse
Affiliation(s)
- Jinchao Feng
- Beijing University of Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Faculty of Information Technology, Beijing, China
- Beijing Laboratory of Advanced Information Networks, Beijing, China
| | - Hu Zhang
- Beijing University of Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Faculty of Information Technology, Beijing, China
| | - Mengfan Geng
- Beijing University of Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Faculty of Information Technology, Beijing, China
| | - Hanliang Chen
- Beijing University of Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Faculty of Information Technology, Beijing, China
| | - Kebin Jia
- Beijing University of Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Faculty of Information Technology, Beijing, China
- Beijing Laboratory of Advanced Information Networks, Beijing, China
| | - Zhonghua Sun
- Beijing University of Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Faculty of Information Technology, Beijing, China
- Beijing Laboratory of Advanced Information Networks, Beijing, China
| | - Zhe Li
- Beijing University of Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Faculty of Information Technology, Beijing, China
- Beijing Laboratory of Advanced Information Networks, Beijing, China
| | - Xu Cao
- Xidian University, Engineering Research Center of Molecular and Neuro Imaging of the Ministry of Education and School of Life Science and Technology, Xi’an, China
| | - Brian W. Pogue
- University of Wisconsin-Madison, Department of Medical Physics, Madison, Wisconsin, United States
| |
Collapse
|
|
6
|
Cao C, Xiao A, Cai M, Shen B, Guo L, Shi X, Tian J, Hu Z. Excitation-based fully connected network for precise NIR-II fluorescence molecular tomography. BIOMEDICAL OPTICS EXPRESS 2022; 13:6284-6299. [PMID: 36589575 PMCID: PMC9774866 DOI: 10.1364/boe.474982] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/06/2022] [Revised: 10/26/2022] [Accepted: 10/30/2022] [Indexed: 06/17/2023]
Abstract
Fluorescence molecular tomography (FMT) is a novel imaging modality to obtain fluorescence biomarkers' three-dimensional (3D) distribution. However, the simplified mathematical model and complicated inverse problem limit it to achieving precise results. In this study, the second near-infrared (NIR-II) fluorescence imaging was adopted to mitigate tissue scattering and reduce noise interference. An excitation-based fully connected network was proposed to model the inverse process of NIR-II photon propagation and directly obtain the 3D distribution of the light source. An excitation block was embedded in the network allowing it to autonomously pay more attention to neurons related to the light source. The barycenter error was added to the loss function to improve the localization accuracy of the light source. Both numerical simulation and in vivo experiments showed the superiority of the novel NIR-II FMT reconstruction strategy over the baseline methods. This strategy was expected to facilitate the application of machine learning in biomedical research.
Collapse
Affiliation(s)
- Caiguang Cao
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- These authors contributed equally
| | - Anqi Xiao
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- These authors contributed equally
| | - Meishan Cai
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Biluo Shen
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Lishuang Guo
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China
| | - Xiaojing Shi
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Jie Tian
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China
- Engineering Research Center of Molecular and Neuro Imaging of Ministry of Education, School of Life Science and Technology, Xidian University, Xi'an, China
| | - Zhenhua Hu
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
|
7
|
Guo L, Cai M, Zhang X, Zhang Z, Shi X, Zhang X, Liu J, Hu Z, Tian J. A novel weighted auxiliary set matching pursuit method for glioma in Cerenkov luminescence tomography reconstruction. JOURNAL OF BIOPHOTONICS 2022; 15:e202200126. [PMID: 36328059 DOI: 10.1002/jbio.202200126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Revised: 06/25/2022] [Accepted: 07/05/2022] [Indexed: 06/16/2023]
Abstract
Cerenkov luminescence tomography (CLT) is a promising three-dimensional imaging technology that has been actively investigated in preclinical studies. However, because of the ill-posedness in the inverse problem of CLT reconstruction, the reconstruction performance is still not satisfactory for broad biomedical applications. In this study, a novel weighted auxiliary set matching pursuit (WASMP) method was explored to enhance the accuracy of CLT reconstruction. The numerical simulations and in vivo imaging studies using tumor-bearing mice models were conducted to evaluate the performance of the WASMP method. The results of the above experiments proved that the WASMP method achieved superior reconstruction performance than other approaches in terms of positional accuracy and shape recovery. It further demonstrates that the atom selection strategy proposed in this study has a positive effect on improving the accuracy of atoms. The proposed WASMP improves the accuracy for CLT reconstruction for biomedical applications.
Collapse
Affiliation(s)
- Lishuang Guo
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China
- Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Meishan Cai
- Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Xiaoning Zhang
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China
- Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Zeyu Zhang
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China
- Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Xiaojing Shi
- Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Xiaojun Zhang
- Department of Nuclear Medicine, Chinese PLA General Hospital, Beijing, China
| | - Jiangang Liu
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China
| | - Zhenhua Hu
- Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Jie Tian
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China
- Beijing Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
|
8
|
Chen Y, Li W, Du M, Su L, Yi H, Zhao F, Li K, Wang L, Cao X. Elastic net-based non-negative iterative three-operator splitting strategy for Cerenkov luminescence tomography. OPTICS EXPRESS 2022; 30:35282-35299. [PMID: 36258483 DOI: 10.1364/oe.465501] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Accepted: 08/18/2022] [Indexed: 06/16/2023]
Abstract
Cerenkov luminescence tomography (CLT) provides a powerful optical molecular imaging technique for non-invasive detection and visualization of radiopharmaceuticals in living objects. However, the severe photon scattering effect causes ill-posedness of the inverse problem, and the location accuracy and shape recovery of CLT reconstruction results are unsatisfactory for clinical application. Here, to improve the reconstruction spatial location accuracy and shape recovery ability, a non-negative iterative three operator splitting (NNITOS) strategy based on elastic net (EN) regularization was proposed. NNITOS formalizes the CLT reconstruction as a non-convex optimization problem and splits it into three operators, the least square, L1/2-norm regularization, and adaptive grouping manifold learning, then iteratively solved them. After stepwise iterations, the result of NNITOS converged progressively. Meanwhile, to speed up the convergence and ensure the sparsity of the solution, shrinking the region of interest was utilized in this strategy. To verify the effectiveness of the method, numerical simulations and in vivo experiments were performed. The result of these experiments demonstrated that, compared to several methods, NNITOS can achieve superior performance in terms of location accuracy, shape recovery capability, and robustness. We hope this work can accelerate the clinical application of CLT in the future.
Collapse
|
|
9
|
Abstract
Malignant tumors rank as a leading cause of death worldwide. Accurate diagnosis and advanced treatment options are crucial to win battle against tumors. In recent years, Cherenkov luminescence (CL) has shown its technical advantages and clinical transformation potential in many important fields, particularly in tumor diagnosis and treatment, such as tumor detection in vivo, surgical navigation, radiotherapy, photodynamic therapy, and the evaluation of therapeutic effect. In this review, we summarize the advances in CL for tumor diagnosis and treatment. We first describe the physical principles of CL and discuss the imaging techniques used in tumor diagnosis, including CL imaging, CL endoscope, and CL tomography. Then we present a broad overview of the current status of surgical resection, radiotherapy, photodynamic therapy, and tumor microenvironment monitoring using CL. Finally, we shed light on the challenges and possible solutions for tumor diagnosis and therapy using CL.
Collapse
|
|
10
|
Zhang Z, He K, Chi C, Hu Z, Tian J. Intraoperative fluorescence molecular imaging accelerates the coming of precision surgery in China. Eur J Nucl Med Mol Imaging 2022; 49:2531-2543. [PMID: 35230491 PMCID: PMC9206608 DOI: 10.1007/s00259-022-05730-y] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Accepted: 02/13/2022] [Indexed: 02/06/2023]
Abstract
Purpose China has the largest cancer population globally. Surgery is the main choice for most solid cancer patients. Intraoperative fluorescence molecular imaging (FMI) has shown its great potential in assisting surgeons in achieving precise resection. We summarized the typical applications of intraoperative FMI and several new trends to promote the development of precision surgery. Methods The academic database and NIH clinical trial platform were systematically evaluated. We focused on the clinical application of intraoperative FMI in China. Special emphasis was placed on a series of typical studies with new technologies or high-level evidence. The emerging strategy of combining FMI with other modalities was also discussed. Results The clinical applications of clinically approved indocyanine green (ICG), methylene blue (MB), or fluorescein are on the rise in different surgical departments. Intraoperative FMI has achieved precise lesion detection, sentinel lymph node mapping, and lymphangiography for many cancers. Nerve imaging is also exploring to reduce iatrogenic injuries. Through different administration routes, these fluorescent imaging agents provided encouraging results in surgical navigation. Meanwhile, designing new cancer-specific fluorescent tracers is expected to be a promising trend to further improve the surgical outcome. Conclusions Intraoperative FMI is in a rapid development in China. In-depth understanding of cancer-related molecular mechanisms is necessary to achieve precision surgery. Molecular-targeted fluorescent agents and multi-modal imaging techniques might play crucial roles in the era of precision surgery.
Collapse
Affiliation(s)
- Zeyu Zhang
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China.,CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Kunshan He
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China.,State Key Laboratory of Computer Science and Beijing Key Lab of Human-Computer Interaction, Institute of Software, Chinese Academy of Sciences, Beijing, China
| | - Chongwei Chi
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China. .,University of Chinese Academy of Sciences, Beijing, China.
| | - Zhenhua Hu
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China. .,University of Chinese Academy of Sciences, Beijing, China.
| | - Jie Tian
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China. .,CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
|
11
|
Zhang X, Cai M, Guo L, Zhang Z, Shen B, Zhang X, Hu Z, Tian J. Attention mechanism-based locally connected network for accurate and stable reconstruction in Cerenkov luminescence tomography. BIOMEDICAL OPTICS EXPRESS 2021; 12:7703-7716. [PMID: 35003861 PMCID: PMC8713679 DOI: 10.1364/boe.443517] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Revised: 10/28/2021] [Accepted: 11/09/2021] [Indexed: 06/14/2023]
Abstract
Cerenkov luminescence tomography (CLT) is a novel and highly sensitive imaging technique, which could obtain the three-dimensional distribution of radioactive probes to achieve accurate tumor detection. However, the simplified radiative transfer equation and ill-conditioned inverse problem cause a reconstruction error. In this study, a novel attention mechanism based locally connected (AMLC) network was proposed to reduce barycenter error and improve morphological restorability. The proposed AMLC network consisted of two main parts: a fully connected sub-network for providing a coarse reconstruction result, and a locally connected sub-network based on an attention matrix for refinement. Both numerical simulations and in vivo experiments were conducted to show the superiority of the AMLC network in accuracy and stability over existing methods (MFCNN, KNN-LC network). This method improved CLT reconstruction performance and promoted the application of machine learning in optical imaging research.
Collapse
Affiliation(s)
- Xiaoning Zhang
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine, Beihang University, Beijing, China
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- Equal contribution
| | - Meishan Cai
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- Equal contribution
| | - Lishuang Guo
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine, Beihang University, Beijing, China
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Zeyu Zhang
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine, Beihang University, Beijing, China
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Biluo Shen
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Xiaojun Zhang
- Department of Nuclear Medicine, Chinese PLA General Hospital, Beijing, 100853, China
| | - Zhenhua Hu
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Jie Tian
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine, Beihang University, Beijing, China
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
|
12
|
Wang L, Zhu W, Zhang Y, Chen S, Yang D. Harnessing the Power of Hybrid Light Propagation Model for Three-Dimensional Optical Imaging in Cancer Detection. Front Oncol 2021; 11:750764. [PMID: 34804938 PMCID: PMC8601256 DOI: 10.3389/fonc.2021.750764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Accepted: 08/30/2021] [Indexed: 12/04/2022] Open
Abstract
Optical imaging is an emerging technology capable of qualitatively and quantitatively observing life processes at the cellular or molecular level and plays a significant role in cancer detection. In particular, to overcome the disadvantages of traditional optical imaging that only two-dimensionally and qualitatively detect biomedical information, the corresponding three-dimensional (3D) imaging technology is intensively explored to provide 3D quantitative information, such as localization and distribution and tumor cell volume. To retrieve these information, light propagation models that reflect the interaction between light and biological tissues are an important prerequisite and basis for 3D optical imaging. This review concentrates on the recent advances in hybrid light propagation models, with particular emphasis on their powerful use for 3D optical imaging in cancer detection. Finally, we prospect the wider application of the hybrid light propagation model and future potential of 3D optical imaging in cancer detection.
Collapse
Affiliation(s)
- Lin Wang
- School of Computer Science and Engineering, Xi'an University of Technology, Xi'an, China
| | - Wentao Zhu
- Zhejiang Lab, Research Center for Healthcare Data Science, Hangzhou, China
| | - Ying Zhang
- Zhejiang Lab, Research Center for Healthcare Data Science, Hangzhou, China
| | - Shangdong Chen
- School of Information Sciences and Technology, Northwest University, Xi'an, China
| | - Defu Yang
- Intelligent Information Processing Laboratory, Hangzhou Dianzi University, Hangzhou, China
| |
Collapse
|
|
13
|
Guo H, Zhao H, Song X, He X. Alternating Direction Method of Multipliers Network for Bioluminescence Tomography Reconstruction. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3109-3113. [PMID: 34891900 DOI: 10.1109/embc46164.2021.9630213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Bioluminescence tomography (BLT) is an effective noninvasive molecular imaging modality for three dimensional visualization of in vivo tumor research in small animals. The approaches of deep learning have shown great potential in the field of optical molecular imaging in recent years. However, the common problem with these existing end-to-end networks is the black box technology, whose solving process is not theoretically proven. In this work, we proposed a novel Alternating Direction Method of Multipliers Network (ADMM-Net) to solve the poor interpretation problem of internal process. The ADMM-Net combines the framework of deep learning on the basis of traditional ADMM algorithm to dynamically learn various parameters of the algorithm in the form of network. To evaluate the performance of our proposed network, we implemented numerical simulation experiments. The results show that the ADMM-Net can accurately reconstruct the location of the source, and the morphological similarity with the real source is also higher.
Collapse
|
|
14
|
Wang L, He X, Yu J. Prior Compensation Algorithm for Cerenkov Luminescence Tomography From Single-View Measurements. Front Oncol 2021; 11:749889. [PMID: 34631587 PMCID: PMC8495210 DOI: 10.3389/fonc.2021.749889] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Accepted: 09/06/2021] [Indexed: 11/13/2022] Open
Abstract
Cerenkov luminescence tomography (CLT) has attracted much attention because of the wide clinically-used probes and three-dimensional (3D) quantification ability. However, due to the serious morbidity of 3D optical imaging, the reconstructed images of CLT are not appreciable, especially when single-view measurements are used. Single-view CLT improves the efficiency of data acquisition. It is much consistent with the actual imaging environment of using commercial imaging system, but bringing the problem that the reconstructed results will be closer to the animal surface on the side where the single-view image is collected. To avoid this problem to the greatest extent possible, we proposed a prior compensation algorithm for CLT reconstruction based on depth calibration strategy. This method takes full account of the fact that the attenuation of light in the tissue will depend heavily on the depth of the light source as well as the distance between the light source and the detection plane. Based on this consideration, a depth calibration matrix was designed to calibrate the attenuation between the surface light flux and the density of the internal light source. The feature of the algorithm was that the depth calibration matrix directly acts on the system matrix of CLT reconstruction, rather than modifying the regularization penalty items. The validity and effectiveness of the proposed algorithm were evaluated with a numerical simulation and a mouse-based experiment, whose results illustrated that it located the radiation sources accurately by using single-view measurements.
Collapse
Affiliation(s)
- Lin Wang
- School of Information Sciences and Technology, Northwest University, Xi'an, China.,School of Computer Science and Engineering, Xi'an University of Technology, Xi'an, China
| | - Xiaowei He
- School of Information Sciences and Technology, Northwest University, Xi'an, China
| | - Jingjing Yu
- School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| |
Collapse
|
|
15
|
Cao X, Li K, Xu XL, Deneen KMV, Geng GH, Chen XL. Development of tomographic reconstruction for three-dimensional optical imaging: From the inversion of light propagation to artificial intelligence. Artif Intell Med Imaging 2020; 1:78-86. [DOI: 10.35711/aimi.v1.i2.78] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/05/2020] [Revised: 08/01/2020] [Accepted: 08/21/2020] [Indexed: 02/06/2023] Open
Abstract
Optical molecular tomography (OMT) is an imaging modality which uses an optical signal, especially near-infrared light, to reconstruct the three-dimensional information of the light source in biological tissue. With the advantages of being low-cost, noninvasive and having high sensitivity, OMT has been applied in preclinical and clinical research. However, due to its serious ill-posedness and ill-condition, the solution of OMT requires heavy data analysis and the reconstruction quality is limited. Recently, the artificial intelligence (commonly known as AI)-based methods have been proposed to provide a different tool to solve the OMT problem. In this paper, we review the progress on OMT algorithms, from conventional methods to AI-based methods, and we also give a prospective towards future developments in this domain.
Collapse
Affiliation(s)
- Xin Cao
- School of Information Science and Technology, Northwest University, Xi’an 710069, Shaanxi Province, China
| | - Kang Li
- School of Information Science and Technology, Northwest University, Xi’an 710069, Shaanxi Province, China
| | - Xue-Li Xu
- School of Information Science and Technology, Northwest University, Xi’an 710069, Shaanxi Province, China
| | - Karen M von Deneen
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, and School of Life Science and Technology, Xidian University, Xi’an 710126, Shaanxi Province, China
| | - Guo-Hua Geng
- School of Information Science and Technology, Northwest University, Xi’an 710069, Shaanxi Province, China
| | - Xue-Li Chen
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, and School of Life Science and Technology, Xidian University, Xi’an 710126, Shaanxi Province, China
| |
Collapse
|
|
16
|
Chen XL, Yan TY, Wang N, von Deneen KM. Rising role of artificial intelligence in image reconstruction for biomedical imaging. Artif Intell Med Imaging 2020; 1:1-5. [DOI: 10.35711/aimi.v1.i1.1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/07/2020] [Revised: 06/09/2020] [Accepted: 06/16/2020] [Indexed: 02/06/2023] Open
Abstract
In this editorial, we review recent progress on the applications of artificial intelligence (AI) in image reconstruction for biomedical imaging. Because it abandons prior information of traditional artificial design and adopts a completely data-driven mode to obtain deeper prior information via learning, AI technology plays an increasingly important role in biomedical image reconstruction. The combination of AI technology and the biomedical image reconstruction method has become a hotspot in the field. Favoring AI, the performance of biomedical image reconstruction has been improved in terms of accuracy, resolution, imaging speed, etc. We specifically focus on how to use AI technology to improve the performance of biomedical image reconstruction, and propose possible future directions in this field.
Collapse
Affiliation(s)
- Xue-Li Chen
- Engineering Research Center of Molecular and Neuro Imaging of Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an 710126, Shaanxi Province, China
| | - Tian-Yu Yan
- Engineering Research Center of Molecular and Neuro Imaging of Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an 710126, Shaanxi Province, China
| | - Nan Wang
- Engineering Research Center of Molecular and Neuro Imaging of Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an 710126, Shaanxi Province, China
| | - Karen M von Deneen
- Engineering Research Center of Molecular and Neuro Imaging of Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an 710126, Shaanxi Province, China
| |
Collapse
|