|
1
|
Plutenko I, Radchuk V, Mayer S, Keil P, Ortleb S, Wagner S, Lehmann V, Rolletschek H, Borisjuk L. MRI-Seed-Wizard: combining deep learning algorithms with magnetic resonance imaging enables advanced seed phenotyping. JOURNAL OF EXPERIMENTAL BOTANY 2025; 76:393-410. [PMID: 39383098 PMCID: PMC11714760 DOI: 10.1093/jxb/erae408] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/28/2024] [Accepted: 10/08/2024] [Indexed: 10/11/2024]
Abstract
Evaluation of relevant seed traits is an essential part of most plant breeding and biotechnology programmes. There is a need for non-destructive, three-dimensional assessment of the morphometry, composition, and internal features of seeds. Here, we introduce a novel tool, MRI-Seed-Wizard, which integrates deep learning algorithms with non-invasive magnetic resonance imaging (MRI) for use in a new domain-plant MRI. The tool enabled in vivo quantification of 23 grain traits, including volumetric parameters of inner seed structure. Several of these features cannot be assessed using conventional techniques, including X-ray computed tomography. MRI-Seed-Wizard was designed to automate the manual processes of identifying, labeling, and analysing digital MRI data. We further provide advanced MRI protocols that allow the evaluation of multiple seeds simultaneously to increase throughput. The versatility of MRI-Seed-Wizard in seed phenotyping is demonstrated for wheat (Triticum aestivum) and barley (Hordeum vulgare) grains, and it is applicable to a wide range of crop seeds. Thus, artificial intelligence, combined with the most versatile imaging modality, MRI, opens up new perspectives in seed phenotyping and crop improvement.
Collapse
Affiliation(s)
- Iaroslav Plutenko
- Leibniz-Institute of Plant Genetics and Crop Plant Research (IPK), Corrensstrasse 3, 06466 Seeland OT Gatersleben, Germany
| | - Volodymyr Radchuk
- Leibniz-Institute of Plant Genetics and Crop Plant Research (IPK), Corrensstrasse 3, 06466 Seeland OT Gatersleben, Germany
| | - Simon Mayer
- Leibniz-Institute of Plant Genetics and Crop Plant Research (IPK), Corrensstrasse 3, 06466 Seeland OT Gatersleben, Germany
- Institute of Experimental Physics 5, University of Würzburg, Am Hubland, 97074 Würzburg, Germany
| | - Peter Keil
- Leibniz-Institute of Plant Genetics and Crop Plant Research (IPK), Corrensstrasse 3, 06466 Seeland OT Gatersleben, Germany
| | - Stefan Ortleb
- Leibniz-Institute of Plant Genetics and Crop Plant Research (IPK), Corrensstrasse 3, 06466 Seeland OT Gatersleben, Germany
| | - Steffen Wagner
- Leibniz-Institute of Plant Genetics and Crop Plant Research (IPK), Corrensstrasse 3, 06466 Seeland OT Gatersleben, Germany
| | - Volker Lehmann
- Bruker BioSpin GmbH, Rudolf-Plank-Str. 23, 76275 Ettlingen, Germany
| | - Hardy Rolletschek
- Leibniz-Institute of Plant Genetics and Crop Plant Research (IPK), Corrensstrasse 3, 06466 Seeland OT Gatersleben, Germany
| | - Ljudmilla Borisjuk
- Leibniz-Institute of Plant Genetics and Crop Plant Research (IPK), Corrensstrasse 3, 06466 Seeland OT Gatersleben, Germany
| |
Collapse
|
|
2
|
Wang L, Fatemi M, Alizad A. Artificial intelligence techniques in liver cancer. Front Oncol 2024; 14:1415859. [PMID: 39290245 PMCID: PMC11405163 DOI: 10.3389/fonc.2024.1415859] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2024] [Accepted: 08/15/2024] [Indexed: 09/19/2024] Open
Abstract
Hepatocellular Carcinoma (HCC), the most common primary liver cancer, is a significant contributor to worldwide cancer-related deaths. Various medical imaging techniques, including computed tomography, magnetic resonance imaging, and ultrasound, play a crucial role in accurately evaluating HCC and formulating effective treatment plans. Artificial Intelligence (AI) technologies have demonstrated potential in supporting physicians by providing more accurate and consistent medical diagnoses. Recent advancements have led to the development of AI-based multi-modal prediction systems. These systems integrate medical imaging with other modalities, such as electronic health record reports and clinical parameters, to enhance the accuracy of predicting biological characteristics and prognosis, including those associated with HCC. These multi-modal prediction systems pave the way for predicting the response to transarterial chemoembolization and microvascular invasion treatments and can assist clinicians in identifying the optimal patients with HCC who could benefit from interventional therapy. This paper provides an overview of the latest AI-based medical imaging models developed for diagnosing and predicting HCC. It also explores the challenges and potential future directions related to the clinical application of AI techniques.
Collapse
Affiliation(s)
- Lulu Wang
- Department of Engineering, School of Technology, Reykjavık University, Reykjavík, Iceland
- Department of Physiology and Biomedical Engineering, Mayo Clinic College of Medicine and Science, Rochester, MN, United States
| | - Mostafa Fatemi
- Department of Physiology and Biomedical Engineering, Mayo Clinic College of Medicine and Science, Rochester, MN, United States
| | - Azra Alizad
- Department of Radiology, Mayo Clinic College of Medicine and Science, Rochester, MN, United States
| |
Collapse
|
|
3
|
Zhang B, Qiu S, Liang T. Dual Attention-Based 3D U-Net Liver Segmentation Algorithm on CT Images. Bioengineering (Basel) 2024; 11:737. [PMID: 39061819 PMCID: PMC11273630 DOI: 10.3390/bioengineering11070737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2024] [Revised: 07/11/2024] [Accepted: 07/17/2024] [Indexed: 07/28/2024] Open
Abstract
The liver is a vital organ in the human body, and CT images can intuitively display its morphology. Physicians rely on liver CT images to observe its anatomical structure and areas of pathology, providing evidence for clinical diagnosis and treatment planning. To assist physicians in making accurate judgments, artificial intelligence techniques are adopted. Addressing the limitations of existing methods in liver CT image segmentation, such as weak contextual analysis and semantic information loss, we propose a novel Dual Attention-Based 3D U-Net liver segmentation algorithm on CT images. The innovations of our approach are summarized as follows: (1) We improve the 3D U-Net network by introducing residual connections to better capture multi-scale information and alleviate semantic information loss. (2) We propose the DA-Block encoder structure to enhance feature extraction capability. (3) We introduce the CBAM module into skip connections to optimize feature transmission in the encoder, reducing semantic gaps and achieving accurate liver segmentation. To validate the effectiveness of the algorithm, experiments were conducted on the LiTS dataset. The results showed that the Dice coefficient and HD95 index for liver images were 92.56% and 28.09 mm, respectively, representing an improvement of 0.84% and a reduction of 2.45 mm compared to 3D Res-UNet.
Collapse
Affiliation(s)
- Benyue Zhang
- Key Laboratory of Spectral Imaging Technology CAS, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China;
- School of Optoelectronics, University of Chinese Academy of Sciences, Beijing 100408, China
| | - Shi Qiu
- Key Laboratory of Spectral Imaging Technology CAS, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China;
| | - Ting Liang
- Department of Radiology, The First Affiliated Hospital of Xi’an Jiaotong University, Xi’an 710119, China
| |
Collapse
|
|
4
|
Li M, Ding N, Yin S, Lu Y, Ji Y, Jin L. Enhancing automatic prediction of clinically significant prostate cancer with deep transfer learning 2.5-dimensional segmentation on bi-parametric magnetic resonance imaging (bp-MRI). Quant Imaging Med Surg 2024; 14:4893-4902. [PMID: 39022227 PMCID: PMC11250323 DOI: 10.21037/qims-24-587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2024] [Accepted: 05/20/2024] [Indexed: 07/20/2024]
Abstract
Background The aggressiveness of prostate cancer (PCa) is crucial in determining treatment method. The purpose of this study was to establish a 2.5-dimensional (2.5D) deep transfer learning (DTL) detection model for the automatic detection of clinically significant PCa (csPCa) based on bi-parametric magnetic resonance imaging (bp-MRI). Methods A total of 231 patients, including 181 with csPCa and 50 with non-clinically significant PCa (non-csPCa), were enrolled. Stratified random sampling was then employed to divide all participants into a training set [185] and a test set [46]. The DTL model was obtained through image acquisition, image segmentation, and model construction. Finally, the diagnostic performance of the 2.5D and 2-dimensional (2D) models in predicting the aggressiveness of PCa was evaluated and compared using receiver operating characteristic (ROC) curves. Results DTL models based on 2D and 2.5D segmentation were established and validated to assess the aggressiveness of PCa. The results demonstrated that the diagnostic efficiency of the DTL model based on 2.5D was superior to that of the 2D model, regardless of whether in a single or combined sequence. Particularly, the 2.5D combined model outperformed other models in differentiating csPCa from non-csPCa. The area under the curve (AUC) values for the 2.5D combined model in the training and test sets were 0.960 and 0.949, respectively. Furthermore, the T2-weighted imaging (T2WI) model showed superiority over the apparent diffusion coefficient (ADC) model, but was not as effective as the combined model, whether based on 2.5D or 2D. Conclusions A DTL model based on 2.5D segmentation was developed to automatically evaluate PCa aggressiveness on bp-MRI, improving the diagnostic performance of the 2D model. The results indicated that the continuous information between adjacent layers can enhance the detection rate of lesions and reduce the misjudgment rate based on the DTL model.
Collapse
Affiliation(s)
- Mengjuan Li
- Department of Radiology, Suzhou Ninth People's Hospital, Suzhou Ninth Hospital Affiliated to Soochow University, Suzhou, China
| | - Ning Ding
- Department of Radiology, Suzhou Ninth People's Hospital, Suzhou Ninth Hospital Affiliated to Soochow University, Suzhou, China
| | - Shengnan Yin
- Department of Radiology, Suzhou Ninth People's Hospital, Suzhou Ninth Hospital Affiliated to Soochow University, Suzhou, China
| | - Yan Lu
- Department of Radiology, Suzhou Ninth People's Hospital, Suzhou Ninth Hospital Affiliated to Soochow University, Suzhou, China
| | - Yiding Ji
- Department of Radiology, Suzhou Ninth People's Hospital, Suzhou Ninth Hospital Affiliated to Soochow University, Suzhou, China
| | - Long Jin
- Department of Radiology, Suzhou Ninth People's Hospital, Suzhou Ninth Hospital Affiliated to Soochow University, Suzhou, China
| |
Collapse
|
|
5
|
Lee IC, Tsai YP, Lin YC, Chen TC, Yen CH, Chiu NC, Hwang HE, Liu CA, Huang JG, Lee RC, Chao Y, Ho SY, Huang YH. A hierarchical fusion strategy of deep learning networks for detection and segmentation of hepatocellular carcinoma from computed tomography images. Cancer Imaging 2024; 24:43. [PMID: 38532511 PMCID: PMC10964581 DOI: 10.1186/s40644-024-00686-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Accepted: 03/08/2024] [Indexed: 03/28/2024] Open
Abstract
BACKGROUND Automatic segmentation of hepatocellular carcinoma (HCC) on computed tomography (CT) scans is in urgent need to assist diagnosis and radiomics analysis. The aim of this study is to develop a deep learning based network to detect HCC from dynamic CT images. METHODS Dynamic CT images of 595 patients with HCC were used. Tumors in dynamic CT images were labeled by radiologists. Patients were randomly divided into training, validation and test sets in a ratio of 5:2:3, respectively. We developed a hierarchical fusion strategy of deep learning networks (HFS-Net). Global dice, sensitivity, precision and F1-score were used to measure performance of the HFS-Net model. RESULTS The 2D DenseU-Net using dynamic CT images was more effective for segmenting small tumors, whereas the 2D U-Net using portal venous phase images was more effective for segmenting large tumors. The HFS-Net model performed better, compared with the single-strategy deep learning models in segmenting small and large tumors. In the test set, the HFS-Net model achieved good performance in identifying HCC on dynamic CT images with global dice of 82.8%. The overall sensitivity, precision and F1-score were 84.3%, 75.5% and 79.6% per slice, respectively, and 92.2%, 93.2% and 92.7% per patient, respectively. The sensitivity in tumors < 2 cm, 2-3, 3-5 cm and > 5 cm were 72.7%, 92.9%, 94.2% and 100% per patient, respectively. CONCLUSIONS The HFS-Net model achieved good performance in the detection and segmentation of HCC from dynamic CT images, which may support radiologic diagnosis and facilitate automatic radiomics analysis.
Collapse
Affiliation(s)
- I-Cheng Lee
- Division of Gastroenterology and Hepatology, Department of Medicine, Taipei Veterans General Hospital, Taipei, Taiwan
- School of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan
| | - Yung-Ping Tsai
- Institute of Bioinformatics and Systems Biology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Yen-Cheng Lin
- Institute of Bioinformatics and Systems Biology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Ting-Chun Chen
- Institute of Bioinformatics and Systems Biology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Chia-Heng Yen
- Institute of Computer Science and Engineering, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Nai-Chi Chiu
- Department of Radiology, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Hsuen-En Hwang
- Department of Radiology, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Chien-An Liu
- Department of Radiology, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Jia-Guan Huang
- National Taiwan University School of Medicine, Taipei, Taiwan
| | - Rheun-Chuan Lee
- Department of Radiology, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Yee Chao
- Cancer Center, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Shinn-Ying Ho
- Institute of Bioinformatics and Systems Biology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan.
- Department of Biological Science and Technology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan.
- Center for Intelligent Drug Systems and Smart Bio-devices (IDS 2 B), National Yang Ming Chiao Tung University, Hsinchu, Taiwan.
- College of Health Sciences, Kaohsiung Medical University, Kaohsiung, Taiwan.
| | - Yi-Hsiang Huang
- Division of Gastroenterology and Hepatology, Department of Medicine, Taipei Veterans General Hospital, Taipei, Taiwan.
- School of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan.
- Institute of Clinical Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan.
- Healthcare and Service Center, Taipei Veterans General Hospital, Taipei, Taiwan.
| |
Collapse
|
|
6
|
Ma X, Song H, Jia X, Wang Z. An improved V-Net lung nodule segmentation model based on pixel threshold separation and attention mechanism. Sci Rep 2024; 14:4743. [PMID: 38413699 PMCID: PMC10899216 DOI: 10.1038/s41598-024-55178-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 02/21/2024] [Indexed: 02/29/2024] Open
Abstract
Accurate labeling of lung nodules in computed tomography (CT) images is crucial in early lung cancer diagnosis and before nodule resection surgery. However, the irregular shape of lung nodules in CT images and the complex lung environment make it much more challenging to segment lung nodules accurately. On this basis, we propose an improved V-Net segmentation method based on pixel threshold separation and attention mechanism for lung nodules. This method first offers a data augment strategy to solve the problem of insufficient samples in 3D medical datasets. In addition, we integrate the feature extraction module based on pixel threshold separation into the model to enhance the feature extraction ability under different thresholds on the one hand. On the other hand, the model introduces channel and spatial attention modules to make the model pay more attention to important semantic information and improve its generalization ability and accuracy. Experiments show that the Dice similarity coefficients of the improved model on the public datasets LUNA16 and LNDb are 94.9% and 81.1% respectively, and the sensitivities reach 92.7% and 76.9% respectively. which is superior to most existing UNet architecture models and comparable to the manual level segmentation results by medical technologists.
Collapse
Affiliation(s)
- Xiaopu Ma
- School of Computer Science and Technology, Nanyang Normal University, Nanyang, 473061, China.
| | - Handing Song
- School of Life Sciences and Agricultural Engineering, Nanyang Normal University, Nanyang, 473061, China
| | - Xiao Jia
- School of Computer Science and Technology, Nanyang Normal University, Nanyang, 473061, China
| | - Zhan Wang
- School of Life Sciences and Agricultural Engineering, Nanyang Normal University, Nanyang, 473061, China
| |
Collapse
|
|
7
|
Li W, Jia M, Yang C, Lin Z, Yu Y, Zhang W. SPA-UNet: A liver tumor segmentation network based on fused multi-scale features. Open Life Sci 2023; 18:20220685. [PMID: 37724113 PMCID: PMC10505346 DOI: 10.1515/biol-2022-0685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 06/26/2023] [Accepted: 07/24/2023] [Indexed: 09/20/2023] Open
Abstract
Liver tumor segmentation is a critical part in the diagnosis and treatment of liver cancer. While U-shaped convolutional neural networks (UNets) have made significant strides in medical image segmentation, challenges remain in accurately segmenting tumor boundaries and detecting small tumors, resulting in low segmentation accuracy. To improve the segmentation accuracy of liver tumors, this work proposes space pyramid attention (SPA)-UNet, a novel image segmentation network with an encoder-decoder architecture. SPA-UNet consists of four modules: (1) Spatial pyramid convolution block (SPCB), extracting multi-scale features by fusing three sets of dilated convolutions with different rates. (2) Spatial pyramid pooling block (SPPB), performing downsampling to reduce image size. (3) Upsample module, integrating dense positional and semantic information. (4) Residual attention block (RA-Block), enabling precise tumor localization. The encoder incorporates 5 SPCBs and 4 SPPBs to capture contextual information. The decoder consists of the Upsample module and RA-Block, and finally a segmentation head outputs segmented images of liver and liver tumor. Experiments using the liver tumor segmentation dataset demonstrate that SPA-UNet surpasses the traditional UNet model, achieving a 1.0 and 2.0% improvement in intersection over union indicators for liver and tumors, respectively, along with increased recall rates by 1.2 and 1.8%. These advancements provide a dependable foundation for liver cancer diagnosis and treatment.
Collapse
Affiliation(s)
- Weikun Li
- School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Maoning Jia
- School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Chen Yang
- School of Business, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Zhenyuan Lin
- School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Yuekang Yu
- School of Information and Communication, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Wenhui Zhang
- School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| |
Collapse
|
|
8
|
Lakshmipriya B, Pottakkat B, Ramkumar G. Deep learning techniques in liver tumour diagnosis using CT and MR imaging - A systematic review. Artif Intell Med 2023; 141:102557. [PMID: 37295904 DOI: 10.1016/j.artmed.2023.102557] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2022] [Revised: 04/15/2023] [Accepted: 04/18/2023] [Indexed: 06/12/2023]
Abstract
Deep learning has become a thriving force in the computer aided diagnosis of liver cancer, as it solves extremely complicated challenges with high accuracy over time and facilitates medical experts in their diagnostic and treatment procedures. This paper presents a comprehensive systematic review on deep learning techniques applied for various applications pertaining to liver images, challenges faced by the clinicians in liver tumour diagnosis and how deep learning bridges the gap between clinical practice and technological solutions with an in-depth summary of 113 articles. Since, deep learning is an emerging revolutionary technology, recent state-of-the-art research implemented on liver images are reviewed with more focus on classification, segmentation and clinical applications in the management of liver diseases. Additionally, similar review articles in literature are reviewed and compared. The review is concluded by presenting the contemporary trends and unaddressed research issues in the field of liver tumour diagnosis, offering directions for future research in this field.
Collapse
Affiliation(s)
- B Lakshmipriya
- Department of Surgical Gastroenterology, Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry, India
| | - Biju Pottakkat
- Department of Surgical Gastroenterology, Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry, India.
| | - G Ramkumar
- Department of Radio Diagnosis, Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry, India
| |
Collapse
|
|
9
|
Wang J, Zhang X, Lv P, Wang H, Cheng Y. Automatic Liver Segmentation Using EfficientNet and Attention-Based Residual U-Net in CT. J Digit Imaging 2022; 35:1479-1493. [PMID: 35711074 PMCID: PMC9712863 DOI: 10.1007/s10278-022-00668-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2021] [Revised: 05/30/2022] [Accepted: 06/03/2022] [Indexed: 10/18/2022] Open
Abstract
This paper proposes a new network framework, which leverages EfficientNetB4, attention gate, and residual learning techniques to achieve automatic and accurate liver segmentation. First, we use EfficientNetB4 as the encoder to extract more feature information during the encoding stage. Then, an attention gate is introduced in the skip connection to eliminate irrelevant regions and highlight features of a specific segmentation task. Finally, to alleviate the problem of gradient vanishment, we replace the traditional convolution of the decoder with a residual block to improve the segmentation accuracy. We verified the proposed method on the LiTS17 and SLiver07 datasets and compared it with classical networks such as FCN, U-Net, attention U-Net, and attention Res-U-Net. In the Sliver07 evaluation, the proposed method achieved the best segmentation performance on all five standard metrics. Meanwhile, in the LiTS17 assessment, the best performance is obtained except for a slight inferior on RVD. The proposed method's qualitative and quantitative results demonstrated its applicability in liver segmentation and proved its good prospect in computer-assisted liver segmentation.
Collapse
Affiliation(s)
- Jinke Wang
- Department of Software Engineering, Harbin University of Science and Technology, No. 2006, Xueyuan Road, Shandong Province, Rongcheng City, 264300, China.
- School of Automation, Harbin University of Science and Technology, Harbin, 150080, China.
| | - Xiangyang Zhang
- School of Automation, Harbin University of Science and Technology, Harbin, 150080, China
| | - Peiqing Lv
- School of Automation, Harbin University of Science and Technology, Harbin, 150080, China
| | - Haiying Wang
- School of Automation, Harbin University of Science and Technology, Harbin, 150080, China
| | - Yuanzhi Cheng
- School of Information Science and Technology, Qingdao University of Science and Technology, Qingdao, 266061, China
| |
Collapse
|
|
10
|
Popescu D, Stanciulescu A, Pomohaci MD, Ichim L. Decision Support System for Liver Lesion Segmentation Based on Advanced Convolutional Neural Network Architectures. Bioengineering (Basel) 2022; 9:bioengineering9090467. [PMID: 36135013 PMCID: PMC9495456 DOI: 10.3390/bioengineering9090467] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Revised: 08/20/2022] [Accepted: 09/09/2022] [Indexed: 11/16/2022] Open
Abstract
Given its essential role in body functions, liver cancer is the third most common cause of death from cancer, despite being the sixth most common type of cancer worldwide. Following advancements in medicine and image processing, medical image segmentation methods are receiving a great deal of attention. As a novelty, the paper proposes an intelligent decision system for segmenting liver and hepatic tumors by integrating four efficient neural networks (ResNet152, ResNeXt101, DenseNet201, and InceptionV3). Images from computed tomography for training, validation, and testing were taken from the public LiTS17 database and preprocessed to better highlight liver tissue and tumors. Global segmentation is done by separately training individual classifiers and the global system of merging individual decisions. For the aforementioned application, classification neural networks have been modified for semantic segmentation. After segmentation based on the neural network system, the images were postprocessed to eliminate artifacts. The segmentation results obtained by the system were better, from the point of view of the Dice coefficient, than those obtained by the individual networks, and comparable with those reported in recent works.
Collapse
|
|
11
|
2.5D lightweight RIU-Net for automatic liver and tumor segmentation from CT. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103567] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
|
|
12
|
Kittipongdaja P, Siriborvornratanakul T. Automatic kidney segmentation using 2.5D ResUNet and 2.5D DenseUNet for malignant potential analysis in complex renal cyst based on CT images. EURASIP JOURNAL ON IMAGE AND VIDEO PROCESSING 2022; 2022:5. [PMID: 35340560 PMCID: PMC8938741 DOI: 10.1186/s13640-022-00581-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/07/2021] [Accepted: 02/23/2022] [Indexed: 05/26/2023]
Abstract
Bosniak renal cyst classification has been widely used in determining the complexity of a renal cyst. However, it turns out that about half of patients undergoing surgery for Bosniak category III, take surgical risks that reward them with no clinical benefit at all. This is because their pathological results reveal that the cysts are actually benign not malignant. This problem inspires us to use recently popular deep learning techniques and study alternative analytics methods for precise binary classification (benign or malignant tumor) on Computerized Tomography (CT) images. To achieve our goal, two consecutive steps are required-segmenting kidney organs or lesions from CT images then classifying the segmented kidneys. In this paper, we propose a study of kidney segmentation using 2.5D ResUNet and 2.5D DenseUNet for efficiently extracting intra-slice and inter-slice features. Our models are trained and validated on the public data set from Kidney Tumor Segmentation (KiTS19) challenge in two different training environments. As a result, all experimental models achieve high mean kidney Dice scores of at least 95% on the KiTS19 validation set consisting of 60 patients. Apart from the KiTS19 data set, we also conduct separate experiments on abdomen CT images of four Thai patients. Based on the four Thai patients, our experimental models show a drop in performance, where the best mean kidney Dice score is 87.60%.
Collapse
Affiliation(s)
- Parin Kittipongdaja
- Graduate School of Applied Statistics, National Institute of Development Administration, Bangkok, Thailand
| | | |
Collapse
|
|
13
|
ThoraxNet: a 3D U-Net based two-stage framework for OAR segmentation on thoracic CT images. Phys Eng Sci Med 2022; 45:189-203. [PMID: 35029804 DOI: 10.1007/s13246-022-01101-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Accepted: 01/06/2022] [Indexed: 10/19/2022]
Abstract
An important phase of radiation treatment planning is the accurate contouring of the organs at risk (OAR), which is necessary for the dose distribution calculation. The manual contouring approach currently used in clinical practice is tedious, time-consuming, and prone to inter and intra-observer variation. Therefore, a deep learning-based auto contouring tool can solve these issues by accurately delineating OARs on the computed tomography (CT) images. This paper proposes a two-stage deep learning-based segmentation model with an attention mechanism that automatically delineates OARs in thoracic CT images. After preprocessing the input CT volume, a 3D U-Net architecture will locate each organ to generate cropped images for the segmentation network. Next, two differently configured U-Net-based networks will perform the segmentation of large organs-left lung, right lung, heart, and small organs-esophagus and spinal cord, respectively. A post-processing step integrates all the individually-segmented organs to generate the final result. The suggested model outperformed the state-of-the-art approaches in terms of dice similarity coefficient (DSC) values for the lungs and the heart. It is worth mentioning that the proposed model achieved a dice score of 0.941, which is 1.1% higher than the best previous dice score, in the case of the heart, an important organ in the human body. Moreover, the clinical acceptance of the results is verified using dosimetric analysis. To delineate all five organs on a CT scan of size [Formula: see text], our model takes only 8.61 s. The proposed open-source automatic contouring tool can generate accurate contours in minimal time, consequently speeding up the treatment time and reducing the treatment cost.
Collapse
|
|
14
|
Ahn JC, Qureshi TA, Singal AG, Li D, Yang JD. Deep learning in hepatocellular carcinoma: Current status and future perspectives. World J Hepatol 2021; 13:2039-2051. [PMID: 35070007 PMCID: PMC8727204 DOI: 10.4254/wjh.v13.i12.2039] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Revised: 07/19/2021] [Accepted: 11/15/2021] [Indexed: 02/06/2023] Open
Abstract
Hepatocellular carcinoma (HCC) is among the leading causes of cancer incidence and death. Despite decades of research and development of new treatment options, the overall outcomes of patients with HCC continue to remain poor. There are areas of unmet need in risk prediction, early diagnosis, accurate prognostication, and individualized treatments for patients with HCC. Recent years have seen an explosive growth in the application of artificial intelligence (AI) technology in medical research, with the field of HCC being no exception. Among the various AI-based machine learning algorithms, deep learning algorithms are considered state-of-the-art techniques for handling and processing complex multimodal data ranging from routine clinical variables to high-resolution medical images. This article will provide a comprehensive review of the recently published studies that have applied deep learning for risk prediction, diagnosis, prognostication, and treatment planning for patients with HCC.
Collapse
Affiliation(s)
- Joseph C Ahn
- Gastroenterology and Hepatology, Mayo Clinic, Rochester, MN 55904, United States
| | - Touseef Ahmad Qureshi
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States
| | - Amit G Singal
- Internal Medicine, University of Texas Southwestern Medical Center, Dallas, TX 75390, United States
| | - Debiao Li
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States
| | - Ju-Dong Yang
- Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States
| |
Collapse
|
|
15
|
Penarrubia L, Pinon N, Roux E, Dávila Serrano EE, Richard JC, Orkisz M, Sarrut D. Improving motion-mask segmentation in thoracic CT with multiplanar U-nets. Med Phys 2021; 49:420-431. [PMID: 34778978 DOI: 10.1002/mp.15347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Revised: 09/30/2021] [Accepted: 10/19/2021] [Indexed: 11/12/2022] Open
Abstract
PURPOSE Motion-mask segmentation from thoracic computed tomography (CT) images is the process of extracting the region that encompasses lungs and viscera, where large displacements occur during breathing. It has been shown to help image registration between different respiratory phases. This registration step is, for example, useful for radiotherapy planning or calculating local lung ventilation. Knowing the location of motion discontinuity, that is, sliding motion near the pleura, allows a better control of the registration preventing unrealistic estimates. Nevertheless, existing methods for motion-mask segmentation are not robust enough to be used in clinical routine. This article shows that it is feasible to overcome this lack of robustness by using a lightweight deep-learning approach usable on a standard computer, and this even without data augmentation or advanced model design. METHODS A convolutional neural-network architecture with three 2D U-nets for the three main orientations (sagittal, coronal, axial) was proposed. Predictions generated by the three U-nets were combined by majority voting to provide a single 3D segmentation of the motion mask. The networks were trained on a database of nonsmall cell lung cancer 4D CT images of 43 patients. Training and evaluation were done with a K-fold cross-validation strategy. Evaluation was based on a visual grading by two experts according to the appropriateness of the segmented motion mask for the registration task, and on a comparison with motion masks obtained by a baseline method using level sets. A second database (76 CT images of patients with early-stage COVID-19), unseen during training, was used to assess the generalizability of the trained neural network. RESULTS The proposed approach outperformed the baseline method in terms of quality and robustness: the success rate increased from 53 % to 79 % without producing any failure. It also achieved a speed-up factor of 60 with GPU, or 17 with CPU. The memory footprint was low: less than 5 GB GPU RAM for training and less than 1 GB GPU RAM for inference. When evaluated on a dataset with images differing by several characteristics (CT device, pathology, and field of view), the proposed method improved the success rate from 53 % to 83 % . CONCLUSION With 5-s processing time on a mid-range GPU and success rates around 80 % , the proposed approach seems fast and robust enough to be routinely used in clinical practice. The success rate can be further improved by incorporating more diversity in training data via data augmentation and additional annotated images from different scanners and diseases. The code and trained model are publicly available.
Collapse
Affiliation(s)
- Ludmilla Penarrubia
- Univ Lyon, Université Claude Bernard Lyon 1, INSA-Lyon, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France
| | - Nicolas Pinon
- Univ Lyon, Université Claude Bernard Lyon 1, INSA-Lyon, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France
| | - Emmanuel Roux
- Univ Lyon, Université Claude Bernard Lyon 1, INSA-Lyon, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France
| | | | - Jean-Christophe Richard
- Univ Lyon, Université Claude Bernard Lyon 1, INSA-Lyon, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France.,Service de Réanimation Médicale, Hôpital de la Croix Rousse, Hospices Civils de Lyon, France
| | - Maciej Orkisz
- Univ Lyon, Université Claude Bernard Lyon 1, INSA-Lyon, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France
| | - David Sarrut
- Univ Lyon, Université Claude Bernard Lyon 1, INSA-Lyon, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France
| |
Collapse
|
|
16
|
Zhao Z, Ma Z, Liu Y, Zeng Z, Chow PK. Multi-Slice Dense-Sparse Learning for Efficient Liver and Tumor Segmentation. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3582-3585. [PMID: 34892013 DOI: 10.1109/embc46164.2021.9629698] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Accurate automatic liver and tumor segmentation plays a vital role in treatment planning and disease monitoring. Recently, deep convolutional neural network (DCNNs) has obtained tremendous success in 2D and 3D medical image segmentation. However, 2D DCNNs cannot fully leverage the inter-slice information, while 3D DCNNs are computationally expensive and memory intensive. To address these issues, we first propose a novel dense-sparse training flow from a data perspective, in which, densely adjacent slices and sparsely adjacent slices are extracted as inputs for regularizing DCNNs, thereby improving the model performance. Moreover, we design a 2.5D light-weight nnU-Net from a network perspective, in which, depthwise separable convolutions are adopted to improve the efficiency. Extensive experiments on the LiTS dataset have demonstrated the superiority of the proposed method.Clinical relevance- The proposed method can effectively segment livers and tumors from CT scans with low complexity, which can be easily implemented into clinical practice.
Collapse
|
|
17
|
Castaldo A, De Lucia DR, Pontillo G, Gatti M, Cocozza S, Ugga L, Cuocolo R. State of the Art in Artificial Intelligence and Radiomics in Hepatocellular Carcinoma. Diagnostics (Basel) 2021; 11:1194. [PMID: 34209197 PMCID: PMC8307071 DOI: 10.3390/diagnostics11071194] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Revised: 06/24/2021] [Accepted: 06/24/2021] [Indexed: 12/12/2022] Open
Abstract
The most common liver malignancy is hepatocellular carcinoma (HCC), which is also associated with high mortality. Often HCC develops in a chronic liver disease setting, and early diagnosis as well as accurate screening of high-risk patients is crucial for appropriate and effective management of these patients. While imaging characteristics of HCC are well-defined in the diagnostic phase, challenging cases still occur, and current prognostic and predictive models are limited in their accuracy. Radiomics and machine learning (ML) offer new tools to address these issues and may lead to scientific breakthroughs with the potential to impact clinical practice and improve patient outcomes. In this review, we will present an overview of these technologies in the setting of HCC imaging across different modalities and a range of applications. These include lesion segmentation, diagnosis, prognostic modeling and prediction of treatment response. Finally, limitations preventing clinical application of radiomics and ML at the present time are discussed, together with necessary future developments to bring the field forward and outside of a purely academic endeavor.
Collapse
Affiliation(s)
- Anna Castaldo
- Department of Advanced Biomedical Sciences, University of Naples “Federico II”, 80131 Naples, Italy; (A.C.); (D.R.D.L.); (G.P.); (S.C.); (L.U.)
| | - Davide Raffaele De Lucia
- Department of Advanced Biomedical Sciences, University of Naples “Federico II”, 80131 Naples, Italy; (A.C.); (D.R.D.L.); (G.P.); (S.C.); (L.U.)
| | - Giuseppe Pontillo
- Department of Advanced Biomedical Sciences, University of Naples “Federico II”, 80131 Naples, Italy; (A.C.); (D.R.D.L.); (G.P.); (S.C.); (L.U.)
| | - Marco Gatti
- Radiology Unit, Department of Surgical Sciences, University of Turin, 10124 Turin, Italy;
| | - Sirio Cocozza
- Department of Advanced Biomedical Sciences, University of Naples “Federico II”, 80131 Naples, Italy; (A.C.); (D.R.D.L.); (G.P.); (S.C.); (L.U.)
| | - Lorenzo Ugga
- Department of Advanced Biomedical Sciences, University of Naples “Federico II”, 80131 Naples, Italy; (A.C.); (D.R.D.L.); (G.P.); (S.C.); (L.U.)
| | - Renato Cuocolo
- Department of Clinical Medicine and Surgery, University of Naples “Federico II”, 80131 Naples, Italy
| |
Collapse
|