1
|
Iqbal S, Zhong X, Khan MA, Wu Z, AlHammadi DA, Liu W, Choudhry IA. Continual and wisdom learning for federated learning: A comprehensive framework for robustness and debiasing. Inf Process Manag 2025; 62:104157. [DOI: 10.1016/j.ipm.2025.104157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/04/2025]
|
2
|
Wang T, Tang X, Du J, Jia Y, Mou W, Lu G. Establishment and evaluation of an automatic multi?sequence MRI segmentation model of primary central nervous system lymphoma based on the nnU?Net deep learning network method. Oncol Lett 2025; 30:334. [PMID: 40400535 PMCID: PMC12093087 DOI: 10.3892/ol.2025.15080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2024] [Accepted: 04/08/2025] [Indexed: 05/23/2025] Open
Abstract
Accurate quantitative assessment using gadolinium-contrast magnetic resonance imaging (MRI) is crucial in therapy planning, surveillance and prognostic assessment of primary central nervous system lymphoma (PCNSL). The present study aimed to develop a multimodal artificial intelligence deep learning segmentation model to address the challenges associated with traditional 2D measurements and manual volume assessments in MRI. Data from 49 pathologically-confirmed patients with PCNSL from six Chinese medical centers were analyzed, and regions of interest were manually segmented on contrast-enhanced T1-weighted and T2-weighted MRI scans for each patient, followed by fully automated voxel-wise segmentation of tumor components using a 3-dimenstional convolutional deep neural network. Furthermore, the efficiency of the model was evaluated using practical indicators and its consistency and accuracy was compared with traditional methods. The performance of the models were assessed using the Dice similarity coefficient (DSC). The Mann-Whitney U test was used to compare continuous clinical variables and the χ2 test was used for comparisons between categorical clinical variables. T1WI sequences exhibited the optimal performance (training dice: 0.923, testing dice: 0.830, outer validation dice: 0.801), while T2WI showed a relatively poor performance (training dice of 0.761, a testing dice of 0.647, and an outer validation dice of 0.643. In conclusion, the automatic multi-sequences MRI segmentation model for PCNSL in the present study displayed high spatial overlap ratio and similar tumor volume with routine manual segmentation, indicating its significant potential.
Collapse
Affiliation(s)
- Tao Wang
- Department of Hematology, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, P.R. China
| | - Xingru Tang
- Department of Clinical Medicine, Shanghai Jiao Tong University, Shanghai 200030, P.R. China
| | - Jun Du
- Department of Hematology, Shanghai Jiao Tong University School of Medicine Affiliated Renji Hospital, Shanghai 200001, P.R. China
| | - Yongqian Jia
- Department of Hematology, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, P.R. China
| | - Weiwei Mou
- Department of Pediatrics, Shengli Oilfield Central Hospital, Dongying, Shandong 257099, P.R. China
| | - Guang Lu
- Department of Hematology, Shandong Second Provincial General Hospital, Jinan, Shandong 250022, P.R. China
| |
Collapse
|
3
|
Zaman A, Yassin MM, Mehmud I, Cao A, Lu J, Hassan H, Kang Y. Challenges, optimization strategies, and future horizons of advanced deep learning approaches for brain lesion segmentation. Methods 2025; 239:140-168. [PMID: 40306473 DOI: 10.1016/j.ymeth.2025.04.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2025] [Revised: 04/17/2025] [Accepted: 04/24/2025] [Indexed: 05/02/2025] Open
Abstract
Brain lesion segmentation is challenging in medical image analysis, aiming to delineate lesion regions precisely. Deep learning (DL) techniques have recently demonstrated promising results across various computer vision tasks, including semantic segmentation, object detection, and image classification. This paper offers an overview of recent DL algorithms for brain tumor and stroke segmentation, drawing on literature from 2021 to 2024. It highlights the strengths, limitations, current research challenges, and unexplored areas in imaging-based brain lesion classification based on insights from over 250 recent review papers. Techniques addressing difficulties like class imbalance and multi-modalities are presented. Optimization methods for improving performance regarding computational and structural complexity and processing speed are discussed. These include lightweight neural networks, multilayer architectures, and computationally efficient, highly accurate network designs. The paper also reviews generic and latest frameworks of different brain lesion detection techniques and highlights publicly available benchmark datasets and their issues. Furthermore, open research areas, application prospects, and future directions for DL-based brain lesion classification are discussed. Future directions include integrating neural architecture search methods with domain knowledge, predicting patient survival levels, and learning to separate brain lesions using patient statistics. To ensure patient privacy, future research is anticipated to explore privacy-preserving learning frameworks. Overall, the presented suggestions serve as a guideline for researchers and system designers involved in brain lesion detection and stroke segmentation tasks.
Collapse
Affiliation(s)
- Asim Zaman
- School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen 518055, China; College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen 518118, China; Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, China
| | - Mazen M Yassin
- School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen 518055, China; College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen 518118, China
| | - Irfan Mehmud
- Department of Urology, The Third Affiliated Hospital of Shenzhen University (Luohu Hospital Group), Shenzhen University, Shenzhen 518000, China; Institute of Urology, South China Hospital, Medicine School, Shenzhen University, Shenzhen 518000, China
| | - Anbo Cao
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen 518118, China; School of Applied Technology, Shenzhen University, Shenzhen 518055, China
| | - Jiaxi Lu
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen 518118, China; School of Applied Technology, Shenzhen University, Shenzhen 518055, China
| | - Haseeb Hassan
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen 518118, China
| | - Yan Kang
- School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen 518055, China; College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen 518118, China; School of Applied Technology, Shenzhen University, Shenzhen 518055, China; Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, China; College of Medicine and Biological Information Engineering, Northeastern University, Shenyang 110169, China.
| |
Collapse
|
4
|
Akbaş CE, Ulman V, Maška M, Kozubek M. DeepFuse: A multi-rater fusion and refinement network for computing silver-standard annotations. Comput Biol Med 2025; 192:110186. [PMID: 40279971 DOI: 10.1016/j.compbiomed.2025.110186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2024] [Revised: 03/12/2025] [Accepted: 04/08/2025] [Indexed: 04/29/2025]
Abstract
Achieving a reliable and accurate biomedical image segmentation is a long-standing problem. In order to train or adapt the segmentation methods or measure their performance, reference segmentation masks are required. Usually gold-standard annotations, i.e. human-origin reference annotations, are used as reference although they are very hard to obtain. The increasing size of the acquired image data, large dimensionality such as 3D or 3D + time, limited human expert time, and annotator variability, typically result in sparsely annotated gold-standard datasets. Reliable silver-standard annotations, i.e. computer-origin reference annotations, are needed to provide dense segmentation annotations by fusing multiple computer-origin segmentation results. The produced dense silver-standard annotations can then be either used as reference annotations directly, or converted into gold-standard ones with much lighter manual curation, which saves experts' time significantly. We propose a novel full-resolution multi-rater fusion convolutional neural network (CNN) architecture for biomedical image segmentation masks, called DeepFuse, which lacks any down-sampling layers. Staying everywhere at the full resolution enables DeepFuse to fully benefit from the enormous feature extraction capabilities of CNNs. DeepFuse outperforms the popular and commonly used fusion methods, STAPLE, SIMPLE and other majority-voting-based approaches with statistical significance on a wide range of benchmark datasets as demonstrated on examples of a challenging task of 2D and 3D cell and cell nuclei instance segmentation for a wide range of microscopy modalities, magnifications, cell shapes and densities. A remarkable feature of the proposed method is that it can apply specialized post-processing to the segmentation masks of each rater separately and recover under-segmented object parts during the refinement phase even if the majority of inputs vote otherwise. Thus, DeepFuse takes a big step towards obtaining fast and reliable computer-origin segmentation annotations for biomedical images.
Collapse
Affiliation(s)
- Cem Emre Akbaş
- Masaryk University, Centre for Biomedical Image Analysis, Faculty of Informatics, Brno, 60200, Czech Republic
| | - Vladimír Ulman
- Masaryk University, Centre for Biomedical Image Analysis, Faculty of Informatics, Brno, 60200, Czech Republic; IT4Innovations, VSB - Technical University of Ostrava, Ostrava, 70800, Czech Republic
| | - Martin Maška
- Masaryk University, Centre for Biomedical Image Analysis, Faculty of Informatics, Brno, 60200, Czech Republic
| | - Michal Kozubek
- Masaryk University, Centre for Biomedical Image Analysis, Faculty of Informatics, Brno, 60200, Czech Republic.
| |
Collapse
|
5
|
Zhu H, Huang J, Chen K, Ying X, Qian Y. multiPI-TransBTS: A multi-path learning framework for brain tumor image segmentation based on multi-physical information. Comput Biol Med 2025; 191:110148. [PMID: 40215867 DOI: 10.1016/j.compbiomed.2025.110148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2024] [Revised: 04/01/2025] [Accepted: 04/04/2025] [Indexed: 04/29/2025]
Abstract
Brain Tumor Segmentation (BraTS) plays a critical role in clinical diagnosis, treatment planning, and monitoring the progression of brain tumors. However, due to the variability in tumor appearance, size, and intensity across different MRI modalities, automated segmentation remains a challenging task. In this study, we propose a novel Transformer-based framework, multiPI-TransBTS, which integrates multi-physical information to enhance segmentation accuracy. The model leverages spatial information, semantic information, and multi-modal imaging data, addressing the inherent heterogeneity in brain tumor characteristics. The multiPI-TransBTS framework consists of an encoder, an Adaptive Feature Fusion (AFF) module, and a multi-source, multi-scale feature decoder. The encoder incorporates a multi-branch architecture to separately extract modality-specific features from different MRI sequences. The AFF module fuses information from multiple sources using channel-wise and element-wise attention, ensuring effective feature recalibration. The decoder combines both common and task-specific features through a Task-Specific Feature Introduction (TSFI) strategy, producing accurate segmentation outputs for Whole Tumor (WT), Tumor Core (TC), and Enhancing Tumor (ET) regions. Comprehensive evaluations on the BraTS2019 and BraTS2020 datasets demonstrate the superiority of multiPI-TransBTS over the state-of-the-art methods. The model consistently achieves better Dice coefficients, Hausdorff distances, and Sensitivity scores, highlighting its effectiveness in addressing the BraTS challenges. Our results also indicate the need for further exploration of the balance between precision and recall in the ET segmentation task. The proposed framework represents a significant advancement in BraTS, with potential implications for improving clinical outcomes for brain tumor patients.
Collapse
Affiliation(s)
- Hongjun Zhu
- School of Software Engineering, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Engineering Research Center of Software Quality Assurance, Testing and Assessment, Chongqing, 400065, China; Key Laboratory of Big Data Intelligent Computing, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China.
| | - Jiaohang Huang
- School of Software Engineering, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China
| | - Kuo Chen
- School of Software Engineering, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Engineering Research Center of Software Quality Assurance, Testing and Assessment, Chongqing, 400065, China
| | - Xuehui Ying
- School of Software Engineering, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Engineering Research Center of Software Quality Assurance, Testing and Assessment, Chongqing, 400065, China
| | - Ying Qian
- School of Software Engineering, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China; Chongqing Engineering Research Center of Software Quality Assurance, Testing and Assessment, Chongqing, 400065, China
| |
Collapse
|
6
|
Verma A, Yadav AK. Brain tumor segmentation with deep learning: Current approaches and future perspectives. J Neurosci Methods 2025; 418:110424. [PMID: 40122469 DOI: 10.1016/j.jneumeth.2025.110424] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2024] [Revised: 02/21/2025] [Accepted: 03/09/2025] [Indexed: 03/25/2025]
Abstract
BACKGROUND Accurate brain tumor segmentation from MRI images is critical in the medical industry, directly impacts the efficacy of diagnostic and treatment plans. Accurate segmentation of tumor region can be challenging, especially when noise and abnormalities are present. METHOD This research provides a systematic review of automatic brain tumor segmentation techniques, with a specific focus on the design of network architectures. The review categorizes existing methods into unsupervised and supervised learning techniques, as well as machine learning and deep learning approaches within supervised techniques. Deep learning techniques are thoroughly reviewed, with a particular focus on CNN-based, U-Net-based, transfer learning-based, transformer-based, and hybrid transformer-based methods. SCOPE AND COVERAGE This survey encompasses a broad spectrum of automatic segmentation methodologies, from traditional machine learning approaches to advanced deep learning frameworks. It provides an in-depth comparison of performance metrics, model efficiency, and robustness across multiple datasets, particularly the BraTS dataset. The study further examines multi-modal MRI imaging and its influence on segmentation accuracy, addressing domain adaptation, class imbalance, and generalization challenges. COMPARISON WITH EXISTING METHODS The analysis highlights the current challenges in Computer-aided Diagnostic (CAD) systems, examining how different models and imaging sequences impact performance. Recent advancements in deep learning, especially the widespread use of U-Net architectures, have significantly enhanced medical image segmentation. This review critically evaluates these developments, focusing the iterative improvements in U-Net models that have driven progress in brain tumor segmentation. Furthermore, it explores various techniques for improving U-Net performance for medical applications, focussing on its potential for improving diagnostic and treatment planning procedures. CONCLUSION The efficiency of these automated segmentation approaches is rigorously evaluated using the BraTS dataset, a benchmark dataset, part of the annual Multimodal Brain Tumor Segmentation Challenge (MICCAI). This evaluation provides insights into the current state-of-the-art and identifies key areas for future research and development.
Collapse
Affiliation(s)
- Akash Verma
- Department of Computer Science & Engineering, NIT Hamirpur (HP), India.
| | - Arun Kumar Yadav
- Department of Computer Science & Engineering, NIT Hamirpur (HP), India.
| |
Collapse
|
7
|
Yu M, Xu Z, Lukasiewicz T. A general survey on medical image super-resolution via deep learning. Comput Biol Med 2025; 193:110345. [PMID: 40412085 DOI: 10.1016/j.compbiomed.2025.110345] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2023] [Revised: 01/30/2025] [Accepted: 05/03/2025] [Indexed: 05/27/2025]
Abstract
Medical image super-resolution (SR) is a classic regression task in low-level vision. Limited by hardware limitations, acquisition time, low radiation dose, and other factors, the spatial resolution of some medical images is not sufficient. To address this problem, many different SR methods have been proposed. Especially in recent years, medical image SR networks based on deep learning have been vigorously developed. This survey provides a modular and detailed introduction to the key components of medical image SR technology based on deep learning. In this paper, we first introduce the background concepts of deep learning and medical image SR task. Subsequently, we present a comprehensive analysis of the key components from the perspectives of effective architecture, upsampling module, learning strategy, and image quality assessment of medical image SR networks. Furthermore, we focus on the urgent problems that need to be addressed in the medical image SR task based on deep learning. And finally we summarize the trends and challenges of future development.
Collapse
Affiliation(s)
- Miao Yu
- Public Health Sciences and Engineering College, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Zhenghua Xu
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, China.
| | - Thomas Lukasiewicz
- Institute of Logic and Computation, TU Wien, Vienna, Austria; Department of Computer Science, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
8
|
Zhang M, Sun Q, Han Y, Zhang M, Wang W, Zhang J. Generative adversarial DacFormer network for MRI brain tumor segmentation. Sci Rep 2025; 15:17840. [PMID: 40404794 PMCID: PMC12098717 DOI: 10.1038/s41598-025-02714-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2025] [Accepted: 05/15/2025] [Indexed: 05/24/2025] Open
Abstract
Current brain tumor segmentation methods often utilize a U-Net architecture based on efficient convolutional neural networks. While effective, these architectures primarily model local dependencies, lacking the ability to capture global interactions like pure Transformer. However, using pure Transformer directly causes the network to lose local feature information. To address this limitation, we propose the Generative Adversarial Dilated Attention Convolutional Transformer(GDacFormer). GDacFormer enhances interactions between tumor regions while balancing global and local information through the integration of adversarial learning with an improved transformer module. Specifically, GDacFormer leverages a generative adversarial segmentation network to learn richer and more detailed features. It integrates a novel Transformer module, DacFormer, featuring multi-scale dilated attention and a next convolution block. This module, embedded within the generator, aggregates semantic multi-scale information, efficiently reduces the redundancy in the self-attention mechanism, and enhances local feature representations, thus refining the brain tumor segmentation results. GDacFormer achieves Dice values for whole tumor, core tumor, and enhancing tumor segmentation of 90.9%/90.8%/93.7%, 84.6%/85.7%/93.5%, and 77.9%/79.3%/86.3% on BraTS2019-2021 datasets. Extensive evaluations demonstrate the effectiveness and competitiveness of GDacFormer. The code for GDacFormer will be made publicly available at https://github.com/MuqinZ/GDacFormer.
Collapse
Affiliation(s)
- Muqing Zhang
- School of Computer Science and Engineering, Dalian Minzu University, Dalian, 116600, China
| | - Qiule Sun
- School of Computer Science and Technology, Dalian University of Technology, Dalian, 116024, China
| | - Yutong Han
- School of Computer Science and Engineering, Dalian Minzu University, Dalian, 116600, China
| | - Mingli Zhang
- Montreal Neurological Institute, Quebec, 116024, Canada
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, 264005, China
| | - Wei Wang
- School of Computer Science and Engineering, Dalian Minzu University, Dalian, 116600, China
| | - Jianxin Zhang
- School of Computer Science and Engineering, Dalian Minzu University, Dalian, 116600, China.
| |
Collapse
|
9
|
Delgado-López PD, Cárdenas Montes M, Troya García J, Ocaña-Tienda B, Cepeda S, Martínez Martínez R, Corrales-García EM. Artificial intelligence in neuro-oncology: methodological bases, practical applications and ethical and regulatory issues. Clin Transl Oncol 2025:10.1007/s12094-025-03948-4. [PMID: 40402414 DOI: 10.1007/s12094-025-03948-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2025] [Accepted: 04/26/2025] [Indexed: 05/23/2025]
Abstract
Artificial Intelligence (AI) is transforming neuro-oncology by enhancing diagnosis, treatment planning, and prognosis prediction. AI-driven approaches-such as CNNs and deep learning-have improved the detection and classification of brain tumors through advanced imaging techniques and genomic analysis. Explainable AI methods mitigate the "black box" problem, promoting model transparency and clinical trust. Mechanistic models complement AI by integrating biological principles, enabling precise tumor growth predictions and treatment response assessments. AI applications also include the creation of digital twins for personalized therapy optimization, virtual clinical trials, and predictive modeling for estimation of tumor resection and pattern of recurrence. However, challenges such as data bias, ethical concerns, and regulatory compliance persist. The European Artificial Intelligence Act and the Health Data Space Regulation impose strict data protection and transparency requirements. This review explores AI's methodological foundations, clinical applications, and ethical challenges in neuro-oncology, emphasizing the need for interdisciplinary collaboration and regulatory adaptation.
Collapse
Affiliation(s)
- Pedro David Delgado-López
- Servicio de Neurocirugía, Hospital Universitario de Burgos, Avda Islas Baleares 3, 09006, Burgos, Spain.
| | - Miguel Cárdenas Montes
- Departamento de Investigación Básica, Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas (CIEMAT), Madrid, Spain
| | - Jesús Troya García
- Servicio de Medicina Interna, Hospital Universitario Infanta Leonor, Madrid, Spain
| | - Beatriz Ocaña-Tienda
- Centro Nacional de Investigaciones Oncológicas (CNIO), Unidad de Bioinformática, Madrid, Spain
| | - Santiago Cepeda
- Servicio de Neurocirugía, Hospital Universitario Rio Hortega, Valladolid, Spain
- Grupo Especializado en Imagen Biomédica y Análisis Computacional (GEIBAC), Instituto de Investigación Biosanitaria de Valladolid (IBioVall), Valladolid, Spain
| | - Ricard Martínez Martínez
- Facultad de Derecho, Cátedra de Privacidad y Transformación Digital de la Universidad de Valencia, Valencia, Spain
| | | |
Collapse
|
10
|
Al-Absi AA, Fu R, Ebrahim N, Al-Absi MA, Kang DK. Brain Tumour Segmentation and Grading Using Local and Global Context-Aggregated Attention Network Architecture. Bioengineering (Basel) 2025; 12:552. [PMID: 40428171 DOI: 10.3390/bioengineering12050552] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2025] [Revised: 05/07/2025] [Accepted: 05/12/2025] [Indexed: 05/29/2025] Open
Abstract
Brain tumours (BTs) are among the most dangerous and life-threatening cancers in humans of all ages, and the early detection of BTs can make a huge difference to their treatment. However, grade recognition is a challenging issue for radiologists involved in automated diagnosis and healthcare monitoring. Recent research has been motivated by the search for deep learning-based mechanisms for segmentation and grading to assist radiologists in diagnostic analysis. Segmentation refers to the identification and delineation of tumour regions in medical images, while classification classifies based on tumour characteristics, such as the size, location and enhancement pattern. The main aim of this research is to design and develop an intelligent model that can detect and grade tumours more effectively. This research develops an aggregated architecture called LGCNet, which combines a local context attention network and a global context attention network. LGCNet makes use of information extracted through the task, dimension and scale. Specifically, a global context attention network is developed for capturing multiple-scale features, and a local context attention network is designed for specific tasks. Thereafter, both networks are aggregated, and the learning network is designed to balance all the tasks by combining the loss functions of the classification and segmentation. The main advantage of LGCNet is its dedicated network for a specific task. The proposed model is evaluated by considering the BraTS2019 dataset with different metrics, such as the Dice score, sensitivity, specificity and Hausdorff score. Comparative analysis with the existing model shows marginal improvement and provides scope for further research into BT segmentation and classification. The scope of this study focuses on the BraTS2019 dataset, with future work aiming to extend the applicability of the model to different clinical and imaging environments.
Collapse
Affiliation(s)
- Ahmed Abdulhakim Al-Absi
- Department of Smart Computing, Kyungdong University, 46 4-gil, Bongpo, Gosung 24764, Republic of Korea
| | - Rui Fu
- College of Language Intelligence, Language & Brain Research Center, Sichuan International Studies University, Chongqing 400031, China
| | - Nadhem Ebrahim
- Department of Computer Science, College of Engineering and Polymer Science, University of Akron Ohio, Akron, OH 44325, USA
| | | | - Dae-Ki Kang
- Department of Computer & Information Engineering, Dongseo University, 47 Jurye-ro, Sasang-gu, Busan 47011, Republic of Korea
| |
Collapse
|
11
|
Li Z, Luo S, Li H, Li Y. DCSLK: Combined large kernel shared convolutional model with dynamic channel Sampling. Neuroimage 2025; 315:121280. [PMID: 40403943 DOI: 10.1016/j.neuroimage.2025.121280] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2024] [Revised: 05/14/2025] [Accepted: 05/16/2025] [Indexed: 05/24/2025] Open
Abstract
This study centers around the competition between Convolutional Neural Networks (CNNs) with large convolutional kernels and Vision Transformers in the domain of computer vision, delving deeply into the issues pertaining to parameters and computational complexity that stem from the utilization of large convolutional kernels. Even though the size of the convolutional kernels has been extended up to 51×51, the enhancement of performance has hit a plateau, and moreover, striped convolution incurs a performance degradation. Enlightened by the hierarchical visual processing mechanism inherent in humans, this research innovatively incorporates a shared parameter mechanism for large convolutional kernels. It synergizes the expansion of the receptive field enabled by large convolutional kernels with the extraction of fine-grained features facilitated by small convolutional kernels. To address the surging number of parameters, a meticulously designed parameter sharing mechanism is employed, featuring fine-grained processing in the central region of the convolutional kernel and wide-ranging parameter sharing in the periphery. This not only curtails the parameter count and mitigates the model complexity but also sustains the model's capacity to capture extensive spatial relationships. Additionally, in light of the problems of spatial feature information loss and augmented memory access during the 1 × 1 convolutional channel compression phase, this study further puts forward a dynamic channel sampling approach, which markedly elevates the accuracy of tumor subregion segmentation. To authenticate the efficacy of the proposed methodology, a comprehensive evaluation has been conducted on three brain tumor segmentation datasets, namely BraTs2020, BraTs2024, and Medical Segmentation Decathlon Brain 2018. The experimental results evince that the proposed model surpasses the current mainstream ConvNet and Transformer architectures across all performance metrics, proffering novel research perspectives and technical stratagems for the realm of medical image segmentation.
Collapse
Affiliation(s)
- Zongren Li
- Doctoral Candidate, Information Security, Medical Image Segmentation, Xinjiang University, Urumqi, China.
| | - Shuping Luo
- Department of Gastroenterology, Joint Logistics Support Force of the Chinese People's Liberation Army, Multi-Modal Medical Image Segmentation, No 940 Hospital, Lanzhou, China.
| | - Hongwei Li
- Information Department, Bio-information Security, Joint Logistics Support Force of the Chinese People's Liberation Army, No 940 Hospital, Lanzhou, China.
| | - Yanbin Li
- School of Computer Science and Technology, Medical Image Segmentation, Xinjiang University, Urumqi, China.
| |
Collapse
|
12
|
Gomaa A, Huang Y, Stephan P, Breininger K, Frey B, Dörfler A, Schnell O, Delev D, Coras R, Donaubauer AJ, Schmitter C, Stritzelberger J, Semrau S, Maier A, Bayer S, Schönecker S, Heiland DH, Hau P, Gaipl US, Bert C, Fietkau R, Schmidt MA, Putz F. A self-supervised multimodal deep learning approach to differentiate post-radiotherapy progression from pseudoprogression in glioblastoma. Sci Rep 2025; 15:17133. [PMID: 40382400 DOI: 10.1038/s41598-025-02026-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2024] [Accepted: 05/08/2025] [Indexed: 05/20/2025] Open
Abstract
Accurate differentiation of pseudoprogression (PsP) from True Progression (TP) following radiotherapy (RT) in glioblastoma patients is crucial for optimal treatment planning. However, this task remains challenging due to the overlapping imaging characteristics of PsP and TP. This study therefore proposes a multimodal deep-learning approach utilizing complementary information from routine anatomical MR images, clinical parameters, and RT treatment planning information for improved predictive accuracy. The approach utilizes a self-supervised Vision Transformer (ViT) to encode multi-sequence MR brain volumes to effectively capture both global and local context from the high dimensional input. The encoder is trained in a self-supervised upstream task on unlabeled glioma MRI datasets from the open BraTS2021, UPenn-GBM, and UCSF-PDGM datasets (n = 2317 MRI studies) to generate compact, clinically relevant representations from FLAIR and T1 post-contrast sequences. These encoded MR inputs are then integrated with clinical data and RT treatment planning information through guided cross-modal attention, improving progression classification accuracy. This work was developed using two datasets from different centers: the Burdenko Glioblastoma Progression Dataset (n = 59) for training and validation, and the GlioCMV progression dataset from the University Hospital Erlangen (UKER) (n = 20) for testing. The proposed method achieved competitive performance, with an AUC of 75.3%, outperforming the current state-of-the-art data-driven approaches. Importantly, the proposed approach relies solely on readily available anatomical MRI sequences, clinical data, and RT treatment planning information, enhancing its clinical feasibility. The proposed approach addresses the challenge of limited data availability for PsP and TP differentiation and could allow for improved clinical decision-making and optimized treatment plans for glioblastoma patients.
Collapse
Affiliation(s)
- Ahmed Gomaa
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany.
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany.
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany.
| | - Yixing Huang
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
| | - Pluvio Stephan
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
| | - Katharina Breininger
- Center for Artificial Intelligence and Data Science, Universität Würzburg, Würzburg, 97074, Germany
| | - Benjamin Frey
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
| | - Arnd Dörfler
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Institute of Neuroradiology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
| | - Oliver Schnell
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Department of Neurosurgery, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
| | - Daniel Delev
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Department of Neurosurgery, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
| | - Roland Coras
- Department of Neurosurgery, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
| | - Anna-Jasmina Donaubauer
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
| | - Charlotte Schmitter
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
| | - Jenny Stritzelberger
- Department of Neurology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
| | - Sabine Semrau
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
| | - Andreas Maier
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Siming Bayer
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Stephan Schönecker
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
- Department of Radiation Oncology, University Hospital Ludwig Maximilian University of Munich, 81377, Munich, Germany
| | - Dieter H Heiland
- Translational Neurosurgery, Alexander-Friedrich-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Department of Neurosurgery, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, 60611, USA
| | - Peter Hau
- Department of Neurology, University Hospital Regensburg, Regensburg, Germany
- Wilhelm Sander-NeuroOncology Unit, University Hospital Regensburg, Regensburg, Germany
| | - Udo S Gaipl
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
| | - Christoph Bert
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
| | - Rainer Fietkau
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
| | - Manuel A Schmidt
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
- Institute of Neuroradiology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
| | - Florian Putz
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91054, Germany
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, 91054, Germany
- Bavarian Cancer Research Center (BZKF), Erlangen, 91052, Germany
| |
Collapse
|
13
|
Akbari H, Bakas S, Sako C, Fathi Kazerooni A, Villanueva-Meyer J, Garcia JA, Mamourian E, Liu F, Cao Q, Shinohara RT, Baid U, Getka A, Pati S, Singh A, Calabrese E, Chang S, Rudie J, Sotiras A, LaMontagne P, Marcus DS, Milchenko M, Nazeri A, Balana C, Capellades J, Puig J, Badve C, Barnholtz-Sloan JS, Sloan AE, Vadmal V, Waite K, Ak M, Colen RR, Park YW, Ahn SS, Chang JH, Choi YS, Lee SK, Alexander GS, Ali AS, Dicker AP, Flanders AE, Liem S, Lombardo J, Shi W, Shukla G, Griffith B, Poisson LM, Rogers LR, Kotrotsou A, Booth TC, Jain R, Lee M, Mahajan A, Chakravarti A, Palmer JD, DiCostanzo D, Fathallah-Shaykh H, Cepeda S, Santonocito OS, Di Stefano AL, Wiestler B, Melhem ER, Woodworth GF, Tiwari P, Valdes P, Matsumoto Y, Otani Y, Imoto R, Aboian M, Koizumi S, Kurozumi K, Kawakatsu T, Alexander K, Satgunaseelan L, Rulseh AM, Bagley SJ, Bilello M, Binder ZA, Brem S, Desai AS, Lustig RA, Maloney E, Prior T, Amankulor N, Nasrallah MP, O’Rourke DM, Mohan S, Davatzikos C. Machine learning-based prognostic subgrouping of glioblastoma: A multicenter study. Neuro Oncol 2025; 27:1102-1115. [PMID: 39665363 PMCID: PMC12083074 DOI: 10.1093/neuonc/noae260] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2024] [Indexed: 12/13/2024] Open
Abstract
BACKGROUND Glioblastoma (GBM) is the most aggressive adult primary brain cancer, characterized by significant heterogeneity, posing challenges for patient management, treatment planning, and clinical trial stratification. METHODS We developed a highly reproducible, personalized prognostication, and clinical subgrouping system using machine learning (ML) on routine clinical data, magnetic resonance imaging (MRI), and molecular measures from 2838 demographically diverse patients across 22 institutions and 3 continents. Patients were stratified into favorable, intermediate, and poor prognostic subgroups (I, II, and III) using Kaplan-Meier analysis (Cox proportional model and hazard ratios [HR]). RESULTS The ML model stratified patients into distinct prognostic subgroups with HRs between subgroups I-II and I-III of 1.62 (95% CI: 1.43-1.84, P < .001) and 3.48 (95% CI: 2.94-4.11, P < .001), respectively. Analysis of imaging features revealed several tumor properties contributing unique prognostic value, supporting the feasibility of a generalizable prognostic classification system in a diverse cohort. CONCLUSIONS Our ML model demonstrates extensive reproducibility and online accessibility, utilizing routine imaging data rather than complex imaging protocols. This platform offers a unique approach to personalized patient management and clinical trial stratification in GBM.
Collapse
Affiliation(s)
- Hamed Akbari
- Department of Bioengineering, School of Engineering, Santa Clara University, Santa Clara, California, USA
| | - Spyridon Bakas
- Department of Pathology & Laboratory Medicine, Indiana University School of Medicine, Indianapolis, Indiana, USA
- Department of Radiology and Imaging Sciences, Indiana University School of Medicine, Indianapolis, Indiana, USA
- Department of Neurological Surgery, Indiana University School of Medicine, Indianapolis, Indiana, USA
- Department of Computer Science, Luddy School of Informatics, Computing, and Engineering, Indiana University, Indianapolis, Indiana, USA
| | - Chiharu Sako
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Anahita Fathi Kazerooni
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Center for Data-Driven Discovery in Biomedicine (D3b), Division of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Javier Villanueva-Meyer
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, California, USA
| | - Jose A Garcia
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Elizabeth Mamourian
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Fang Liu
- Biostatistics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Quy Cao
- Biostatistics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Russell T Shinohara
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Biostatistics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Ujjwal Baid
- Department of Pathology & Laboratory Medicine, Indiana University School of Medicine, Indianapolis, Indiana, USA
- Department of Radiology and Imaging Sciences, Indiana University School of Medicine, Indianapolis, Indiana, USA
| | - Alexander Getka
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Sarthak Pati
- Department of Pathology & Laboratory Medicine, Indiana University School of Medicine, Indianapolis, Indiana, USA
| | - Ashish Singh
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Evan Calabrese
- Department of Radiology, Duke University, Durham, North Carolina, USA
| | - Susan Chang
- Department of Neurological Surgery, University of California San Francisco, San Francisco, California, USA
| | - Jeffrey Rudie
- Department of Radiology, University of California San Diego, San Diego, California, USA
| | - Aristeidis Sotiras
- Department of Radiology, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Pamela LaMontagne
- Department of Radiology, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Daniel S Marcus
- Department of Radiology, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Mikhail Milchenko
- Department of Radiology, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Arash Nazeri
- Department of Radiology, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Carmen Balana
- B-ARGO Group, Institut Investigació Germans Trias i Pujol (IGTP), Badalona (Barcelona), Catalonia, Spain
| | - Jaume Capellades
- Research Unit (IDIR), Image Diagnosis Institute, Badalona, Spain
| | - Josep Puig
- Department of Radiology (CDI), Hospital Clínic and IDIBAPS, Barcelona, Spain
| | - Chaitra Badve
- Department of Radiology, Case Western Reserve University and University Hospitals of Cleveland, Cleveland, Ohio, USA
| | - Jill S Barnholtz-Sloan
- Trans-Divisional Research Program (TDRP), Division of Cancer Epidemiology and Genetics (DCEG), National Cancer Institute, Bethesda, Maryland, USA
- Central Brain Tumor Registry of the United States, Hinsdale, Illinois, USA
- Center for Biomedical Informatics and Information Technology (CBIIT), National Cancer Institute, Bethesda, Maryland, USA
| | - Andrew E Sloan
- Brain and Tumor Neurosurgery, Neurosurgical Oncology, Piedmont Health, Atlanta, Georgia, USA
- Seidman Cancer Center and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio, USA
| | - Vachan Vadmal
- Department of Population and Quantitative Health Sciences, Case Western Reserve University School of Medicine, Cleveland, Ohio, USA
| | - Kristin Waite
- Center for Biomedical Informatics and Information Technology (CBIIT), National Cancer Institute, Bethesda, Maryland, USA
- Department of Population and Quantitative Health Sciences, Case Western Reserve University School of Medicine, Cleveland, Ohio, USA
| | - Murat Ak
- Department of Radiology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Rivka R Colen
- Department of Radiology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
- Hillman Cancer Center, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania, USA
| | - Yae Won Park
- Department of Radiology, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Sung Soo Ahn
- Department of Radiology, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Jong Hee Chang
- Department of Neurosurgery, Yonsei University College of Medicine, Seoul, Republic of Korea
- Brain Tumor Center, Severance Hospital, Yonsei University Health System, Seoul, Republic of Korea
| | - Yoon Seong Choi
- Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- Clinical Imaging Research Centre, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Seung-Koo Lee
- Department of Radiology, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Gregory S Alexander
- Department of Radiation Oncology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Ayesha S Ali
- Department of Radiation Oncology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Adam P Dicker
- Department of Radiation Oncology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Adam E Flanders
- Department of Radiology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Spencer Liem
- Department of Radiation Oncology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Joseph Lombardo
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
- Department of Radiation Oncology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Wenyin Shi
- Department of Radiation Oncology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Gaurav Shukla
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Radiation Oncology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
- Department of Radiation Oncology, Christiana Care Health System, Philadelphia, Pennsylvania, USA
| | - Brent Griffith
- Department of Radiology, Henry Ford Health System, Detroit, Michigan, USA
| | - Laila M Poisson
- Department of Public Health Sciences, Center for Bioinformatics, Henry Ford Health System, Detroit, Michigan, USA
- Department of Neurosurgery, Hermelin Brain Tumor Center, Henry Ford Cancer Institute, Henry Ford Health, Detroit, Michigan, USA
| | - Lisa R Rogers
- Department of Neurosurgery, Hermelin Brain Tumor Center, Henry Ford Cancer Institute, Henry Ford Health, Detroit, Michigan, USA
| | | | - Thomas C Booth
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK
- Department of Neuroradiology, Ruskin Wing, King’s College Hospital NHS Foundation Trust, London, UK
| | - Rajan Jain
- Department of Radiology, New York University Langone Health, New York, New York, USA
- Department of Neurosurgery, New York University Langone Health, New York, New York, USA
| | - Matthew Lee
- Department of Radiology, New York University Langone Health, New York, New York, USA
| | - Abhishek Mahajan
- Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, India
- The Clatterbridge Cancer Centre NHS Foundation Trust, Liverpool, UK
| | - Arnab Chakravarti
- Department of Radiation Oncology, The James Cancer Hospital at the Ohio State University Wexner Medical Center, Columbus, Ohio, USA
| | - Joshua D Palmer
- Department of Radiation Oncology, The James Cancer Hospital at the Ohio State University Wexner Medical Center, Columbus, Ohio, USA
| | - Dominic DiCostanzo
- Department of Radiation Oncology, The James Cancer Hospital at the Ohio State University Wexner Medical Center, Columbus, Ohio, USA
| | | | - Santiago Cepeda
- Department of Neurosurgery, University Hospital Río Hortega, Valladolid, Spain
| | - Orazio Santo Santonocito
- Division of Neurosurgery, Spedali Riuniti di Livorno-Azienda USL Toscana Nord-Ovest, Livorno, Italy
| | - Anna Luisa Di Stefano
- Division of Neurosurgery, Spedali Riuniti di Livorno-Azienda USL Toscana Nord-Ovest, Livorno, Italy
| | - Benedikt Wiestler
- Department of Neuroradiology, Technical University of Munich, Munchen, Germany
| | - Elias R Melhem
- Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, Maryland, USA
| | - Graeme F Woodworth
- Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, Maryland, USA
- Department of Neurosurgery, University of Maryland School of Medicine, Baltimore, Maryland, USA
| | - Pallavi Tiwari
- Department of Radiology, University of Wisconsin, Madison, Wisconsin, USA
- Department of Biomedical Engineering, University of Wisconsin, Madison, Wisconsin, USA
| | - Pablo Valdes
- Department of Neurosurgery, University of Texas Medical Branch, Galveston, Texas, USA
| | - Yuji Matsumoto
- Department of Neurological Surgery, Okayama University, Okayama, Japan
| | - Yoshihiro Otani
- Department of Neurological Surgery, Okayama University, Okayama, Japan
| | - Ryoji Imoto
- Department of Neurological Surgery, Okayama University, Okayama, Japan
| | - Mariam Aboian
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Shinichiro Koizumi
- Department of Neurosurgery, Hamamatsu University School of Medicine, Hamamatsu, Shizuoka, Japan
| | - Kazuhiko Kurozumi
- Department of Neurosurgery, Hamamatsu University School of Medicine, Hamamatsu, Shizuoka, Japan
| | - Toru Kawakatsu
- Department of Neurosurgery, Hamamatsu University School of Medicine, Hamamatsu, Shizuoka, Japan
| | - Kimberley Alexander
- Department of Neurosurgery, Chris O’Brien Lifehouse, Camperdown, Australia
- Faculty of Medicine and Health, University of Sydney, Camperdown, Australia
- Department of Neuropathology, Royal Prince Alfred Hospital, Camperdown, Australia
| | - Laveniya Satgunaseelan
- Department of Neurosurgery, Chris O’Brien Lifehouse, Camperdown, Australia
- Faculty of Medicine and Health, University of Sydney, Camperdown, Australia
- Department of Neuropathology, Royal Prince Alfred Hospital, Camperdown, Australia
| | - Aaron M Rulseh
- Department of Radiology, Na Homolce Hospital, Prague, Czechia
| | - Stephen J Bagley
- Department of Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- GBM Translational Center of Excellence, Abramson Cancer Center, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Michel Bilello
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Zev A Binder
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- GBM Translational Center of Excellence, Abramson Cancer Center, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Steven Brem
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- GBM Translational Center of Excellence, Abramson Cancer Center, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Arati S Desai
- GBM Translational Center of Excellence, Abramson Cancer Center, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Robert A Lustig
- Department of Radiation-Oncology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Eileen Maloney
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Timothy Prior
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Nduka Amankulor
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- GBM Translational Center of Excellence, Abramson Cancer Center, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - MacLean P Nasrallah
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- GBM Translational Center of Excellence, Abramson Cancer Center, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Pathology & Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Donald M O’Rourke
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- GBM Translational Center of Excellence, Abramson Cancer Center, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Suyash Mohan
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Christos Davatzikos
- Center for Data Science and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Statistics in Imaging and Visualization Center, and Center for Clinical Epidemiology ce and AI for Integrated Diagnostics (AI2D), and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
14
|
El Jurdi R, Varoquaux G, Colliot O. Confidence intervals for performance estimates in brain MRI segmentation. Med Image Anal 2025; 103:103565. [PMID: 40367699 DOI: 10.1016/j.media.2025.103565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2024] [Revised: 03/18/2025] [Accepted: 03/23/2025] [Indexed: 05/16/2025]
Abstract
Medical segmentation models are evaluated empirically. As such an evaluation is based on a limited set of example images, it is unavoidably noisy. Beyond a mean performance measure, reporting confidence intervals is thus crucial. However, this is rarely done in medical image segmentation. The width of the confidence interval depends on the test set size and on the spread of the performance measure (its standard-deviation across the test set). For classification, many test images are needed to avoid wide confidence intervals. Segmentation, however, has not been studied, and it differs by the amount of information brought by a given test image. In this paper, we study the typical confidence intervals in the context of segmentation in 3D brain magnetic resonance imaging (MRI). We carry experiments on using the standard nnU-net framework, two datasets from the Medical Decathlon challenge that concern brain MRI (hippocampus and brain tumor segmentation) and two performance measures: the Dice Similarity Coefficient and the Hausdorff distance. We show that the parametric confidence intervals are reasonable approximations of the bootstrap estimates for varying test set sizes and spread of the performance metric. Importantly, we show that the test size needed to achieve a given precision is often much lower than for classification tasks. Typically, a 1% wide confidence interval requires about 100-200 test samples when the spread is low (standard-deviation around 3%). More difficult segmentation tasks may lead to higher spreads and require over 1000 samples. The corresponding code and notebooks are available on GitHub at https://github.com/rosanajurdi/SegVal_Repo.
Collapse
Affiliation(s)
- Rosana El Jurdi
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié-Salpêtrière, F-75013, Paris, France.
| | | | - Olivier Colliot
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié-Salpêtrière, F-75013, Paris, France
| |
Collapse
|
15
|
Nie Z, Yang J, Li C, Wang Y, Tang J. DiffBTS: A Lightweight Diffusion Model for 3D Multimodal Brain Tumor Segmentation. SENSORS (BASEL, SWITZERLAND) 2025; 25:2985. [PMID: 40431779 DOI: 10.3390/s25102985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2025] [Revised: 04/09/2025] [Accepted: 04/22/2025] [Indexed: 05/29/2025]
Abstract
Denoising diffusion probabilistic models (DDPMs) have achieved remarkable success across various research domains. However, their high complexity when processing 3D images remains a limitation. To mitigate this, researchers typically preprocess data into 2D slices, enabling the model to perform segmentation in a reduced 2D space. This paper introduces DiffBTS, an end-to-end, lightweight diffusion model specifically designed for 3D brain tumor segmentation. DiffBTS replaces the conventional self-attention module in the traditional diffusion models by introducing an efficient 3D self-attention mechanism. The mechanism is applied between down-sampling and jump connections in the model, allowing it to capture long-range dependencies and global semantic information more effectively. This design prevents computational complexity from growing in square steps. Prediction accuracy and model stability are crucial in brain tumor segmentation; we propose the Edge-Blurring Guided (EBG) algorithm, which directs the diffusion model to focus more on the accuracy of segmentation boundaries during the iterative sampling process. This approach enhances prediction accuracy and stability. To assess the performance of DiffBTS, we compared it with seven state-of-the-art models on the BraTS 2020 and BraTS 2021 datasets. DiffBTS achieved an average Dice score of 89.99 and an average HD95 value of 1.928 mm on BraTS2021 and 86.44 and 2.466 mm on BraTS2020, respectively. Extensive experimental results demonstrate that DiffBTS achieves state-of-the-art performance in brain tumor segmentation, outperforming all competing models.
Collapse
Affiliation(s)
- Zuxin Nie
- College of Information Science and Engineering, Hunan Normal University, No. 36, Lushan Road, Changsha 410081, China
| | - Jiahong Yang
- College of Information Science and Engineering, Hunan Normal University, No. 36, Lushan Road, Changsha 410081, China
| | - Chengxuan Li
- College of Information Science and Engineering, Hunan Normal University, No. 36, Lushan Road, Changsha 410081, China
| | - Yaqin Wang
- College of Information Science and Engineering, Hunan Normal University, No. 36, Lushan Road, Changsha 410081, China
| | - Jun Tang
- School of Educational Sciences, Hunan Normal University, No. 36, Lushan Road, Changsha 410081, China
| |
Collapse
|
16
|
Zhang J, LaBella D, Zhang D, Houk JL, Rudie JD, Zou H, Warman P, Mazurowski MA, Calabrese E. Development and Evaluation of Automated Artificial Intelligence-Based Brain Tumor Response Assessment in Patients with Glioblastoma. AJNR Am J Neuroradiol 2025; 46:990-998. [PMID: 39542725 DOI: 10.3174/ajnr.a8580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2024] [Accepted: 10/19/2024] [Indexed: 11/17/2024]
Abstract
This project aimed to develop and evaluate an automated, AI-based, volumetric brain tumor MRI response assessment algorithm on a large cohort of patients treated at a high-volume brain tumor center. We retrospectively analyzed data from 634 patients treated for glioblastoma at a single brain tumor center over a 5-year period (2017-2021). The mean age was 56 ± 13 years. 372/634 (59%) patients were male, and 262/634 (41%) patients were female. Study data consisted of 3,403 brain MRI exams and corresponding standardized, radiologist-based brain tumor response assessments (BT-RADS). An artificial intelligence (AI)-based brain tumor response assessment (AI-VTRA) algorithm was developed using automated, volumetric tumor segmentation. AI-VTRA results were evaluated for agreement with radiologist-based response assessments and ability to stratify patients by overall survival. Metrics were computed to assess the agreement using BT-RADS as the ground-truth, fixed-time point survival analysis was conducted to evaluate the survival stratification, and associated P-values were calculated. For all BT-RADS categories, AI-VTRA showed moderate agreement with radiologist response assessments (F1 = 0.587-0.755). Kaplan-Meier survival analysis revealed statistically worse overall fixed time point survival for patients assessed as image worsening equivalent to RANO progression by human alone compared to by AI alone (log-rank P = .007). Cox proportional hazard model analysis showed a disadvantage to AI-based assessments for overall survival prediction (P = .012). In summary, our proposed AI-VTRA, following BT-RADS criteria, yielded moderate agreement for replicating human response assessments and slightly worse stratification by overall survival.
Collapse
Affiliation(s)
- Jikai Zhang
- From the Departments of Electrical and Computer Engineering (J.Z., M.A.M.), Duke University, Durham, North Carolina
- Duke Center for Artificial Intelligence in Radiology (J.Z., E.C.), Duke University Medical Center, Durham, North Carolina
| | - Dominic LaBella
- Departments of Radiation Oncology (D.L.), Duke University Medical Center, Durham, North Carolina
| | - Dylan Zhang
- Departments of Radiology (D.Z., J.L.H., M.A.M., E.C.), Duke University Medical Center, Durham, North Carolina
| | - Jessica L Houk
- Departments of Radiology (D.Z., J.L.H., M.A.M., E.C.), Duke University Medical Center, Durham, North Carolina
| | - Jeffrey D Rudie
- Department of Radiology (J.D.R.), University of California San Diego, San Diego, California
| | - Haotian Zou
- Department of Biostatistics and Bioinformatics (H.Z., M.A.M.), Duke University School of Medicine, Durham, North Carolina
| | - Pranav Warman
- Duke University School of Medicine(P.W.), Durham, North Carolina
| | - Maciej A Mazurowski
- From the Departments of Electrical and Computer Engineering (J.Z., M.A.M.), Duke University, Durham, North Carolina
- Department of Computer Science (M.A.M.), Duke University, Durham, North Carolina
- Department of Biostatistics and Bioinformatics (H.Z., M.A.M.), Duke University School of Medicine, Durham, North Carolina
- Departments of Radiology (D.Z., J.L.H., M.A.M., E.C.), Duke University Medical Center, Durham, North Carolina
| | - Evan Calabrese
- Duke Center for Artificial Intelligence in Radiology (J.Z., E.C.), Duke University Medical Center, Durham, North Carolina
- Departments of Radiology (D.Z., J.L.H., M.A.M., E.C.), Duke University Medical Center, Durham, North Carolina
| |
Collapse
|
17
|
Mansoorian S, Schmidt M, Weissmann T, Delev D, Heiland DH, Coras R, Stritzelberger J, Saake M, Höfler D, Schubert P, Schmitter C, Lettmaier S, Filimonova I, Frey B, Gaipl US, Distel LV, Semrau S, Bert C, Eze C, Schönecker S, Belka C, Blümcke I, Uder M, Schnell O, Dörfler A, Fietkau R, Putz F. Reirradiation for recurrent glioblastoma: the significance of the residual tumor volume. J Neurooncol 2025:10.1007/s11060-025-05042-9. [PMID: 40310485 DOI: 10.1007/s11060-025-05042-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2025] [Accepted: 04/09/2025] [Indexed: 05/02/2025]
Abstract
PURPOSE Recurrent glioblastoma has a poor prognosis, and its optimal management remains unclear. Reirradiation (re-RT) is a promising treatment option, but long-term outcomes and optimal patient selection criteria are not well established. METHODS This study analyzed 71 patients with recurrent CNS WHO grade 4, IDHwt glioblastoma (GBM) who underwent re-RT at the University of Erlangen-Nuremberg between January 2009 and June 2019. Imaging follow-ups were conducted every 3 months. Progression-free survival (PFS) was defined using RANO criteria. Outcomes, feasibility, and toxicity of re-RT were evaluated. Contrast-enhancing tumor volume was measured using a deep learning auto-segmentation pipeline with expert validation and jointly evaluated with clinical and molecular-pathologic factors. RESULTS Most patients were prescribed conventionally fractionated re-RT (84.5%) with 45 Gy in 1.8 Gy fractions, combined with temozolomide (TMZ, 49.3%) or lomustine (CCNU, 12.7%). Re-RT was completed as planned in 94.4% of patients. After a median follow-up of 73.8 months, 88.7% of patients had died. The median overall survival was 9.6 months, and the median progression-free survival was 5.3 months. Multivariate analysis identified residual contrast-enhancing tumor volume at re-RT (HR 1.040 per cm3, p < 0.001) as the single dominant predictor of overall survival. CONCLUSION Conventional fractionated re-RT is a feasible and effective treatment for recurrent high-grade glioma. The significant prognostic impact of residual tumor volume highlights the importance of combining maximum-safe resection with re-RT for improved outcomes.
Collapse
Affiliation(s)
- Sina Mansoorian
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Department of Radiation Oncology, University Hospital, Ludwig-Maximilians-Universität München, Munich, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Manuel Schmidt
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Department of Neuroradiology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Thomas Weissmann
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Daniel Delev
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Department of Neurosurgery, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Dieter Henrik Heiland
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Department of Neurosurgery, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Roland Coras
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Department of Neuropathology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Jenny Stritzelberger
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Epilepsy Center, Department of Neurology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Marc Saake
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Institute of Radiology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Daniel Höfler
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Philipp Schubert
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Charlotte Schmitter
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Sebastian Lettmaier
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Irina Filimonova
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Benjamin Frey
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Udo S Gaipl
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Luitpold V Distel
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Sabine Semrau
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Christoph Bert
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Chukwuka Eze
- Department of Radiation Oncology, University Hospital, Ludwig-Maximilians-Universität München, Munich, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Stephan Schönecker
- Department of Radiation Oncology, University Hospital, Ludwig-Maximilians-Universität München, Munich, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Claus Belka
- Department of Radiation Oncology, University Hospital, Ludwig-Maximilians-Universität München, Munich, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Ingmar Blümcke
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Department of Neuropathology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Michael Uder
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Institute of Radiology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Oliver Schnell
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Department of Neurosurgery, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Arnd Dörfler
- Bavarian Cancer Research Center (BZKF), Munich, Germany
- Department of Neuroradiology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Rainer Fietkau
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Florian Putz
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitaetsstraße 27, 91054, Erlangen, Germany.
- Bavarian Cancer Research Center (BZKF), Munich, Germany.
| |
Collapse
|
18
|
Han Q, Du H. MHWT: Wide-range attention modeling using window transformer for multi-modal MRI reconstruction. Magn Reson Imaging 2025; 118:110362. [PMID: 39988183 DOI: 10.1016/j.mri.2025.110362] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2025] [Revised: 02/18/2025] [Accepted: 02/18/2025] [Indexed: 02/25/2025]
Abstract
The Swin Transformer, with its window-based attention mechanism, demonstrates strong feature modeling capabilities. However, it struggles with high-resolution feature maps due to its fixed window size, particularly when capturing long-range dependencies in magnetic resonance image reconstruction tasks. To overcome this, we propose a novel multi-modal hybrid window attention Transformer (MHWT) that introduces a retractable attention mechanism combined with shape-alternating window design. This approach expands attention coverage while maintaining computational efficiency. Additionally, we employ a variable and shifted window attention strategy to model both local and global dependencies more flexibly. Improvements to the Transformer encoder, including adjustments to normalization and attention score computation, enhance training stability and reconstruction performance. Experimental results on multiple public datasets show that our method outperforms state-of-the-art approaches in both single-modal and multi-modal scenarios, demonstrating superior image reconstruction ability and adaptability. The code is publicly available at https://github.com/EnieHan/MHWT.
Collapse
Affiliation(s)
- Qiuyi Han
- College of Information Science and Technology, University of Science and Technology of China, Hefei, 230026, Anhui, China
| | - Hongwei Du
- College of Information Science and Technology, University of Science and Technology of China, Hefei, 230026, Anhui, China.
| |
Collapse
|
19
|
Wu X, Hou Q, Xu Z, Tang C, Wang S, Sun J, Zhang Y. FCFDiff-Net: full-conditional feature diffusion embedded network for 3D brain tumor segmentation. Quant Imaging Med Surg 2025; 15:4217-4234. [PMID: 40384687 PMCID: PMC12084721 DOI: 10.21037/qims-24-2300] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2024] [Accepted: 02/21/2025] [Indexed: 05/20/2025]
Abstract
Background Brain tumor segmentation (BraTS) plays a critical role in medical imaging for early diagnosis and treatment planning. Recently, diffusion models have provided new insights into image segmentation, achieving significant success due to their ability to model nonlinearities. However, existing methods still face challenges, such as false negatives and false positives, caused by image blurring and noise interference, which remain major obstacles. This study aimed to develop a novel neural network approach to address these challenges in three-dimensional (3D) BraTS. Methods We propose a novel full-conditional feature diffusion embedded network (FCFDiff-Net) for 3D BraTS. This model enhances segmentation accuracy and robustness, particularly in noisy or ambiguous regions. This model introduces the full-conditional feature embedding (FCFE) module and employs a more comprehensive conditional embedding approach, fully integrating feature information from the original image into the diffusion model. It establishes an effective connection between the decoder side of the denoising network and the encoder side of the diffusion model, thereby improving the model's ability to capture the tumor target region and its boundaries. To further optimize performance and minimize discrepancies between conditional features and the denoising module, we introduce the multi-head attention residual fusion (MHARF) module. The MHARF module integrates features from the FCFE with noisy features generated during the denoising process. Using multi-head attention aligns semantic and noise information refining the segmentation results. This fusion enhances segmentation accuracy and stability by reducing noise impact and ensuring greater consistency across tumor regions. Results The BraTS 2020 and BraTS 2021 datasets served as the primary training and evaluation datasets. The proposed architecture was assessed using metrics such as Dice similarity coefficient (DSC), Hausdorff distance at the 95th percentile (HD95), specificity, and false positive rate (FPR). For the BraTS 2020 dataset, the DSC scores for the whole tumor (WT), tumor core (TC), and enhancing tumor (ET) were 0.916, 0.860, and 0.786, respectively. The HD95 values were 1.917, 2.571, and 2.581 mm, whereas specificity values were 0.998, 0.999, and 0.999, and FPR values were 0.002, 0.001, and 0.001, respectively. On the BraTS 2021 dataset, the DSC scores for the same regions were 0.926, 0.903, and 0.869, with HD95 values of 2.156, 1.834, and 1.583 mm, respectively. Specificity and FPR values were 0.999 across the board, and FPR values were consistently low at 0.001. These results demonstrate the model's excellent performance across the three regions. Conclusions The proposed FCFDiff-Net provides an efficient and robust solution for 3D BraTS, outperforming existing models in terms of accuracy and robustness. Future work will focus on exploring the model's generalization capabilities and conducting lightweight experiments to further enhance its applicability.
Collapse
Affiliation(s)
- Xiaosheng Wu
- School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Qingyi Hou
- School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Zhaozhao Xu
- School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Chaosheng Tang
- School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Shuihua Wang
- Department of Biological Sciences, Xi’an Jiaotong-Liverpool University, Suzhou, China
| | - Junding Sun
- School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Yudong Zhang
- School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK
- Department of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia
| |
Collapse
|
20
|
Zhu H, Liu Y, Li Y, Ding Y, Shen N, Xie Y, Yan S, Fu Y, Zhang J, Liu D, Zhang X, Li L, Zhu W. Amide Proton Transfer-weighted (APTw) Imaging and Derived Quantitative Metrics in Evaluating Gliomas: Improved Performance Compared to Magnetization Transfer Ratio Asymmetry (MTR asym). Acad Radiol 2025; 32:2919-2930. [PMID: 39809602 DOI: 10.1016/j.acra.2024.12.054] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2024] [Revised: 12/10/2024] [Accepted: 12/23/2024] [Indexed: 01/16/2025]
Abstract
RATIONALE AND OBJECTIVES Isocitrate dehydrogenase (IDH) status, glioma subtypes and tumor proliferation are important for glioma evaluation. We comprehensively compare the diagnostic performance of amide proton transfer-weighted (APTw) MRI and its related metrics in glioma diagnosis, in the context of the latest classification. MATERIALS AND METHODS Totally 110 patients with adult-type diffuse gliomas underwent APTw imaging. The magnetization transfer ratio asymmetry (MTRasym), magnetization transfer ratio normalized by reference signal (MTRnormref), and spillover-corrected magnetization transfer ratio yielding Rex (MTRRex), and metrics based on Lorentzian fitting (Fit-amide, Fit-MTRnormref, and Fit-MTRRex) were calculated. Group differences were compared between IDH genotypes, and among three glioma subtypes. The diagnostic performances were assessed using the receiver operating characteristic (ROC) analysis and compared. The correlations with Ki-67 expression were also analyzed. RESULTS All APTw-related metrics exhibited significantly higher values in IDH-wildtype gliomas than in IDH-mutant gliomas (all p < 0.001). Fit-MTRnormref had the best area under the curve (AUC) of 0.858. All APTw-related metrics in glioblastomas were significantly higher than oligodendrogliomas (all p < 0.01) and astrocytomas (all p < 0.001). No metrics had significant difference between oligodendrogliomas and astrocytomas. The highest AUCs was 0.870 for Fit-MTRnormref in distinguishing astrocytomas from glioblastomas, and 0.867 for Fit-MTRRex in distinguishing oligodendrogliomas from glioblastomas. Besides, Fit-MTRnormref had the highest correlation coefficient with Ki-67 expression of 0.578. CONCLUSION APTw-related metrics can effectively evaluate glioma IDH status, tumor subtypes and proliferation. The combination of Lorentzian fitting and the reference signal normalization could further improve the diagnostic performance, and perform better than MTRasym.
Collapse
Affiliation(s)
- Hongquan Zhu
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Yufei Liu
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Yuanhao Li
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Yuejie Ding
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Nanxi Shen
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Yan Xie
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Su Yan
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Yan Fu
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Jiaxuan Zhang
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Dong Liu
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Xiaoxiao Zhang
- Department of Clinical, Philips Healthcare, Wuhan, China (X.Z.)
| | - Li Li
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.)
| | - Wenzhen Zhu
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China (H.Z., Y.L., Y.L., Y.D., N.S., Y.X., S.Y., Y.F., J.Z., D.L., L.L., W.Z.).
| |
Collapse
|
21
|
Kuang H, Wang Y, Tan X, Yang J, Sun J, Liu J, Qiu W, Zhang J, Zhang J, Yang C, Wang J, Chen Y. LW-CTrans: A lightweight hybrid network of CNN and Transformer for 3D medical image segmentation. Med Image Anal 2025; 102:103545. [PMID: 40107117 DOI: 10.1016/j.media.2025.103545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2024] [Revised: 02/22/2025] [Accepted: 03/07/2025] [Indexed: 03/22/2025]
Abstract
Recent models based on convolutional neural network (CNN) and Transformer have achieved the promising performance for 3D medical image segmentation. However, these methods cannot segment small targets well even when equipping large parameters. Therefore, We design a novel lightweight hybrid network that combines the strengths of CNN and Transformers (LW-CTrans) and can boost the global and local representation capability at different stages. Specifically, we first design a dynamic stem that can accommodate images of various resolutions. In the first stage of the hybrid encoder, to capture local features with fewer parameters, we propose a multi-path convolution (MPConv) block. In the middle stages of the hybrid encoder, to learn global and local features meantime, we propose a multi-view pooling based Transformer (MVPFormer) which projects the 3D feature map onto three 2D subspaces to deal with small objects, and use the MPConv block for enhancing local representation learning. In the final stage, to mostly capture global features, only the proposed MVPFormer is used. Finally, to reduce the parameters of the decoder, we propose a multi-stage feature fusion module. Extensive experiments on 3 public datasets for three tasks: stroke lesion segmentation, pancreas cancer segmentation and brain tumor segmentation, show that the proposed LW-CTrans achieves Dices of 62.35±19.51%, 64.69±20.58% and 83.75±15.77% on the 3 datasets, respectively, outperforming 16 state-of-the-art methods, and the numbers of parameters (2.08M, 2.14M and 2.21M on 3 datasets, respectively) are smaller than the non-lightweight 3D methods and close to the lightweight methods. Besides, LW-CTrans also achieves the best performance for small lesion segmentation.
Collapse
Affiliation(s)
- Hulin Kuang
- Hunan Provincial Key Lab on Bioinformatics, School of Computer Science and Engineering, Central South University, Changsha 410000, China
| | - Yahui Wang
- Hunan Provincial Key Lab on Bioinformatics, School of Computer Science and Engineering, Central South University, Changsha 410000, China
| | - Xianzhen Tan
- Hunan Provincial Key Lab on Bioinformatics, School of Computer Science and Engineering, Central South University, Changsha 410000, China
| | - Jialin Yang
- Hunan Provincial Key Lab on Bioinformatics, School of Computer Science and Engineering, Central South University, Changsha 410000, China
| | - Jiarui Sun
- School of Computer Science and Engineering, Southeast University, Nanjing 210096, China; Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, Nanjing 210096, China
| | - Jin Liu
- Hunan Provincial Key Lab on Bioinformatics, School of Computer Science and Engineering, Central South University, Changsha 410000, China
| | - Wu Qiu
- School of Life Science and Technology, Huazhong University of Science and Technology, Wuhan 430000, China
| | - Jingyang Zhang
- School of Computer Science and Engineering, Southeast University, Nanjing 210096, China; Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, Nanjing 210096, China
| | - Jiulou Zhang
- Department of Radiology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, 210096, China; Lab for Artificial Intelligence in Medical Imaging (LAIMI), School of Medical Imaging, Nanjing Medical University, Nanjing, 210096, China
| | - Chunfeng Yang
- School of Computer Science and Engineering, Southeast University, Nanjing 210096, China; Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, Nanjing 210096, China.
| | - Jianxin Wang
- Hunan Provincial Key Lab on Bioinformatics, School of Computer Science and Engineering, Central South University, Changsha 410000, China; Xinjiang Engineering Research Center of Big Data and Intelligent Software, School of Software, Xinjiang University, Urumqi, 830091, Xinjiang, China.
| | - Yang Chen
- School of Computer Science and Engineering, Southeast University, Nanjing 210096, China; Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, Nanjing 210096, China
| |
Collapse
|
22
|
Barbano R, Denker A, Chung H, Roh TH, Arridge S, Maass P, Jin B, Ye JC. Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction. IEEE TRANSACTIONS ON MEDICAL IMAGING 2025; 44:2093-2104. [PMID: 40030859 DOI: 10.1109/tmi.2024.3524797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/05/2025]
Abstract
Denoising diffusion models have emerged as the go-to generative framework for solving inverse problems in imaging. A critical concern regarding these models is their performance on out-of-distribution tasks, which remains an under-explored challenge. Using a diffusion model on an out-of-distribution dataset, realistic reconstructions can be generated, but with hallucinating image features that are uniquely present in the training dataset. To address this discrepancy and improve reconstruction accuracy, we introduce a novel test-time adaptation sampling framework called Steerable Conditional Diffusion. Specifically, this framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. Utilising the proposed method, we achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities, advancing the robust deployment of denoising diffusion models in real-world applications.
Collapse
|
23
|
Yue G, Zhuo G, Zhou T, Liu W, Wang T, Jiang Q. Adaptive Cross-Feature Fusion Network With Inconsistency Guidance for Multi-Modal Brain Tumor Segmentation. IEEE J Biomed Health Inform 2025; 29:3148-3158. [PMID: 38150339 DOI: 10.1109/jbhi.2023.3347556] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2023]
Abstract
In the context of contemporary artificial intelligence, increasing deep learning (DL) based segmentation methods have been recently proposed for brain tumor segmentation (BraTS) via analysis of multi-modal MRI. However, known DL-based works usually directly fuse the information of different modalities at multiple stages without considering the gap between modalities, leaving much room for performance improvement. In this paper, we introduce a novel deep neural network, termed ACFNet, for accurately segmenting brain tumor in multi-modal MRI. Specifically, ACFNet has a parallel structure with three encoder-decoder streams. The upper and lower streams generate coarse predictions from individual modality, while the middle stream integrates the complementary knowledge of different modalities and bridges the gap between them to yield fine prediction. To effectively integrate the complementary information, we propose an adaptive cross-feature fusion (ACF) module at the encoder that first explores the correlation information between the feature representations from upper and lower streams and then refines the fused correlation information. To bridge the gap between the information from multi-modal data, we propose a prediction inconsistency guidance (PIG) module at the decoder that helps the network focus more on error-prone regions through a guidance strategy when incorporating the features from the encoder. The guidance is obtained by calculating the prediction inconsistency between upper and lower streams and highlights the gap between multi-modal data. Extensive experiments on the BraTS 2020 dataset show that ACFNet is competent for the BraTS task with promising results and outperforms six mainstream competing methods.
Collapse
|
24
|
Bao R, Weiss RJ, Bates SV, Song Y, He S, Li J, Bjornerud A, Hirschtick RL, Grant PE, Ou Y. PARADISE: Personalized and regional adaptation for HIE disease identification and segmentation. Med Image Anal 2025; 102:103419. [PMID: 40147073 DOI: 10.1016/j.media.2024.103419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2024] [Revised: 09/16/2024] [Accepted: 11/28/2024] [Indexed: 03/29/2025]
Abstract
Hypoxic ischemic encephalopathy (HIE) is a brain dysfunction occurring in approximately 1-5/1000 term-born neonates. Accurate segmentation of HIE lesions in brain MRI is crucial for prognosis and diagnosis but presents a unique challenge due to the diffuse and small nature of these abnormalities, which resulted in a substantial gap between the performance of machine learning-based segmentation methods and clinical expert annotations for HIE. To address this challenge, we introduce ParadiseNet, an algorithm specifically designed for HIE lesion segmentation. ParadiseNet incorporates global-local learning, progressive uncertainty learning, and self-evolution learning modules, all inspired by clinical interpretation of neonatal brain MRIs. These modules target issues such as unbalanced data distribution, boundary uncertainty, and imprecise lesion detection, respectively. Extensive experiments demonstrate that ParadiseNet significantly enhances small lesion detection (<1%) accuracy in HIE, achieving an over 4% improvement in Dice, 6% improvement in NSD compared to U-Net and other general medical image segmentation algorithms.
Collapse
Affiliation(s)
- Rina Bao
- Boston Children's Hospital, Boston, MA, USA; Harvard Medical School, Boston, MA, USA.
| | | | | | | | - Sheng He
- Boston Children's Hospital, Boston, MA, USA; Harvard Medical School, Boston, MA, USA
| | - Jingpeng Li
- Boston Children's Hospital, Boston, MA, USA; Oslo University Hospital; University of Oslo, Norway
| | | | - Randy L Hirschtick
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA; Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA
| | - P Ellen Grant
- Boston Children's Hospital, Boston, MA, USA; Harvard Medical School, Boston, MA, USA
| | - Yangming Ou
- Boston Children's Hospital, Boston, MA, USA; Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
25
|
Xie X, Zhang X, Tang X, Zhao J, Xiong D, Ouyang L, Yang B, Zhou H, Ling BWK, Teo KL. MACTFusion: Lightweight Cross Transformer for Adaptive Multimodal Medical Image Fusion. IEEE J Biomed Health Inform 2025; 29:3317-3328. [PMID: 38640042 DOI: 10.1109/jbhi.2024.3391620] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/21/2024]
Abstract
Multimodal medical image fusion aims to integrate complementary information from different modalities of medical images. Deep learning methods, especially recent vision Transformers, have effectively improved image fusion performance. However, there are limitations for Transformers in image fusion, such as lacks of local feature extraction and cross-modal feature interaction, resulting in insufficient multimodal feature extraction and integration. In addition, the computational cost of Transformers is higher. To address these challenges, in this work, we develop an adaptive cross-modal fusion strategy for unsupervised multimodal medical image fusion. Specifically, we propose a novel lightweight cross Transformer based on cross multi-axis attention mechanism. It includes cross-window attention and cross-grid attention to mine and integrate both local and global interactions of multimodal features. The cross Transformer is further guided by a spatial adaptation fusion module, which allows the model to focus on the most relevant information. Moreover, we design a special feature extraction module that combines multiple gradient residual dense convolutional and Transformer layers to obtain local features from coarse to fine and capture global features. The proposed strategy significantly boosts the fusion performance while minimizing computational costs. Extensive experiments, including clinical brain tumor image fusion, have shown that our model can achieve clearer texture details and better visual quality than other state-of-the-art fusion methods.
Collapse
|
26
|
Qiu Y, Jiang K, Yao H, Wang Z, Satoh S. Does Adding a Modality Really Make Positive Impacts in Incomplete Multi-Modal Brain Tumor Segmentation? IEEE TRANSACTIONS ON MEDICAL IMAGING 2025; 44:2194-2205. [PMID: 40031068 DOI: 10.1109/tmi.2025.3526818] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/05/2025]
Abstract
Previous incomplete multi-modal brain tumor segmentation technologies, while effective in integrating diverse modalities, commonly deliver under-expected performance gains. The reason lies in that the new modality may cause confused predictions due to uncertain and inconsistent patterns and quality in some positions, where the direct fusion consequently raises the negative gain for the final decision. In this paper, considering the potentially negative impacts within a modality, we propose multi-modal Positive-Negative impact region Double Calibration pipeline, called PNDC, to mitigate misinformation transfer of modality fusion. Concretely, PNDC involves two elaborate pipelines, Reverse Audit and Forward Checksum. The former is to identify negative regions impacts of each modality. The latter calibrates whether the fusion prediction is reliable in these regions by integrating the positive impacts regions of each modality. Finally, the negative impacts region from each modality and miss-match reliable fusion predictions are utilized to enhance the learning of individual modalities and fusion process. It is noted that PNDC adopts the standard training strategy without specific architectural choices and does not introduce any learning parameters, and thus can be easily plugged into existing network training for incomplete multi-modal brain tumor segmentation. Extensive experiments confirm that our PNDC greatly alleviates the performance degradation of current state-of-the-art incomplete medical multi-modal methods, arising from overlooking the positive/negative impacts regions of the modality. The code is released at PNDC.
Collapse
|
27
|
Zou J, Liu L, Chen Q, Wang S, Hu Z, Xing X, Qin J. MMR-Mamba: Multi-modal MRI reconstruction with Mamba and spatial-frequency information fusion. Med Image Anal 2025; 102:103549. [PMID: 40127589 DOI: 10.1016/j.media.2025.103549] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2024] [Revised: 03/07/2025] [Accepted: 03/08/2025] [Indexed: 03/26/2025]
Abstract
Multi-modal MRI offers valuable complementary information for diagnosis and treatment; however, its clinical utility is limited by prolonged scanning time. To accelerate the acquisition process, a practical approach is to reconstruct images of the target modality, which requires longer scanning time, from under-sampled k-space data using the fully-sampled reference modality with shorter scanning time as guidance. The primary challenge of this task lies in comprehensively and efficiently integrating complementary information from different modalities to achieve high-quality reconstruction. Existing methods struggle with this challenge: (1) convolution-based models fail to capture long-range dependencies; (2) transformer-based models, while excelling in global feature modeling, suffer from quadratic computational complexity. To address this dilemma, we propose MMR-Mamba, a novel framework that thoroughly and efficiently integrates multi-modal features for MRI reconstruction, leveraging Mamba's capability to capture long-range dependencies with linear computational complexity while exploiting global properties of the Fourier domain. Specifically, we first design a Target modality-guided Cross Mamba (TCM) module in the spatial domain, which maximally restores the target modality information by selectively incorporating relevant information from the reference modality. Then, we introduce a Selective Frequency Fusion (SFF) module to efficiently integrate global information in the Fourier domain and recover high-frequency signals for the reconstruction of structural details. Furthermore, we devise an Adaptive Spatial-Frequency Fusion (ASFF) module, which mutually enhances the spatial and frequency domains by supplementing less informative channels from one domain with corresponding channels from the other. Extensive experiments on the BraTS and fastMRI knee datasets demonstrate the superiority of our MMR-Mamba over state-of-the-art reconstruction methods. The code is publicly available at https://github.com/zoujing925/MMR-Mamba.
Collapse
Affiliation(s)
- Jing Zou
- Center for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Hong Kong Special Administrative Region of China
| | - Lanqing Liu
- Center for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Hong Kong Special Administrative Region of China
| | - Qi Chen
- Department of Electronic Engineering and Information Science, University of Science and Technology of China, Anhui, China
| | - Shujun Wang
- Department of Biomedical Engineering, The Hong Kong Polytechnic University, Hong Kong Special Administrative Region of China
| | - Zhanli Hu
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Xiaohan Xing
- Department of Radiation Oncology, Stanford University, Stanford, CA 94305, USA.
| | - Jing Qin
- Center for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Hong Kong Special Administrative Region of China
| |
Collapse
|
28
|
Zhang X, Ou N, Liu C, Zhuo Z, Matthews PM, Liu Y, Ye C, Bai W. Unsupervised brain MRI tumour segmentation via two-stage image synthesis. Med Image Anal 2025; 102:103568. [PMID: 40199108 DOI: 10.1016/j.media.2025.103568] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2024] [Revised: 03/24/2025] [Accepted: 03/25/2025] [Indexed: 04/10/2025]
Abstract
Deep learning shows promise in automated brain tumour segmentation, but it depends on costly expert annotations. Recent advances in unsupervised learning offer an alternative by using synthetic data for training. However, the discrepancy between real and synthetic data limits the accuracy of the unsupervised approaches. In this paper, we propose an approach for unsupervised brain tumour segmentation on magnetic resonance (MR) images via a two-stage image synthesis strategy. This approach accounts for the domain gap between real and synthetic data and aims to generate realistic synthetic data for model training. In the first stage, we train a junior segmentation model using synthetic brain tumour images generated by hand-crafted tumour shape and intensity models, and employs a validation set with distribution shift for model selection. The trained junior model is applied to segment unlabelled real tumour images, generating pseudo labels that capture realistic tumour shape, intensity, and texture. In the second stage, realistic synthetic tumour images are generated by mixing brain images with tumour pseudo labels, closing the domain gap between real and synthetic images. The generated synthetic data is then used to train a senior model for final segmentation. In experiments on five brain imaging datasets, the proposed approach, named as SynthTumour, surpasses existing unsupervised methods and demonstrates high performance for both brain tumour segmentation and ischemic stroke lesion segmentation tasks.
Collapse
Affiliation(s)
- Xinru Zhang
- School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, China; Department of Brain Sciences, Imperial College London, London, United Kingdom
| | - Ni Ou
- School of Automation, Beijing Institute of Technology, Beijing, China
| | - Chenghao Liu
- School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, China
| | - Zhizheng Zhuo
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Paul M Matthews
- Department of Brain Sciences, Imperial College London, London, United Kingdom; UK Dementia Research Institute, Imperial College London, London, United Kingdom
| | - Yaou Liu
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
| | - Chuyang Ye
- School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, China.
| | - Wenjia Bai
- Department of Brain Sciences, Imperial College London, London, United Kingdom; Department of Computing, Imperial College London, London, United Kingdom.
| |
Collapse
|
29
|
Duenias D, Nichyporuk B, Arbel T, Riklin Raviv T. Hyperfusion: A hypernetwork approach to multimodal integration of tabular and medical imaging data for predictive modeling. Med Image Anal 2025; 102:103503. [PMID: 40037055 DOI: 10.1016/j.media.2025.103503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2024] [Revised: 01/11/2025] [Accepted: 02/10/2025] [Indexed: 03/06/2025]
Abstract
The integration of diverse clinical modalities such as medical imaging and the tabular data extracted from patients' Electronic Health Records (EHRs) is a crucial aspect of modern healthcare. Integrative analysis of multiple sources can provide a comprehensive understanding of the clinical condition of a patient, improving diagnosis and treatment decision. Deep Neural Networks (DNNs) consistently demonstrate outstanding performance in a wide range of multimodal tasks in the medical domain. However, the complex endeavor of effectively merging medical imaging with clinical, demographic and genetic information represented as numerical tabular data remains a highly active and ongoing research pursuit. We present a novel framework based on hypernetworks to fuse clinical imaging and tabular data by conditioning the image processing on the EHR's values and measurements. This approach aims to leverage the complementary information present in these modalities to enhance the accuracy of various medical applications. We demonstrate the strength and generality of our method on two different brain Magnetic Resonance Imaging (MRI) analysis tasks, namely, brain age prediction conditioned by subject's sex and multi-class Alzheimer's Disease (AD) classification conditioned by tabular data. We show that our framework outperforms both single-modality models and state-of-the-art MRI tabular data fusion methods. A link to our code can be found at https://github.com/daniel4725/HyperFusion.
Collapse
Affiliation(s)
- Daniel Duenias
- Ben Gurion University of the Negev, blvd 1, Beer Sheva 84105, Israel
| | - Brennan Nichyporuk
- Centre for Intelligent Machines, McGill University, 3480 University St, Montreal, QC, H3A 0E9, Canada; Mila - Quebec AI Institute, 6666 Rue Saint-Urbain, Montréal, QC H2S 3H1, Canada
| | - Tal Arbel
- Centre for Intelligent Machines, McGill University, 3480 University St, Montreal, QC, H3A 0E9, Canada; Mila - Quebec AI Institute, 6666 Rue Saint-Urbain, Montréal, QC H2S 3H1, Canada
| | | |
Collapse
|
30
|
Yi PH, Bachina P, Bharti B, Garin SP, Kanhere A, Kulkarni P, Li D, Parekh VS, Santomartino SM, Moy L, Sulam J. Pitfalls and Best Practices in Evaluation of AI Algorithmic Biases in Radiology. Radiology 2025; 315:e241674. [PMID: 40392092 DOI: 10.1148/radiol.241674] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/22/2025]
Abstract
Despite growing awareness of problems with fairness in artificial intelligence (AI) models in radiology, evaluation of algorithmic biases, or AI biases, remains challenging due to various complexities. These include incomplete reporting of demographic information in medical imaging datasets, variability in definitions of demographic categories, and inconsistent statistical definitions of bias. To guide the appropriate evaluation of AI biases in radiology, this article summarizes the pitfalls in the evaluation and measurement of algorithmic biases. These pitfalls span the spectrum from the technical (eg, how different statistical definitions of bias impact conclusions about whether an AI model is biased) to those associated with social context (eg, how different conventions of race and ethnicity impact identification or masking of biases). Actionable best practices and future directions to avoid these pitfalls are summarized across three key areas: (a) medical imaging datasets, (b) demographic definitions, and (c) statistical evaluations of bias. Although AI bias in radiology has been broadly reviewed in the recent literature, this article focuses specifically on underrecognized potential pitfalls related to the three key areas. By providing awareness of these pitfalls along with actionable practices to avoid them, exciting AI technologies can be used in radiology for the good of all people.
Collapse
Affiliation(s)
- Paul H Yi
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - Preetham Bachina
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - Beepul Bharti
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - Sean P Garin
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - Adway Kanhere
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - Pranav Kulkarni
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - David Li
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - Vishwa S Parekh
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - Samantha M Santomartino
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - Linda Moy
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| | - Jeremias Sulam
- From the Department of Radiology, St Jude Children's Research Hospital, 262 Danny Thomas Pl, Memphis, TN 38105-3678 (P.H.Y.); Johns Hopkins University School of Medicine, Baltimore, Md (P.B.); Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Md (B.B., J.S.); Uniformed Services University of the Health Sciences, Bethesda, Md (S.P.G.); Institute for Health Computing, University of Maryland School of Medicine, Baltimore, Md (A.K., P.K.); Department of Medical Imaging, Western University Schulich School of Medicine & Dentistry, London, Ontario, Canada (D.L.); Department of Diagnostic and Interventional Imaging, McGovern Medical School at The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Tex (V.S.P.); Drexel University School of Medicine, Philadelphia, Pa (S.M.S.); and Department of Radiology, New York University Grossman School of Medicine, New York, NY (L.M.)
| |
Collapse
|
31
|
Cao J, Liu J, Chen J. A brain tumor segmentation method based on attention mechanism. Sci Rep 2025; 15:15229. [PMID: 40307461 PMCID: PMC12043955 DOI: 10.1038/s41598-025-98355-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2024] [Accepted: 04/10/2025] [Indexed: 05/02/2025] Open
Abstract
The rise in brain tumor incidence due to the global population aging has intensified the need for precise segmentation methods in clinical settings. Current segmentation networks often fail to capture comprehensive contextual information and fine edge details of brain tumors, which are crucial for accurate diagnosis and treatment. To address these challenges, we introduce BSAU-Net, a novel segmentation algorithm that employs attention mechanisms and edge feature extraction modules to enhance performance. Our approach aims to assist clinicians in making more accurate diagnostic and therapeutic decisions. BSAU-Net incorporates an edge feature extraction module (EA) based on the Sobel operator, enhancing the model's sensitivity to tumor regions while preserving edge contours. Additionally, a spatial attention module (SPA) is introduced to establish global feature correlations, critical for accurate tumor segmentation. To address class imbalance, which can hinder performance, we propose BADLoss, a loss function tailored to mitigate this issue. Experimental results on the BraTS2018 and BraTS2021 datasets demonstrate the effectiveness of BSAU-Net, achieving average Dice coefficients of 0.7506 and 0.7556, PPV of 0.7863 and 0.7843, sensitivity of 0.8998 and 0.9017, and HD95 of 2.1701 and 2.1543, respectively. These results highlight BSAU-Net's potential to significantly improve brain tumor segmentation in clinical practice.
Collapse
Affiliation(s)
- Juan Cao
- School of Information Science and Engineering, Chongqing Jiaotong University, Chongqing, 400074, China
| | - Jinjia Liu
- School of Information Science and Engineering, Chongqing Jiaotong University, Chongqing, 400074, China.
| | - Jiaran Chen
- School of Information Science and Engineering, Chongqing Jiaotong University, Chongqing, 400074, China
| |
Collapse
|
32
|
Lteif D, Appapogu D, Bargal SA, Plummer BA, Kolachalama VB. Anatomy-guided, modality-agnostic segmentation of neuroimaging abnormalities. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2025:2025.04.29.25326682. [PMID: 40343040 PMCID: PMC12060938 DOI: 10.1101/2025.04.29.25326682] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 05/11/2025]
Abstract
Magnetic resonance imaging (MRI) offers multiple sequences that provide complementary views of brain anatomy and pathology. However, real-world datasets often exhibit variability in sequence availability due to clinical and logistical constraints. This variability complicates radiological interpretation and limits the generalizability of machine learning models that depend on consistent multimodal input. In this work, we propose an anatomy-guided and modality-agnostic framework for assessing disease-related abnormalities in brain MRI, leveraging structural context to enhance robustness across diverse input configurations. We introduce a novel augmentation strategy, Region ModalMix, which integrates anatomical priors during training to improve model performance when some modalities are absent or variable. We conducted extensive experiments on brain tumor segmentation using the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 dataset (n=369). The results demonstrate that our proposed framework outperforms state-of-the-art methods on various missing modality conditions, especially by an average 9.68 mm reduction in 95th percentile Hausdorff Distance and a 1.36% improvement in Dice Similarity Coefficient over baseline models with only one available modailty. Our method is model-agnostic, training-compatible, and broadly applicable to multi-modal neuroimaging pipelines, enabling more reliable abnormality detection in settings with heterogeneous data availability.
Collapse
Affiliation(s)
- Diala Lteif
- Department of Computer Science, Boston University, Boston, MA, USA
- Department of Medicine, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
| | - Divya Appapogu
- Department of Computer Science, Boston University, Boston, MA, USA
- Department of Medicine, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
| | - Sarah A Bargal
- Department of Computer Science, Georgetown University, Washington, DC, USA
| | - Bryan A Plummer
- Department of Computer Science, Boston University, Boston, MA, USA
| | - Vijaya B Kolachalama
- Department of Computer Science, Boston University, Boston, MA, USA
- Department of Medicine, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
- Faculty of Computing & Data Sciences, Boston University, Boston, MA, USA
| |
Collapse
|
33
|
Gao Y, Jiang Y, Peng Y, Yuan F, Zhang X, Wang J. Medical Image Segmentation: A Comprehensive Review of Deep Learning-Based Methods. Tomography 2025; 11:52. [PMID: 40423254 DOI: 10.3390/tomography11050052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2025] [Revised: 04/23/2025] [Accepted: 04/28/2025] [Indexed: 05/28/2025] Open
Abstract
Medical image segmentation is a critical application of computer vision in the analysis of medical images. Its primary objective is to isolate regions of interest in medical images from the background, thereby assisting clinicians in accurately identifying lesions, their sizes, locations, and their relationships with surrounding tissues. However, compared to natural images, medical images present unique challenges, such as low resolution, poor contrast, inconsistency, and scattered target regions. Furthermore, the accuracy and stability of segmentation results are subject to more stringent requirements. In recent years, with the widespread application of Convolutional Neural Networks (CNNs) in computer vision, deep learning-based methods for medical image segmentation have become a focal point of research. This paper categorizes, reviews, and summarizes the current representative methods and research status in the field of medical image segmentation. A comparative analysis of relevant experiments is presented, along with an introduction to commonly used public datasets, performance evaluation metrics, and loss functions in medical image segmentation. Finally, potential future research directions and development trends in this field are predicted and analyzed.
Collapse
Affiliation(s)
- Yuxiao Gao
- College of Artificial Intelligence, Taiyuan University of Technology, Jinzhong 036000, China
| | - Yang Jiang
- College of Mechanical Engineering, Chongqing University of Technology, Chongqing 400054, China
| | - Yanhong Peng
- College of Mechanical Engineering, Chongqing University of Technology, Chongqing 400054, China
| | - Fujiang Yuan
- School of Computer Science and Technology, Taiyuan Normal University, Taiyuan 030619, China
| | - Xinyue Zhang
- College of Artificial Intelligence, Taiyuan University of Technology, Jinzhong 036000, China
| | - Jianfeng Wang
- School of Software, Taiyuan University of Technology, Jinzhong 036000, China
| |
Collapse
|
34
|
Karakas AB, Govsa F, Ozer MA, Biceroglu H, Eraslan C, Tanir D. From pixels to prognosis: leveraging radiomics and machine learning to predict IDH1 genotype in gliomas. Neurosurg Rev 2025; 48:396. [PMID: 40299088 PMCID: PMC12040993 DOI: 10.1007/s10143-025-03515-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2024] [Revised: 03/17/2025] [Accepted: 04/05/2025] [Indexed: 04/30/2025]
Abstract
Gliomas are the most common primary tumors of the central nervous system, and advances in genetics and molecular medicine have significantly transformed their classification and treatment. This study aims to predict the IDH1 genotype in gliomas using radiomics and machine learning (ML) methods. Retrospective data from 108 glioma patients were analyzed, including MRI data supported by demographic details such as age, sex, and comorbidities. Tumor segmentation was manually performed using 3D Slicer software, and 112 radiomic features were extracted with the PyRadiomics library. Feature selection using the mRMR algorithm identified 17 significant radiomic features. Various ML algorithms, including KNN, Ensemble, DT, LR, Discriminant and SVM, were applied to predict the IDH1 genotype. The KNN and Ensemble models achieved the highest sensitivity (92-100%) and specificity (100%), emerging as the most successful models. Comparative analyses demonstrated that KNN achieved an accuracy of 92.59%, sensitivity of 92.38%, specificity of 100%, precision of 100%, and an F1-score of 95.02%. Similarly, the Ensemble model achieved an accuracy of 90.74%, sensitivity of 90.65%, specificity of 100%, precision of 100%, and an F1-score of 95.13%. To evaluate their effectiveness, KNN and Ensemble models were compared with commonly used machine learning approaches in glioma classification. LR, a conventional statistical approach, exhibited lower predictive performance with an accuracy of 79.63%, while SVM, a frequently utilized ML model for radiomics-based tumor classification, achieved an accuracy of 85.19%. Our findings are consistent with previous research indicating that radiomics-based ML models achieve high accuracy in IDH1 mutation prediction, with reported performances typically exceeding 80%. These findings suggest that KNN and Ensemble models are more effective in capturing the non-linear radiomic patterns associated with IDH1 status, compared to traditional ML approaches. Our findings indicate that radiomic analyses provide comprehensive genotypic classification by assessing the entire tumor and present a safer, faster, and more patient-friendly alternative to traditional biopsies. This study highlights the potential of radiomics and ML techniques, particularly KNN, Ensemble, and SVM, as powerful tools for predicting the molecular characteristics of gliomas and developing personalized treatment strategies.
Collapse
Affiliation(s)
- Asli Beril Karakas
- Department of Anatomy, Faculty of Medicine, Kastamonu University, Kastamonu, 37200, Turkey.
| | - Figen Govsa
- Department of Anatomy, Faculty of Medicine, Ege University, Izmir, Turkey
| | - Mehmet Asim Ozer
- Department of Anatomy, Faculty of Medicine, Ege University, Izmir, Turkey
| | - Huseyin Biceroglu
- Department of Neurosurgery, Faculty of Medicine, Ege University, Izmir, Turkey
| | - Cenk Eraslan
- Department of Radiology, Faculty of Medicine, Ege University, Izmir, Turkey
| | - Deniz Tanir
- Department of Management Information Systems, Faculty of Economics and Administrative Sciences, Kafkas University, Kars, Turkey
| |
Collapse
|
35
|
Boga Z, Sándor C, Kovács P. A Multidimensional Particle Swarm Optimization-Based Algorithm for Brain MRI Tumor Segmentation. SENSORS (BASEL, SWITZERLAND) 2025; 25:2800. [PMID: 40363239 PMCID: PMC12074365 DOI: 10.3390/s25092800] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/18/2025] [Revised: 04/18/2025] [Accepted: 04/22/2025] [Indexed: 05/15/2025]
Abstract
Particle Swarm Optimization (PSO) has been extensively applied to optimization tasks in various domains, including image segmentation. In this work, we present a clustering-based segmentation algorithm that employs a multidimensional variant of PSO. Unlike conventional methods that require a predefined number of segments, our approach automatically selects an optimal segmentation granularity based on specified similarity criteria. This strategy effectively isolates brain tumors by incorporating both grayscale intensity and spatial information across multiple MRI modalities, allowing the method to be reliably tuned using a limited amount of training data. We further demonstrate how integrating these initial segmentations with a random forest classifier (RFC) enhances segmentation precision. Using MRI data from the RSNA-ASNR-MICCAI brain tumor segmentation (BraTS) challenge, our method achieves robust results with reduced reliance on extensive labeled datasets, offering a more efficient path toward accurate, clinically relevant tumor segmentation.
Collapse
Affiliation(s)
- Zsombor Boga
- Faculty of Mathematics and Computer Science, Babeș-Bolyai University, 400084 Cluj-Napoca, Romania;
| | - Csanád Sándor
- Faculty of Mathematics and Computer Science, Babeș-Bolyai University, 400084 Cluj-Napoca, Romania;
| | - Péter Kovács
- Faculty of Informatics, Eötvös Loránd University, 1117 Budapest, Hungary;
| |
Collapse
|
36
|
Yin Z, Ni Z, Ren Y, Nie D, Tang Z. Multi-sequence brain tumor segmentation boosted by deep semantic features. Med Phys 2025. [PMID: 40296197 DOI: 10.1002/mp.17845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2024] [Revised: 02/10/2025] [Accepted: 04/13/2025] [Indexed: 04/30/2025] Open
Abstract
BACKGROUND The main task of deep learning (DL) based brain tumor segmentation is to get accurate projection from learned image features to their corresponding semantic labels (i.e., brain tumor sub-regions). To achieve this goal, segmentation networks are required to learn image features with high intra-class consistency. However, brain tumor are known to be heterogeneous, and it often causes high diversity in image gray values which further influences the learned image features. Therefore, projecting such diverse image features (i.e., low intra-class consistency) to the same semantic label is often difficult and inefficient. PURPOSE The purpose of this study is to address the issue of low intra-class consistency of image features learned from heterogeneous brain tumor regions and ease the projection of image features to their corresponding semantic labels. In this way, accurate segmentation of brain tumor can be achieved. METHODS We propose a new DL-based method for brain tumor segmentation, where a semantic feature module (SFM) is introduced to consolidate image features with meaningful semantic information and enhance their intra-class consistency. Specifically, in the SFM, deep semantic vectors are derived and used as prototypes to re-encode image features learned in the segmentation network. Since the relatively consistent deep semantic vectors, diversity of the resulting image features can be reduced; moreover, semantic information in the resulting image features can also be enriched, both facilitating accurate projection to the final semantic labels. RESULTS In the experiment, a public brain tumor dataset, BraTS2022 containing, multi-sequence MR images of 1251 patients is used to evaluate our method in the task of brain tumor sub-region segmentation, and the experimental results demonstrate that, benefiting from the SFM, our method outperforms the state-of-the-art methods with statistical significance (p < 0.05 $p<0.05$ using the Wilcoxon signed rank test). Further ablation study shows that the proposed SFM can yield an improvement in segmentation accuracy (Dice index) of up to 11% comparing with that without the SFM. CONCLUSIONS In DL-based segmentation, low intra-class consistency of learned image features degrades segmentation performance. The proposed SFM can effectively enhance the intra-class consistency with high-level semantic information, making the projection of image features to their corresponding semantic labels more accurate.
Collapse
Affiliation(s)
- Ziman Yin
- School of Computer, Beihang University, Beijing, China
| | - Zhengze Ni
- School of Computer, Beihang University, Beijing, China
| | - Yuxiang Ren
- School of Computer, Beihang University, Beijing, China
| | - Dong Nie
- Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Zhenyu Tang
- School of Computer, Beihang University, Beijing, China
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
| |
Collapse
|
37
|
Li J, Peng H, Li B, Liu Z, Lugu R, He B. Global-Local Feature Fusion Network Based on Nonlinear Spiking Neural Convolutional Model for MRI Brain Tumor Segmentation. Int J Neural Syst 2025:2550036. [PMID: 40289786 DOI: 10.1142/s0129065725500364] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/30/2025]
Abstract
Due to the differences in size, shape, and location of brain tumors, brain tumor segmentation differs greatly from that of other organs. The purpose of brain tumor segmentation is to accurately locate and segment tumors from MRI images to assist doctors in diagnosis, treatment planning and surgical navigation. NSNP-like convolutional model is a new neural-like convolutional model inspired by nonlinear spiking mechanism of nonlinear spiking neural P (NSNP) systems. Therefore, this paper proposes a global-local feature fusion network based on NSNP-like convolutional model for MRI brain tumor segmentation. To this end, we have designed three characteristic modules that take full advantage of the NSNP-like convolution model: dilated SNP module (DSNP), multi-path dilated SNP pooling module (MDSP) and Poolformer module. The DSNP and MDSP modules are employed to construct the encoders. These modules help address the issue of feature loss and enable the fusion of more high-level features. On the other hand, the Poolformer module is used in the decoder. It processes features that contain global context information and facilitates the interaction between local and global features. In addition, channel spatial attention (CSA) module is designed at the skip connection between encoder and decoder to establish the long-range dependence between the same layers, thereby enhancing the relationship between channels and making the model have global modeling capabilities. In the experiments, our model achieves Dice coefficients of 85.71[Formula: see text], 92.32[Formula: see text], 87.75[Formula: see text] for ET, WT, and TC, respectively, on the N-BraTS2021 dataset. Moreover, our model achieves Dice coefficients of 83.91[Formula: see text], 91.96[Formula: see text], 90.14[Formula: see text] and 85.05[Formula: see text], 92.30[Formula: see text], 90.31[Formula: see text] on the BraTS2018 and BraTS2019 datasets respectively. Experimental results also indicate that our model not only achieves good brain tumor segmentation performance, but also has good generalization ability. The code is already available on GitHub: https://github.com/Li-JJ-1/NSNP-brain-tumor-segmentation.
Collapse
Affiliation(s)
- Junjie Li
- School of Computer and Software Engineering, Xihua University, Chengdu 610039, P. R. China
| | - Hong Peng
- School of Computer and Software Engineering, Xihua University, Chengdu 610039, P. R. China
| | - Bing Li
- School of Computer and Software Engineering, Xihua University, Chengdu 610039, P. R. China
| | - Zhicai Liu
- School of Computer and Software Engineering, Xihua University, Chengdu 610039, P. R. China
| | - Rikong Lugu
- School of Computer and Software Engineering, Xihua University, Chengdu 610039, P. R. China
| | - Bingyan He
- Glagow College, University of Electronic Science and Technology of China, Chengdu 611731, P. R. China
| |
Collapse
|
38
|
Gao X, Zhang M, Li J, Zhao S, Zhuo Z, Qu L, Weng J, Chai L, Duan Y, Ye C, Liu Y. Anatomy-guided slice-description interaction for multimodal brain disease diagnosis based on 3D image and radiological report. Comput Med Imaging Graph 2025; 123:102556. [PMID: 40300226 DOI: 10.1016/j.compmedimag.2025.102556] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2024] [Revised: 03/19/2025] [Accepted: 04/13/2025] [Indexed: 05/01/2025]
Abstract
Accurate brain disease diagnosis based on radiological images is desired in clinical practice as it can facilitate early intervention and reduce the risk of damage. However, existing unimodal image-based models struggle to process high-dimensional 3D brain imaging data effectively. Multimodal disease diagnosis approaches based on medical images and corresponding radiological reports achieved promising progress with the development of vision-language models. However, most multimodal methods handle 2D images and cannot be directly applied to brain disease diagnosis that uses 3D images. Therefore, in this work we develop a multimodal brain disease diagnosis model that takes 3D brain images and their radiological reports as input. Motivated by the fact that radiologists scroll through image slices and write important descriptions into the report accordingly, we propose a slice-description cross-modality interaction mechanism to realize fine-grained multimodal data interaction. Moreover, since previous medical research has demonstrated potential correlation between anatomical location of anomalies and diagnosis results, we further explore the use of brain anatomical prior knowledge to improve the multimodal interaction. Based on the report description, the prior knowledge filters the image information by suppressing irrelevant regions and enhancing relevant slices. Our method was validated with two brain disease diagnosis tasks. The results indicate that our model outperforms competing unimodal and multimodal methods for brain disease diagnosis. In particular, it has yielded an average accuracy improvement of 15.87% and 7.39% compared with the image-based and multimodal competing methods, respectively.
Collapse
Affiliation(s)
- Xin Gao
- School of Computer Science, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing, 100081, China
| | - Meihui Zhang
- School of Computer Science, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing, 100081, China.
| | - Junjie Li
- Department of Radiology, Beijing Tiantan Hospital, 6 Tiantan Xili, Dongcheng District, Beijing, 100070, China
| | - Shanbo Zhao
- School of Computer Science, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing, 100081, China
| | - Zhizheng Zhuo
- Department of Radiology, Beijing Tiantan Hospital, 6 Tiantan Xili, Dongcheng District, Beijing, 100070, China
| | - Liying Qu
- Department of Radiology, Beijing Tiantan Hospital, 6 Tiantan Xili, Dongcheng District, Beijing, 100070, China
| | - Jinyuan Weng
- Philips Healthcare, Philips (China) Investment Co. Ltd., Building 718, Lingshi Road, Jingan District, Shanghai, 200072, China
| | - Li Chai
- Department of Radiology, Beijing Tiantan Hospital, 6 Tiantan Xili, Dongcheng District, Beijing, 100070, China
| | - Yunyun Duan
- Department of Radiology, Beijing Tiantan Hospital, 6 Tiantan Xili, Dongcheng District, Beijing, 100070, China
| | - Chuyang Ye
- School of Integrated Circuits and Electronics, Beijing Institute of Technology, 5 Zhongguancun South Street, Haidian District, Beijing, 100081, China.
| | - Yaou Liu
- Department of Radiology, Beijing Tiantan Hospital, 6 Tiantan Xili, Dongcheng District, Beijing, 100070, China.
| |
Collapse
|
39
|
Wu ML, Peng YF. Multi-modality multiorgan image segmentation using continual learning with enhanced hard attention to the task. Med Phys 2025. [PMID: 40268717 DOI: 10.1002/mp.17842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2024] [Revised: 01/27/2025] [Accepted: 03/29/2025] [Indexed: 04/25/2025] Open
Abstract
BACKGROUND Enabling a deep neural network (DNN) to learn multiple tasks using the concept of continual learning potentially better mimics human brain functions. However, current continual learning studies for medical image segmentation are mostly limited to single-modality images at identical anatomical locations. PURPOSE To propose and evaluate a continual learning method termed eHAT (enhanced hard attention to the task) for performing multi-modality, multiorgan segmentation tasks using a DNN. METHODS Four public datasets covering the lumbar spine, heart, and brain acquired by magnetic resonance imaging (MRI) and computed tomography (CT) were included to segment the vertebral bodies, the right ventricle, and brain tumors, respectively. Three-task (spine CT, heart MRI, and brain MRI) and four-task (spine CT, heart MRI, brain MRI, and spine MRI) models were tested for eHAT, with the three-task results compared with state-of-the-art continual learning methods. The effectiveness of multitask performance was measured using the forgetting rate, defined as the average difference in Dice coefficients and Hausdorff distances between multiple-task and single-task models. The ability to transfer knowledge to different tasks was evaluated using backward transfer (BWT). RESULTS The forgetting rates were -2.51% to -0.60% for the three-task eHAT models with varying task orders, substantially better than the -18.13% to -3.59% using original hard attention to the task (HAT), while those in four-task models were -2.54% to -1.59%. In addition, four-task U-net models with eHAT using only half the number of channels (1/4 parameters) yielded nearly equal performance with or without regularization. A retrospective model comparison showed that eHAT with fixed or automatic regularization had significantly superior BWT (-3% to 0%) compared to HAT (-22% to -4%). CONCLUSION We demonstrate for the first time that eHAT effectively achieves continual learning of multi-modality, multiorgan segmentation tasks using a single DNN, with improved forgetting rates compared with HAT.
Collapse
Affiliation(s)
- Ming-Long Wu
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan
- Institute of Medical Informatics, National Cheng Kung University, Tainan, Taiwan
| | - Yi-Fan Peng
- Institute of Medical Informatics, National Cheng Kung University, Tainan, Taiwan
| |
Collapse
|
40
|
Al Hasan S, Mahim SM, Hossen ME, Hasan MO, Islam MK, Livreri P, Khan SU, Alibakhshikenari M, Miah MS. DSIT UNet a dual stream iterative transformer based UNet architecture for segmenting brain tumors from FLAIR MRI images. Sci Rep 2025; 15:13815. [PMID: 40259039 PMCID: PMC12012032 DOI: 10.1038/s41598-025-98464-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2024] [Accepted: 04/11/2025] [Indexed: 04/23/2025] Open
Abstract
Brain tumor segmentation remains challenging in medical imaging with conventional therapies and rehabilitation owing to the complex morphology and heterogeneous nature of tumors. Although convolutional neural networks (CNNs) have advanced medical image segmentation, they struggle with long-range dependencies because of their limited receptive fields. We propose Dual-Stream Iterative Transformer UNet (DSIT-UNet), a novel framework that combines Iterative Transformer (IT) modules with a dual-stream encoder-decoder architecture. Our model incorporates a transformed spatial-hybrid attention optimization (TSHAO) module to enhance multiscale feature interactions and balance local details with the global context. We evaluated DSIT-UNet using three benchmark datasets: The Cancer Imaging Archive (TCIA) from The Cancer Genome Atlas (TCGA), BraTS2020, and BraTS2021. On TCIA, our model achieved a Mean Intersection over Union of 95.21%, mean Dice Coefficient of 96.23%, precision of 95.91%, and recall of 96.55%. BraTS2020 attained a Mean IoU of 95.88%, mDice of 96.32%, precision of 96.21%, and recall of 96.44%, surpassing the performance of the existing methods. The superior results of DSIT-UNet demonstrate its effectiveness in capturing tumor boundaries and improving segmentation robustness through hierarchical attention mechanisms and multiscale feature extraction. This architecture advances automated brain tumor segmentation, with potential applications in clinical neuroimaging and future extensions to 3D volumetric segmentation.
Collapse
Affiliation(s)
- Shakib Al Hasan
- Department of Biomedical Engineering, Islamic University, Kushtia, 7003, Bangladesh
- Bio-Imaging Research Lab, BME, Islamic University, Kushtia, 7003, Bangladesh
| | - S M Mahim
- Department of Biomedical Engineering, Islamic University, Kushtia, 7003, Bangladesh
- Bio-Imaging Research Lab, BME, Islamic University, Kushtia, 7003, Bangladesh
| | - Md Emamul Hossen
- Department of Biomedical Engineering, Islamic University, Kushtia, 7003, Bangladesh
- Bio-Imaging Research Lab, BME, Islamic University, Kushtia, 7003, Bangladesh
| | - Md Olid Hasan
- Department of Biomedical Engineering, Islamic University, Kushtia, 7003, Bangladesh
- Bio-Imaging Research Lab, BME, Islamic University, Kushtia, 7003, Bangladesh
| | - Md Khairul Islam
- Department of Biomedical Engineering, Islamic University, Kushtia, 7003, Bangladesh.
- Bio-Imaging Research Lab, BME, Islamic University, Kushtia, 7003, Bangladesh.
| | - Patrizia Livreri
- Department of Engineering, University of Palermo, 90128, Palermo, Italy.
| | - Salah Uddin Khan
- Sustainable Energy Technologies Center, College of Engineering, College of Engineering, King Saud University, 11421, Riyadh, Saudi Arabia
- King Salman Center for Disability Research, 11614, Riyadh, Saudi Arabia
| | | | - Md Sipon Miah
- Department of Information and Communication Technology, Islamic University, Kushtia, 7003, Bangladesh.
- Machine Learning-aided Wireless Communications (WCML) Research Laboratory, Islamic University, Kushtia, 7003, Bangladesh.
| |
Collapse
|
41
|
Li L, Zhu L, Wang Q, Dong Z, Liao T, Li P. DSMR: Dual-Stream Networks with Refinement Module for Unsupervised Multi-modal Image Registration. Interdiscip Sci 2025:10.1007/s12539-025-00707-5. [PMID: 40252168 DOI: 10.1007/s12539-025-00707-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2024] [Revised: 03/25/2025] [Accepted: 03/27/2025] [Indexed: 04/21/2025]
Abstract
Multi-modal medical image registration aims to align images from different modalities to establish spatial correspondences. Although deep learning-based methods have shown great potential, the lack of explicit reference relations makes unsupervised multi-modal registration still a challenging task. In this paper, we propose a novel unsupervised dual-stream multi-modal registration framework (DSMR), which combines a dual-stream registration network with a refinement module. Unlike existing methods that treat multi-modal registration as a uni-modal problem using a translation network, DSMR leverages the moving, fixed and translated images to generate two deformation fields. Specifically, we first utilize a translation network to convert a moving image into a translated image similar to a fixed image. Then, we employ the dual-stream registration network to compute two deformation fields respectively: the initial deformation field generated from the fixed image and the moving image, and the translated deformation field generated from the translated image and the fixed image. The translated deformation field acts as a pseudo-ground truth to refine the initial deformation field and mitigate issues such as artificial features introduced by translation. Finally, we use the refinement module to enhance the deformation field by integrating registration errors and contextual information. Extensive experimental results show that our DSMR achieves exceptional performance, demonstrating its strong generalization in learning the spatial relationships between images from unsupervised modalities. The source code of this work is available at https://github.com/raylihaut/DSMR .
Collapse
Affiliation(s)
- Lei Li
- Key Laboratory of Grain Information Processing and Control, Henan University of Technology, Zhengzhou, 450001, China.
| | - Liumin Zhu
- College of Information Science and Engineering, Henan University of Technology, Zhengzhou, 450001, China
| | - Qifu Wang
- Institute of Applied Physics, Henan Academy of Sciences, Zhengzhou, 450001, China
| | - Zhuoli Dong
- College of Information Science and Engineering, Henan University of Technology, Zhengzhou, 450001, China
| | - Tianli Liao
- College of Information Science and Engineering, Henan University of Technology, Zhengzhou, 450001, China
| | - Peng Li
- Institute of Complexity Science, Henan University of Technology, Zhengzhou, 450001, China
| |
Collapse
|
42
|
Huang X, Wang Z, Zhou W, Yang K, Wen K, Liu H, Huang S, Lyu M. Tailored self-supervised pretraining improves brain MRI diagnostic models. Comput Med Imaging Graph 2025; 123:102560. [PMID: 40252479 DOI: 10.1016/j.compmedimag.2025.102560] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2024] [Revised: 04/05/2025] [Accepted: 04/16/2025] [Indexed: 04/21/2025]
Abstract
Self-supervised learning has shown potential in enhancing deep learning methods, yet its application in brain magnetic resonance imaging (MRI) analysis remains underexplored. This study seeks to leverage large-scale, unlabeled public brain MRI datasets to improve the performance of deep learning models in various downstream tasks for the development of clinical decision support systems. To enhance training efficiency, data filtering methods based on image entropy and slice positions were developed, condensing a combined dataset of approximately 2 million images from fastMRI-brain, OASIS-3, IXI, and BraTS21 into a more focused set of 250 K images enriched with brain features. The Momentum Contrast (MoCo) v3 algorithm was then employed to learn these image features, resulting in robustly pretrained models specifically tailored to brain MRI. The pretrained models were subsequently evaluated in tumor classification, lesion detection, hippocampal segmentation, and image reconstruction tasks. The results demonstrate that our brain MRI-oriented pretraining outperformed both ImageNet pretraining and pretraining on larger multi-organ, multi-modality medical datasets, achieving a ∼2.8 % increase in 4-class tumor classification accuracy, a ∼0.9 % improvement in tumor detection mean average precision, a ∼3.6 % gain in adult hippocampal segmentation Dice score, and a ∼0.1 PSNR improvement in reconstruction at 2-fold acceleration. This study underscores the potential of self-supervised learning for brain MRI using large-scale, tailored datasets derived from public sources.
Collapse
Affiliation(s)
- Xinhao Huang
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China; College of Applied Sciences, Shenzhen University, Shenzhen, China; Guangdong-Hongkong-Macau CNS Regeneration Institute, Key Laboratory of CNS Regeneration (Jinan University)-Ministry of Education, Jinan University, Guangzhou, China
| | - Zihao Wang
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China; College of Applied Sciences, Shenzhen University, Shenzhen, China; Guangdong-Hongkong-Macau CNS Regeneration Institute, Key Laboratory of CNS Regeneration (Jinan University)-Ministry of Education, Jinan University, Guangzhou, China
| | - Weichen Zhou
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
| | - Kexin Yang
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China; College of Applied Sciences, Shenzhen University, Shenzhen, China
| | - Kaihua Wen
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
| | - Haiguang Liu
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
| | - Shoujin Huang
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
| | - Mengye Lyu
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China; College of Applied Sciences, Shenzhen University, Shenzhen, China; Guangdong-Hongkong-Macau CNS Regeneration Institute, Key Laboratory of CNS Regeneration (Jinan University)-Ministry of Education, Jinan University, Guangzhou, China.
| |
Collapse
|
43
|
Mayeta-Revilla L, Cavieres EP, Salinas M, Mellado D, Ponce S, Torres Moyano F, Chabert S, Querales M, Sotelo J, Salas R. Radiomics-driven neuro-fuzzy framework for rule generation to enhance explainability in MRI-based brain tumor segmentation. Front Neuroinform 2025; 19:1550432. [PMID: 40313917 PMCID: PMC12043696 DOI: 10.3389/fninf.2025.1550432] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2024] [Accepted: 03/24/2025] [Indexed: 05/03/2025] Open
Abstract
Introduction Brain tumors are a leading cause of mortality worldwide, with early and accurate diagnosis being essential for effective treatment. Although Deep Learning (DL) models offer strong performance in tumor detection and segmentation using MRI, their black-box nature hinders clinical adoption due to a lack of interpretability. Methods We present a hybrid AI framework that integrates a 3D U-Net Convolutional Neural Network for MRI-based tumor segmentation with radiomic feature extraction. Dimensionality reduction is performed using machine learning, and an Adaptive Neuro-Fuzzy Inference System (ANFIS) is employed to produce interpretable decision rules. Each experiment is constrained to a small set of high-impact radiomic features to enhance clarity and reduce complexity. Results The framework was validated on the BraTS2020 dataset, achieving an average DICE Score of 82.94% for tumor core segmentation and 76.06% for edema segmentation. Classification tasks yielded accuracies of 95.43% for binary (healthy vs. tumor) and 92.14% for multi-class (healthy vs. tumor core vs. edema) problems. A concise set of 18 fuzzy rules was generated to provide clinically interpretable outputs. Discussion Our approach balances high diagnostic accuracy with enhanced interpretability, addressing a critical barrier in applying DL models in clinical settings. Integrating of ANFIS and radiomics supports transparent decision-making, facilitating greater trust and applicability in real-world medical diagnostics assistance.
Collapse
Affiliation(s)
- Leondry Mayeta-Revilla
- PhD Program in Health Sciences and Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Faculty of Engineering, School of Biomedical Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Center of Interdisciplinary Biomedical and Engineering Research for Health - MEDING, Universidad de Valparaíso, Valparaíso, Chile
- Millennium Institute for Intelligent Healthcare Engineering (iHealth), Santiago, Chile
| | - Eduardo P. Cavieres
- Faculty of Engineering, School of Biomedical Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Center of Interdisciplinary Biomedical and Engineering Research for Health - MEDING, Universidad de Valparaíso, Valparaíso, Chile
- Millennium Institute for Intelligent Healthcare Engineering (iHealth), Santiago, Chile
| | - Matías Salinas
- PhD Program in Health Sciences and Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Faculty of Engineering, School of Biomedical Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Center of Interdisciplinary Biomedical and Engineering Research for Health - MEDING, Universidad de Valparaíso, Valparaíso, Chile
| | - Diego Mellado
- PhD Program in Health Sciences and Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Faculty of Engineering, School of Biomedical Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Center of Interdisciplinary Biomedical and Engineering Research for Health - MEDING, Universidad de Valparaíso, Valparaíso, Chile
- Millennium Institute for Intelligent Healthcare Engineering (iHealth), Santiago, Chile
| | - Sebastian Ponce
- PhD Program in Health Sciences and Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Center of Interdisciplinary Biomedical and Engineering Research for Health - MEDING, Universidad de Valparaíso, Valparaíso, Chile
- Millennium Institute for Intelligent Healthcare Engineering (iHealth), Santiago, Chile
- Faculty of Medicine, School of Medical Technology, Universidad de Valparaíso, Valparaíso, Chile
| | - Francisco Torres Moyano
- Faculty of Engineering, School of Biomedical Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Millennium Institute for Intelligent Healthcare Engineering (iHealth), Santiago, Chile
- Servicio de Imagenología, Hospital Carlos van Buren, Valparaíso, Chile
- Centro para la Investigación Traslacional en Neurofarmacología (CITNE), Universidad de Valparaíso, Valparaíso, Chile
| | - Steren Chabert
- Faculty of Engineering, School of Biomedical Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Center of Interdisciplinary Biomedical and Engineering Research for Health - MEDING, Universidad de Valparaíso, Valparaíso, Chile
- Millennium Institute for Intelligent Healthcare Engineering (iHealth), Santiago, Chile
| | - Marvin Querales
- Center of Interdisciplinary Biomedical and Engineering Research for Health - MEDING, Universidad de Valparaíso, Valparaíso, Chile
- Faculty of Medicine, School of Medical Technology, Universidad de Valparaíso, Valparaíso, Chile
| | - Julio Sotelo
- Departamento de Informática, Universidad Técnica Federico Santa María, Santiago, Chile
| | - Rodrigo Salas
- Faculty of Engineering, School of Biomedical Engineering, Universidad de Valparaíso, Valparaíso, Chile
- Center of Interdisciplinary Biomedical and Engineering Research for Health - MEDING, Universidad de Valparaíso, Valparaíso, Chile
- Millennium Institute for Intelligent Healthcare Engineering (iHealth), Santiago, Chile
| |
Collapse
|
44
|
Wu Y, Liu Y, Yang Y, Yao MS, Yang W, Shi X, Yang L, Li D, Liu Y, Yin S, Lei C, Zhang M, Gee JC, Yang X, Wei W, Gu S. A concept-based interpretable model for the diagnosis of choroid neoplasias using multimodal data. Nat Commun 2025; 16:3504. [PMID: 40223097 PMCID: PMC11994757 DOI: 10.1038/s41467-025-58801-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2024] [Accepted: 04/02/2025] [Indexed: 04/15/2025] Open
Abstract
Diagnosing rare diseases remains a critical challenge in clinical practice, often requiring specialist expertise. Despite the promising potential of machine learning, the scarcity of data on rare diseases and the need for interpretable, reliable artificial intelligence (AI) models complicates development. This study introduces a multimodal concept-based interpretable model tailored to distinguish uveal melanoma (0.4-0.6 per million in Asians) from hemangioma and metastatic carcinoma following the clinical practice. We collected a comprehensive dataset on Asians to date on choroid neoplasm imaging with radiological reports, encompassing over 750 patients from 2013 to 2019. Our model integrates domain expert insights from radiological reports and differentiates between three types of choroidal tumors, achieving an F1 score of 0.91. This performance not only matches senior ophthalmologists but also improves the diagnostic accuracy of less experienced clinicians by 42%. The results underscore the potential of interpretable AI to enhance rare disease diagnosis and pave the way for future advancements in medical AI.
Collapse
Affiliation(s)
- Yifan Wu
- University of Pennsylvania, Philadelphia, PA, USA
| | - Yang Liu
- University of Electronic Science and Technology of China, Chengdu, China
| | - Yue Yang
- University of Pennsylvania, Philadelphia, PA, USA
| | | | - Wenli Yang
- Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Ophthalmology and Visual Sciences Key Lab, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Xuehui Shi
- Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Ophthalmology and Visual Sciences Key Lab, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Lihong Yang
- Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Ophthalmology and Visual Sciences Key Lab, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Dongjun Li
- Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Ophthalmology and Visual Sciences Key Lab, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Yueming Liu
- Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Ophthalmology and Visual Sciences Key Lab, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Shiyi Yin
- Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Tongren Hospital, Capital Medical University, Beijing, China
- Beijing Ophthalmology and Visual Sciences Key Lab, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Chunyan Lei
- Department of Ophthalmology and Research Laboratory of Macular Disease, West China Hospital, Sichuan University, Chengdu, China
| | - Meixia Zhang
- Department of Ophthalmology and Research Laboratory of Macular Disease, West China Hospital, Sichuan University, Chengdu, China
| | - James C Gee
- University of Pennsylvania, Philadelphia, PA, USA
| | - Xuan Yang
- Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China.
- Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Tongren Hospital, Capital Medical University, Beijing, China.
- Beijing Ophthalmology and Visual Sciences Key Lab, Beijing Tongren Hospital, Capital Medical University, Beijing, China.
| | - Wenbin Wei
- Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China.
- Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Tongren Hospital, Capital Medical University, Beijing, China.
- Beijing Ophthalmology and Visual Sciences Key Lab, Beijing Tongren Hospital, Capital Medical University, Beijing, China.
| | - Shi Gu
- University of Electronic Science and Technology of China, Chengdu, China.
- College of Computer Science and Technology, Zhejiang University, Hangzhou, China.
- State Key Laboratory of Brain Machine Intelligence, Zhejiang University, Hangzhou, China.
| |
Collapse
|
45
|
Alphonse S, Mathew F, Dhanush K, Dinesh V. Federated learning with integrated attention multiscale model for brain tumor segmentation. Sci Rep 2025; 15:11889. [PMID: 40195402 PMCID: PMC11976911 DOI: 10.1038/s41598-025-96416-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2024] [Accepted: 03/27/2025] [Indexed: 04/09/2025] Open
Abstract
Brain tumors are an extremely deadly condition and the growth of abnormal cells that have formed inside the brain causes the illness. According to studies, Magnetic Resonance Imaging (MRI) is a fundamental imaging method that is frequently used in medical diagnostics to identify, treat, and routinely check for brain cancers. These images include extremely private and delicate details regarding the brain health of the individuals and it must be treated with much care to ensure anonymity of patients. However, traditional brain tumor segmentation techniques usually rely on centralized data storage and analysis, which might result in privacy issues and violations. Federated learning offers a solution by enabling the cooperative development of brain tumor segmentation models without necessitating the transfer of raw patient data to a centralized location. All the data are held securely within their institution. A Reinforcement Learning-based Federated Averaging (RL-FedAvg) model is proposed that fuses the Federated Averaging (FedAvg) model with Reinforcement Learning (RL). To optimize the global model for image segmentation jobs as well as to govern the consumption of client resources, the model dynamically updates client hyperparameters upon real-time performance feedback. A Double Attention-based Multiscale Dense-U-Net model, known as mixed-fed-UNet, is proposed in the work that uses the RL-FedAvg algorithm. The proposed technique achieves 98.24% accuracy and 93.28% dice coefficient on BraTs 2020 dataset. While comparing the developed model with the other existing methods, the proposed methodology shows better performance.
Collapse
Affiliation(s)
- Sherly Alphonse
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, India.
| | - Fidal Mathew
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, India
| | - K Dhanush
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, India
| | - V Dinesh
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, India
| |
Collapse
|
46
|
Murugesan B, Adiga Vasudeva S, Liu B, Lombaert H, Ben Ayed I, Dolz J. Neighbor-aware calibration of segmentation networks with penalty-based constraints. Med Image Anal 2025; 101:103501. [PMID: 39978014 DOI: 10.1016/j.media.2025.103501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 01/08/2025] [Accepted: 02/08/2025] [Indexed: 02/22/2025]
Abstract
Ensuring reliable confidence scores from deep neural networks is of paramount significance in critical decision-making systems, particularly in real-world domains such as healthcare. Recent literature on calibrating deep segmentation networks has resulted in substantial progress. Nevertheless, these approaches are strongly inspired by the advancements in classification tasks, and thus their uncertainty is usually modeled by leveraging the information of individual pixels, disregarding the local structure of the object of interest. Indeed, only the recent Spatially Varying Label Smoothing (SVLS) approach considers pixel spatial relationships across classes, by softening the pixel label assignments with a discrete spatial Gaussian kernel. In this work, we first present a constrained optimization perspective of SVLS and demonstrate that it enforces an implicit constraint on soft class proportions of surrounding pixels. Furthermore, our analysis shows that SVLS lacks a mechanism to balance the contribution of the constraint with the primary objective, potentially hindering the optimization process. Based on these observations, we propose NACL (Neighbor Aware CaLibration), a principled and simple solution based on equality constraints on the logit values, which enables to control explicitly both the enforced constraint and the weight of the penalty, offering more flexibility. Comprehensive experiments on a wide variety of well-known segmentation benchmarks demonstrate the superior calibration performance of the proposed approach, without affecting its discriminative power. Furthermore, ablation studies empirically show the model agnostic nature of our approach, which can be used to train a wide span of deep segmentation networks. The code is available at https://github.com/Bala93/MarginLoss.
Collapse
|
47
|
Kuang Z, Yan Z, Abayazeed A, Wagner F, Yu L, Reyes M. ROXSI: Robust Cross-Sequence Semantic Interaction for Brain Tumor Segmentation on Multi-Sequence MR Images. IEEE J Biomed Health Inform 2025; 29:2899-2910. [PMID: 40030420 DOI: 10.1109/jbhi.2024.3513479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/05/2025]
Abstract
Deep learning-based brain tumor segmentation on multi-sequence magnetic resonance imaging (MRI) has gained widespread attention due to its great potential in supporting brain disease diagnosis. Although, compared to single-sequence images, more information is available from multi-sequence MR images, noise and artifacts on any given MR sequence can result in significant performance degradations. As in clinical routine, it is not always possible to maintain high imaging quality across all MR sequences (e.g., foreign bodies, ventricular drainage, shunts, involuntary patient motion, etc.), ensuring robustness of brain tumor segmentation from multi-sequence MR images is of great importance in clinical practice, but rarely explored. Accordingly, in this paper, we propose a robust brain tumor segmentation framework to mitigate the performance degradation caused by noise and artifacts on multi-sequence MR images. Specifically, based on semantic affinity, we propose a unique cross-sequence semantic interaction module (CSSI) to exploit inter-sequence correlations and extract noise-resilient features. In addition, we incorporate a batch-level covariance mechanism to suppress the redundant background information and improve the semantic enhancement effect of the CSSI module. In order to further improve segmentation performance, we also incorporate a sequence-level variance regularization mechanism to exploit sequence-specific features. To validate the robustness of ROXSI, brain tumor segmentation performance was evaluated under the existence of four common artifacts, at five different perturbation levels. We further performed a blinded qualitative clinical evaluation with two experienced neuro-radiologists, evaluating results from ROXSI and other popular CNN and Transformer-based segmentation models. Experimental results on two benchmark datasets demonstrate the superior robustness of ROXSI over other state-of-the-art segmentation methods.
Collapse
|
48
|
Wang Y, Song K, Liu Y, Ma S, Yan Y, Carneiro G. Leveraging labelled data knowledge: A cooperative rectification learning network for semi-supervised 3D medical image segmentation. Med Image Anal 2025; 101:103461. [PMID: 40032434 DOI: 10.1016/j.media.2025.103461] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2024] [Revised: 11/24/2024] [Accepted: 01/08/2025] [Indexed: 03/05/2025]
Abstract
Semi-supervised 3D medical image segmentation aims to achieve accurate segmentation using few labelled data and numerous unlabelled data. The main challenge in the design of semi-supervised learning methods consists in the effective use of the unlabelled data for training. A promising solution consists of ensuring consistent predictions across different views of the data, where the efficacy of this strategy depends on the accuracy of the pseudo-labels generated by the model for this consistency learning strategy. In this paper, we introduce a new methodology to produce high-quality pseudo-labels for a consistency learning strategy to address semi-supervised 3D medical image segmentation. The methodology has three important contributions. The first contribution is the Cooperative Rectification Learning Network (CRLN) that learns multiple prototypes per class to be used as external knowledge priors to adaptively rectify pseudo-labels at the voxel level. The second contribution consists of the Dynamic Interaction Module (DIM) to facilitate pairwise and cross-class interactions between prototypes and multi-resolution image features, enabling the production of accurate voxel-level clues for pseudo-label rectification. The third contribution is the Cooperative Positive Supervision (CPS), which optimises uncertain representations to align with unassertive representations of their class distributions, improving the model's accuracy in classifying uncertain regions. Extensive experiments on three public 3D medical segmentation datasets demonstrate the effectiveness and superiority of our semi-supervised learning method.
Collapse
Affiliation(s)
- Yanyan Wang
- School of Mechanical Engineering and Automation, Northeastern University, China; Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK.
| | - Kechen Song
- School of Mechanical Engineering and Automation, Northeastern University, China.
| | - Yuyuan Liu
- Australian Institute for Machine Learning, University of Adelaide, Australia.
| | - Shuai Ma
- School of Mechanical Engineering and Automation, Northeastern University, China.
| | - Yunhui Yan
- School of Mechanical Engineering and Automation, Northeastern University, China.
| | - Gustavo Carneiro
- Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK.
| |
Collapse
|
49
|
Song H, Wang J, Zhou J, Wang L. Tackling Modality-Heterogeneous Client Drift Holistically for Heterogeneous Multimodal Federated Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2025; 44:1931-1941. [PMID: 40030772 DOI: 10.1109/tmi.2024.3523378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/05/2025]
Abstract
Multimodal Federated Learning (MFL) has emerged as a collaborative paradigm for training models across decentralized devices, harnessing various data modalities to facilitate effective learning while respecting data ownership. In this realm, notably, a pivotal shift from homogeneous to heterogeneous MFL has taken place. While the former assumes uniformity in input modalities across clients, the latter accommodates modality-incongruous setups, which is often the case in practical situations. For example, while some advanced medical institutions have the luxury of utilizing both MRI and CT for disease diagnosis, remote hospitals often find themselves constrained to employ CT exclusively due to its cost-effectiveness. Although heterogeneous MFL can apply to a broader scenario, it introduces a new challenge: modality-heterogeneous client drift, arising from diverse modality-coupled local optimization. To address this, we introduce FedMM, a simple yet effective approach. During local optimization, FedMM employs modality dropout, randomly masking available modalities, and promoting weight alignment while preserving model expressivity on its original modality combination. To enhance the modality dropout process, FedMM incorporates a task-specific inter- and intra-modal regularizer, which acts as an additional constraint, forcing that weight distribution remains more consistent across diverse input modalities and therefore eases the optimization process with modality dropout enabled. By combining them, our approach holistically addresses client drift. It fosters convergence among client models while considering each client's unique input modalities, enhancing heterogeneous MFL performance. Comprehensive evaluations in three medical image segmentation datasets demonstrate FedMM's superiority over state-of-the-art heterogeneous MFL methods.
Collapse
|
50
|
Zheng S, Ye X, Yang C, Yu L, Li W, Gao X, Zhao Y. Asymmetric Adaptive Heterogeneous Network for Multi-Modality Medical Image Segmentation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2025; 44:1836-1852. [PMID: 40031190 DOI: 10.1109/tmi.2025.3526604] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/05/2025]
Abstract
Existing studies of multi-modality medical image segmentation tend to aggregate all modalities without discrimination and employ multiple symmetric encoders or decoders for feature extraction and fusion. They often overlook the different contributions to visual representation and intelligent decisions among multi-modality images. Motivated by this discovery, this paper proposes an asymmetric adaptive heterogeneous network for multi-modality image feature extraction with modality discrimination and adaptive fusion. For feature extraction, it uses a heterogeneous two-stream asymmetric feature-bridging network to extract complementary features from auxiliary multi-modality and leading single-modality images, respectively. For feature adaptive fusion, the proposed Transformer-CNN Feature Alignment and Fusion (T-CFAF) module enhances the leading single-modality information, and the Cross-Modality Heterogeneous Graph Fusion (CMHGF) module further fuses multi-modality features at a high-level semantic layer adaptively. Comparative evaluation with ten segmentation models on six datasets demonstrates significant efficiency gains as well as highly competitive segmentation accuracy. (Our code is publicly available at https://github.com/joker-527/AAHN).
Collapse
|