1
|
Cai L, Pfob A. Artificial intelligence in abdominal and pelvic ultrasound imaging: current applications. Abdom Radiol (NY) 2025; 50:1775-1789. [PMID: 39487919 PMCID: PMC11947003 DOI: 10.1007/s00261-024-04640-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2024] [Revised: 10/06/2024] [Accepted: 10/10/2024] [Indexed: 11/04/2024]
Abstract
BACKGROUND In recent years, the integration of artificial intelligence (AI) techniques into medical imaging has shown great potential to transform the diagnostic process. This review aims to provide a comprehensive overview of current state-of-the-art applications for AI in abdominal and pelvic ultrasound imaging. METHODS We searched the PubMed, FDA, and ClinicalTrials.gov databases for applications of AI in abdominal and pelvic ultrasound imaging. RESULTS A total of 128 titles were identified from the database search and were eligible for screening. After screening, 57 manuscripts were included in the final review. The main anatomical applications included multi-organ detection (n = 16, 28%), gynecology (n = 15, 26%), hepatobiliary system (n = 13, 23%), and musculoskeletal (n = 8, 14%). The main methodological applications included deep learning (n = 37, 65%), machine learning (n = 13, 23%), natural language processing (n = 5, 9%), and robots (n = 2, 4%). The majority of the studies were single-center (n = 43, 75%) and retrospective (n = 56, 98%). We identified 17 FDA approved AI ultrasound devices, with only a few being specifically used for abdominal/pelvic imaging (infertility monitoring and follicle development). CONCLUSION The application of AI in abdominal/pelvic ultrasound shows promising early results for disease diagnosis, monitoring, and report refinement. However, the risk of bias remains high because very few of these applications have been prospectively validated (in multi-center studies) or have received FDA clearance.
Collapse
Affiliation(s)
- Lie Cai
- Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120, Heidelberg, Germany
| | - André Pfob
- Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120, Heidelberg, Germany.
- National Center for Tumor Diseases (NCT) and German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
2
|
Nciki AI, Hlabangana LT. Perceptions and attitudes towards AI among trainee and qualified radiologists at selected South African training hospitals. SA J Radiol 2025; 29:3026. [PMID: 39968514 PMCID: PMC11830846 DOI: 10.4102/sajr.v29i1.3026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2024] [Accepted: 11/17/2024] [Indexed: 02/20/2025] Open
Abstract
Background Artificial intelligence (AI) is transforming industries, but its adoption in healthcare, especially radiology, remains contentious. Objectives This study evaluated the perceptions and attitudes of trainee and qualified radiologists towards the adoption of AI in practice. Method A cross-sectional survey using a paper-based questionnaire was completed by trainee and qualified radiologists. Survey questions covered AI knowledge, perceptions, attitudes, and AI training in the registrar programme on a 3-point Likert scale. Results A total of 100 participants completed the survey; 54% were aged 26-65 years and 61% were female, with none currently using AI in daily radiology practice. The majority (78%) of participants understood the basics and knew the role of AI in radiology. Most knew about AI from media reports (77%) and majority (95%) were never involved in AI training; only 3% of participants had no knowledge of AI at all. Participants agreed that AI could reliably detect pathological conditions (89%), reach reliable diagnosis (89%), improve daily work (78%), and 89% favoured AI practice; 89% believed that in the future, machine learning will not be independent of the radiologist. Participants were willing to learn (98%) and contribute towards advancing AI software (97%) and agreed that AI will improve the registrars' programme (97%), also noting that AI applications are as important as medical skills (87%). Conclusion The findings suggest AI in radiology is in its infancy, with a need for educational programmes to upskill radiologists. Contribution Participants were positive about AI implementation in practice and in the registrar learning programme.
Collapse
Affiliation(s)
- Ayanda I Nciki
- Department of Radiology, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
| | - Linda T Hlabangana
- Department of Radiology, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
| |
Collapse
|
3
|
Ervik Ø, Rødde M, Hofstad EF, Tveten I, Langø T, Leira HO, Amundsen T, Sorger H. A New Deep Learning-Based Method for Automated Identification of Thoracic Lymph Node Stations in Endobronchial Ultrasound (EBUS): A Proof-of-Concept Study. J Imaging 2025; 11:10. [PMID: 39852323 PMCID: PMC11766424 DOI: 10.3390/jimaging11010010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2024] [Revised: 12/26/2024] [Accepted: 01/04/2025] [Indexed: 01/26/2025] Open
Abstract
Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is a cornerstone in minimally invasive thoracic lymph node sampling. In lung cancer staging, precise assessment of lymph node position is crucial for clinical decision-making. This study aimed to demonstrate a new deep learning method to classify thoracic lymph nodes based on their anatomical location using EBUS images. Bronchoscopists labeled lymph node stations in real-time according to the Mountain Dressler nomenclature. EBUS images were then used to train and test a deep neural network (DNN) model, with intraoperative labels as ground truth. In total, 28,134 EBUS images were acquired from 56 patients. The model achieved an overall classification accuracy of 59.5 ± 5.2%. The highest precision, sensitivity, and F1 score were observed in station 4L, 77.6 ± 13.1%, 77.6 ± 15.4%, and 77.6 ± 15.4%, respectively. The lowest precision, sensitivity, and F1 score were observed in station 10L. The average processing and prediction time for a sequence of ten images was 0.65 ± 0.04 s, demonstrating the feasibility of real-time applications. In conclusion, the new DNN-based model could be used to classify lymph node stations from EBUS images. The method performance was promising with a potential for clinical use.
Collapse
Affiliation(s)
- Øyvind Ervik
- Clinic of Medicine, Nord-Trøndelag Hospital Trust, Levanger Hospital, 7601 Levanger, Norway;
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, 7030 Trondheim, Norway; (H.O.L.); (T.A.)
| | - Mia Rødde
- Department of Health Research, SINTEF Digital, 7034 Trondheim, Norway; (M.R.); (E.F.H.); (I.T.); (T.L.)
| | - Erlend Fagertun Hofstad
- Department of Health Research, SINTEF Digital, 7034 Trondheim, Norway; (M.R.); (E.F.H.); (I.T.); (T.L.)
| | - Ingrid Tveten
- Department of Health Research, SINTEF Digital, 7034 Trondheim, Norway; (M.R.); (E.F.H.); (I.T.); (T.L.)
| | - Thomas Langø
- Department of Health Research, SINTEF Digital, 7034 Trondheim, Norway; (M.R.); (E.F.H.); (I.T.); (T.L.)
- National Research Center for Minimally Invasive and Image-Guided Diagnostics and Therapy, St. Olavs Hospital, 7030 Trondheim, Norway
| | - Håkon O. Leira
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, 7030 Trondheim, Norway; (H.O.L.); (T.A.)
- National Research Center for Minimally Invasive and Image-Guided Diagnostics and Therapy, St. Olavs Hospital, 7030 Trondheim, Norway
- Department of Thoracic Medicine, St. Olavs Hospital, Trondheim University Hospital, 7030 Trondheim, Norway
| | - Tore Amundsen
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, 7030 Trondheim, Norway; (H.O.L.); (T.A.)
- Department of Thoracic Medicine, St. Olavs Hospital, Trondheim University Hospital, 7030 Trondheim, Norway
| | - Hanne Sorger
- Clinic of Medicine, Nord-Trøndelag Hospital Trust, Levanger Hospital, 7601 Levanger, Norway;
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, 7030 Trondheim, Norway; (H.O.L.); (T.A.)
| |
Collapse
|
4
|
Yan S, Xiong F, Xin Y, Zhou Z, Liu W. Automated assessment of endometrial receptivity for screening recurrent pregnancy loss risk using deep learning-enhanced ultrasound and clinical data. Front Physiol 2024; 15:1404418. [PMID: 39777360 PMCID: PMC11703864 DOI: 10.3389/fphys.2024.1404418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2024] [Accepted: 12/11/2024] [Indexed: 01/11/2025] Open
Abstract
Background Recurrent pregnancy loss (RPL) poses significant challenges in clinical management due to an unclear etiology in over half the cases. Traditional screening methods, including ultrasonographic evaluation of endometrial receptivity (ER), have been debated for their efficacy in identifying high-risk individuals. Despite the potential of artificial intelligence, notably deep learning (DL), to enhance medical imaging analysis, its application in ER assessment for RPL risk stratification remains underexplored. Objective This study aims to leverage DL techniques in the analysis of routine clinical and ultrasound examination data to refine ER assessment within RPL management. Methods Employing a retrospective, controlled design, this study included 346 individuals with unexplained RPL and 369 controls to assess ER. Participants were allocated into training (n = 485) and testing (n = 230) datasets for model construction and performance evaluation, respectively. DL techniques were applied to analyze conventional grayscale ultrasound images and clinical data, utilizing a pre-trained ResNet-50 model for imaging analysis and TabNet for tabular data interpretation. The model outputs were calibrated to generate probabilistic scores, representing the risk of RPL. Both comparative analyses and ablation studies were performed using ResNet-50, TabNet, and a combined fusion model. These were evaluated against other state-of-the-art DL and machine learning (ML) models, with the results validated against the testing dataset. Results The comparative analysis demonstrated that the ResNet-50 model outperformed other DL architectures, achieving the highest accuracy and the lowest Brier score. Similarly, the TabNet model exceeded the performance of traditional ML models. Ablation studies demonstrated that the fusion model, which integrates both data modalities and is presented through a nomogram, provided the most accurate predictions, with an area under the curve of 0.853. The radiological DL model made a more significant contribution to the overall performance of the fusion model, underscoring its superior predictive capability. Conclusion This investigation demonstrates the superiority of a DL-enhanced fusion model that integrates routine ultrasound and clinical data for accurate stratification of RPL risk, offering significant advancements over traditional methods.
Collapse
Affiliation(s)
- Shanling Yan
- Department of Ultrasound, Deyang People’s Hospital, Deyang, Sichuan, China
| | - Fei Xiong
- Department of Ultrasound, Deyang People’s Hospital, Deyang, Sichuan, China
| | - Yanfen Xin
- Department of Ultrasound, Deyang People’s Hospital, Deyang, Sichuan, China
| | - Zhuyu Zhou
- Department of Ultrasound, Deyang People’s Hospital, Deyang, Sichuan, China
| | - Wanqing Liu
- Department of Obstetrics and Gynecology, Deyang People’s Hospital, Deyang, Sichuan, China
| |
Collapse
|
5
|
Raina D, Chandrashekhara SH, Voyles R, Wachs J, Saha SK. Deep-Learning Model for Quality Assessment of Urinary Bladder Ultrasound Images Using Multiscale and Higher-Order Processing. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2024; 71:1451-1463. [PMID: 38598406 DOI: 10.1109/tuffc.2024.3386919] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/12/2024]
Abstract
Autonomous ultrasound image quality assessment (US-IQA) is a promising tool to aid the interpretation by practicing sonographers and to enable the future robotization of ultrasound procedures. However, autonomous US-IQA has several challenges. Ultrasound images contain many spurious artifacts, such as noise due to handheld probe positioning, errors in the selection of probe parameters, and patient respiration during the procedure. Furthermore, these images are highly variable in appearance with respect to the individual patient's physiology. We propose to use a deep convolutional neural network (CNN), USQNet, which utilizes a multiscale and local-to-global second-order pooling (MS-L2GSoP) classifier to conduct the sonographer-like assessment of image quality. This classifier first extracts features at multiple scales to encode the interpatient anatomical variations, similar to a sonographer's understanding of anatomy. Then, it uses SoP in the intermediate layers (local) and at the end of the network (global) to exploit the second-order statistical dependency of MS structural and multiregion textural features. The L2GSoP will capture the higher-order relationships between different spatial locations and provide the seed for correlating local patches, much like a sonographer prioritizes regions across the image. We experimentally validated the USQNet for a new dataset of the human urinary bladder (UB) ultrasound images. The validation involved first with the subjective assessment by experienced radiologists' annotation, and then with state-of-the-art (SOTA) CNN networks for US-IQA and its ablated counterparts. The results demonstrate that USQNet achieves a remarkable accuracy of 92.4% and outperforms the SOTA models by 3%-14% while requiring comparable computation time.
Collapse
|
6
|
McKnight S, Tunukovic V, Gareth Pierce S, Mohseni E, Pyle R, MacLeod CN, O'Hare T. Advancing Carbon Fiber Composite Inspection: Deep Learning-Enabled Defect Localization and Sizing via 3-D U-Net Segmentation of Ultrasonic Data. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2024; 71:1106-1119. [PMID: 38829751 DOI: 10.1109/tuffc.2024.3408314] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2024]
Abstract
In nondestructive evaluation (NDE), accurately characterizing defects within components relies on accurate sizing and localization to evaluate the severity or criticality of defects. This study presents for the first time a deep learning (DL) methodology using 3-D U-Net to localize and size defects in carbon fiber reinforced polymer (CFRP) composites through volumetric segmentation of ultrasonic testing (UT) data. Using a previously developed approach, synthetic training data, closely representative of experimental data, was used for the automatic generation of ground truth segmentation masks. The model's performance was compared to the conventional amplitude 6 dB drop analysis method used in the industry against ultrasonic defect responses from 40 defects fabricated in CFRP components. The results showed good agreement with the 6 dB drop method for in-plane localization and excellent through-thickness localization, with mean absolute errors (MAEs) of 0.57 and 0.08 mm, respectively. Initial sizing results consistently oversized defects with a 55% higher mean average error than the 6 dB drop method. However, when a correction factor was applied to account for variation between the experimental and synthetic domains, the final sizing accuracy resulted in a 35% reduction in MAE compared to the 6 dB drop technique. By working with volumetric ultrasonic data (as opposed to 2-D images), this approach reduces preprocessing (such as signal gating) and allows for the generation of 3-D defect masks which can be used for the generation of computer-aided design files; greatly reducing the qualification reporting burden of NDE operators.
Collapse
|
7
|
Ye RZ, Lipatov K, Diedrich D, Bhattacharyya A, Erickson BJ, Pickering BW, Herasevich V. Automatic ARDS surveillance with chest X-ray recognition using convolutional neural networks. J Crit Care 2024; 82:154794. [PMID: 38552452 DOI: 10.1016/j.jcrc.2024.154794] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 11/20/2023] [Accepted: 12/01/2023] [Indexed: 06/01/2024]
Abstract
OBJECTIVE This study aims to design, validate and assess the accuracy a deep learning model capable of differentiation Chest X-Rays between pneumonia, acute respiratory distress syndrome (ARDS) and normal lungs. MATERIALS AND METHODS A diagnostic performance study was conducted using Chest X-Ray images from adult patients admitted to a medical intensive care unit between January 2003 and November 2014. X-ray images from 15,899 patients were assigned one of three prespecified categories: "ARDS", "Pneumonia", or "Normal". RESULTS A two-step convolutional neural network (CNN) pipeline was developed and tested to distinguish between the three patterns with sensitivity ranging from 91.8% to 97.8% and specificity ranging from 96.6% to 98.8%. The CNN model was validated with a sensitivity of 96.3% and specificity of 96.6% using a previous dataset of patients with Acute Lung Injury (ALI)/ARDS. DISCUSSION The results suggest that a deep learning model based on chest x-ray pattern recognition can be a useful tool in distinguishing patients with ARDS from patients with normal lungs, providing faster results than digital surveillance tools based on text reports. CONCLUSION A CNN-based deep learning model showed clinically significant performance, providing potential for faster ARDS identification. Future research should prospectively evaluate these tools in a clinical setting.
Collapse
Affiliation(s)
- Run Zhou Ye
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA.; Division of Endocrinology, Department of Medicine, Centre de Recherche du CHUS, Sherbrooke QC J1H 5N4, Canada
| | - Kirill Lipatov
- Critical Care Medicine, Mayo Clinic, Eau Claire, WI, United States
| | - Daniel Diedrich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | | | - Bradley J Erickson
- Department of Diagnostic Radiology, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Brian W Pickering
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Vitaly Herasevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA..
| |
Collapse
|
8
|
Wang Z, Wu M, Liu Q, Wang X, Yan C, Song T. Multiclassification of Hepatic Cystic Echinococcosis by Using Multiple Kernel Learning Framework and Ultrasound Images. ULTRASOUND IN MEDICINE & BIOLOGY 2024; 50:1034-1044. [PMID: 38679514 DOI: 10.1016/j.ultrasmedbio.2024.03.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 03/10/2024] [Accepted: 03/30/2024] [Indexed: 05/01/2024]
Abstract
To properly treat and care for hepatic cystic echinococcosis (HCE), it is essential to make an accurate diagnosis before treatment. OBJECTIVE The objective of this study was to assess the diagnostic accuracy of computer-aided diagnosis techniques in classifying HCE ultrasound images into five subtypes. METHODS A total of 1820 HCE ultrasound images collected from 967 patients were included in the study. A multi-kernel learning method was developed to learn the texture and depth features of the ultrasound images. Combined kernel functions were built-in Support Vector Machine (MK-SVM) for the classification work. The experimental results were evaluated using five-fold cross-validation. Finally, our approach was compared with three other machine learning algorithms: the decision tree classifier, random forest, and gradient boosting decision tree. RESULTS Among all the methods used in the study, the MK-SVM achieved the highest accuracy of 96.6% on the fused feature set. CONCLUSION The multi-kernel learning method effectively learns different image features from ultrasound images by utilizing various kernels. The MK-SVM method, which combines the learning of texture features and depth features separately, has significant application value in HCE classification tasks.
Collapse
Affiliation(s)
- Zhengye Wang
- Center for Disease Control and Prevention, Xinjiang Production and Construction Corps, Urumqi, China; Ultrasound Department, State Key Laboratory of Pathogenesis, Prevention and Treatment of High Incidence Disease in Central Asia, The First Affiliated Hospital of Xinjiang Medical University, Urumqi, China
| | - Miao Wu
- College of Medical Engineering and Technology, Xinjiang Medical University, Urumqi, China
| | - Qian Liu
- Basic Medical College, Xinjiang Medical University, Urumqi, China
| | - Xiaorong Wang
- Ultrasound Department, State Key Laboratory of Pathogenesis, Prevention and Treatment of High Incidence Disease in Central Asia, The First Affiliated Hospital of Xinjiang Medical University, Urumqi, China
| | - Chuanbo Yan
- College of Medical Engineering and Technology, Xinjiang Medical University, Urumqi, China
| | - Tao Song
- Ultrasound Department, State Key Laboratory of Pathogenesis, Prevention and Treatment of High Incidence Disease in Central Asia, The First Affiliated Hospital of Xinjiang Medical University, Urumqi, China.
| |
Collapse
|
9
|
Rousta F, Esteki A, Shalbaf A, Sadeghi A, Moghadam PK, Voshagh A. Application of artificial intelligence in pancreas endoscopic ultrasound imaging- A systematic review. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 250:108205. [PMID: 38703435 DOI: 10.1016/j.cmpb.2024.108205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/04/2024] [Revised: 04/13/2024] [Accepted: 04/24/2024] [Indexed: 05/06/2024]
Abstract
The pancreas is a vital organ in digestive system which has significant health implications. It is imperative to evaluate and identify malignant pancreatic lesions promptly in light of the high mortality rate linked to such malignancies. Endoscopic Ultrasound (EUS) is a non-invasive precise technique to detect pancreas disorders, but it is highly operator dependent. Artificial intelligence (AI), including traditional machine learning (ML) and deep learning (DL) techniques can play a pivotal role to enhancing the performance of EUS regardless of operator. AI performs a critical function in the detection, classification, and segmentation of medical images. The utilization of AI-assisted systems has improved the accuracy and productivity of pancreatic analysis, including the detection of diverse pancreatic disorders (e.g., pancreatitis, masses, and cysts) as well as landmarks and parenchyma. This systematic review examines the rapidly developing domain of AI-assisted system in EUS of the pancreas. Its objective is to present a thorough study of the present research status and developments in this area. This paper explores the significant challenges of AI-assisted system in pancreas EUS imaging, highlights the potential of AI techniques in addressing these challenges, and suggests the scope for future research in domain of AI-assisted EUS systems.
Collapse
Affiliation(s)
- Fatemeh Rousta
- Department of Biomedical Engineering and Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Ali Esteki
- Department of Biomedical Engineering and Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Ahmad Shalbaf
- Department of Biomedical Engineering and Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - Amir Sadeghi
- Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Pardis Ketabi Moghadam
- Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Ardalan Voshagh
- Faculty of Electrical Engineering, Shahid Beheshti University, Tehran, Iran
| |
Collapse
|
10
|
Viswanathan AV, Pokaprakarn T, Kasaro MP, Shah HR, Prieto JC, Benabdelkader C, Sebastião YV, Sindano N, Stringer E, Stringer JSA. Deep learning to estimate gestational age from fly-to cineloop videos: A novel approach to ultrasound quality control. Int J Gynaecol Obstet 2024; 165:1013-1021. [PMID: 38189177 PMCID: PMC11214162 DOI: 10.1002/ijgo.15321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 12/05/2023] [Accepted: 12/10/2023] [Indexed: 01/09/2024]
Abstract
OBJECTIVE Low-cost devices have made obstetric sonography possible in settings where it was previously unfeasible, but ensuring quality and consistency at scale remains a challenge. In the present study, we sought to create a tool to reduce substandard fetal biometry measurement while minimizing care disruption. METHODS We developed a deep learning artificial intelligence (AI) model to estimate gestational age (GA) in the second and third trimester from fly-to cineloops-brief videos acquired during routine ultrasound biometry-and evaluated its performance in comparison to expert sonographer measurement. We then introduced random error into fetal biometry measurements and analyzed the ability of the AI model to flag grossly inaccurate measurements such as those that might be obtained by a novice. RESULTS The mean absolute error (MAE) of our model (±standard error) was 3.87 ± 0.07 days, compared to 4.80 ± 0.10 days for expert biometry (difference -0.92 days; 95% CI: -1.10 to -0.76). Based on simulated novice biometry with average absolute error of 7.5%, our model reliably detected cases where novice biometry differed from expert biometry by 10 days or more, with an area under the receiver operating characteristics curve of 0.93 (95% CI: 0.92, 0.95), sensitivity of 81.0% (95% CI: 77.9, 83.8), and specificity of 89.9% (95% CI: 88.1, 91.5). These results held across a range of sensitivity analyses, including where the model was provided suboptimal truncated fly-to cineloops. CONCLUSIONS Our AI model estimated GA more accurately than expert biometry. Because fly-to cineloop videos can be obtained without any change to sonographer workflow, the model represents a no-cost guardrail that could be incorporated into both low-cost and commercial ultrasound devices to prevent reporting of most gross GA estimation errors.
Collapse
Affiliation(s)
- Ambika V Viswanathan
- Department of Obstetrics and Gynecology, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
| | - Teeranan Pokaprakarn
- Department of Obstetrics and Gynecology, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
- Department of Biostatistics, University of North Carolina Gillings School of Global Public Health, Chapel Hill, North Carolina, USA
| | - Margaret P Kasaro
- Department of Obstetrics and Gynecology, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
- UNC Global Projects - Zambia LLC, Lusaka, Zambia
| | - Hina R Shah
- Department of Psychiatry, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
| | - Juan C Prieto
- Department of Psychiatry, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
| | - Chiraz Benabdelkader
- Department of Obstetrics and Gynecology, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
| | - Yuri V Sebastião
- Department of Obstetrics and Gynecology, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
| | | | - Elizabeth Stringer
- Department of Obstetrics and Gynecology, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
- UNC Global Projects - Zambia LLC, Lusaka, Zambia
| | - Jeffrey S A Stringer
- Department of Obstetrics and Gynecology, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
- UNC Global Projects - Zambia LLC, Lusaka, Zambia
| |
Collapse
|
11
|
Abubakar H, Al-Turjman F, Ameen ZS, Mubarak AS, Altrjman C. A hybridized feature extraction for COVID-19 multi-class classification on computed tomography images. Heliyon 2024; 10:e26939. [PMID: 38463848 PMCID: PMC10920381 DOI: 10.1016/j.heliyon.2024.e26939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2023] [Revised: 02/20/2024] [Accepted: 02/21/2024] [Indexed: 03/12/2024] Open
Abstract
COVID-19 has killed more than 5 million individuals worldwide within a short time. It is caused by SARS-CoV-2 which continuously mutates and produces more transmissible new different strains. It is therefore of great significance to diagnose COVID-19 early to curb its spread and reduce the death rate. Owing to the COVID-19 pandemic, traditional diagnostic methods such as reverse-transcription polymerase chain reaction (RT-PCR) are ineffective for diagnosis. Medical imaging is among the most effective techniques of respiratory disorders detection through machine learning and deep learning. However, conventional machine learning methods depend on extracted and engineered features, whereby the optimum features influence the classifier's performance. In this study, Histogram of Oriented Gradient (HOG) and eight deep learning models were utilized for feature extraction while K-Nearest Neighbour (KNN) and Support Vector Machines (SVM) were used for classification. A combined feature of HOG and deep learning feature was proposed to improve the performance of the classifiers. VGG-16 + HOG achieved 99.4 overall accuracy with SVM. This indicates that our proposed concatenated feature can enhance the SVM classifier's performance in COVID-19 detection.
Collapse
Affiliation(s)
- Hassana Abubakar
- Biomedical Engineering Department, Faculty of Engineering, Near East University, Mersin 10, Turkey
| | - Fadi Al-Turjman
- Artificial Intelligence Engineering Department, AI and Robotics Institute, Near East University, Mersin 10, Turkey
- Research Center for AI and IoT, Faculty of Engineering, University of Kyrenia, Mersin 10, Turkey
| | - Zubaida S. Ameen
- Operational Research Center in Healthcare, Near East University, Mersin 10, Turkey
| | - Auwalu S. Mubarak
- Operational Research Center in Healthcare, Near East University, Mersin 10, Turkey
| | - Chadi Altrjman
- Waterloo University, 200 University Avenue West. Waterloo, ON, Canada
| |
Collapse
|
12
|
Oliveira M, Wilming R, Clark B, Budding C, Eitel F, Ritter K, Haufe S. Benchmarking the influence of pre-training on explanation performance in MR image classification. Front Artif Intell 2024; 7:1330919. [PMID: 38469161 PMCID: PMC10925627 DOI: 10.3389/frai.2024.1330919] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 02/09/2024] [Indexed: 03/13/2024] Open
Abstract
Convolutional Neural Networks (CNNs) are frequently and successfully used in medical prediction tasks. They are often used in combination with transfer learning, leading to improved performance when training data for the task are scarce. The resulting models are highly complex and typically do not provide any insight into their predictive mechanisms, motivating the field of "explainable" artificial intelligence (XAI). However, previous studies have rarely quantitatively evaluated the "explanation performance" of XAI methods against ground-truth data, and transfer learning and its influence on objective measures of explanation performance has not been investigated. Here, we propose a benchmark dataset that allows for quantifying explanation performance in a realistic magnetic resonance imaging (MRI) classification task. We employ this benchmark to understand the influence of transfer learning on the quality of explanations. Experimental results show that popular XAI methods applied to the same underlying model differ vastly in performance, even when considering only correctly classified examples. We further observe that explanation performance strongly depends on the task used for pre-training and the number of CNN layers pre-trained. These results hold after correcting for a substantial correlation between explanation and classification performance.
Collapse
Affiliation(s)
- Marta Oliveira
- Division 8.44, Physikalisch-Technische Bundesanstalt, Berlin, Germany
| | - Rick Wilming
- Computer Science Department, Technische Universität Berlin, Berlin, Germany
| | - Benedict Clark
- Division 8.44, Physikalisch-Technische Bundesanstalt, Berlin, Germany
| | - Céline Budding
- Berlin Center for Advanced Neuroimaging (BCAN), Charité— Universitätsmedizin Berlin, Berlin, Germany
| | - Fabian Eitel
- Berlin Center for Advanced Neuroimaging (BCAN), Charité— Universitätsmedizin Berlin, Berlin, Germany
| | - Kerstin Ritter
- Berlin Center for Advanced Neuroimaging (BCAN), Charité— Universitätsmedizin Berlin, Berlin, Germany
| | - Stefan Haufe
- Division 8.44, Physikalisch-Technische Bundesanstalt, Berlin, Germany
- Computer Science Department, Technische Universität Berlin, Berlin, Germany
- Berlin Center for Advanced Neuroimaging (BCAN), Charité— Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
13
|
Li J, Jiang P, An Q, Wang GG, Kong HF. Medical image identification methods: A review. Comput Biol Med 2024; 169:107777. [PMID: 38104516 DOI: 10.1016/j.compbiomed.2023.107777] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Revised: 10/30/2023] [Accepted: 11/28/2023] [Indexed: 12/19/2023]
Abstract
The identification of medical images is an essential task in computer-aided diagnosis, medical image retrieval and mining. Medical image data mainly include electronic health record data and gene information data, etc. Although intelligent imaging provided a good scheme for medical image analysis over traditional methods that rely on the handcrafted features, it remains challenging due to the diversity of imaging modalities and clinical pathologies. Many medical image identification methods provide a good scheme for medical image analysis. The concepts pertinent of methods, such as the machine learning, deep learning, convolutional neural networks, transfer learning, and other image processing technologies for medical image are analyzed and summarized in this paper. We reviewed these recent studies to provide a comprehensive overview of applying these methods in various medical image analysis tasks, such as object detection, image classification, image registration, segmentation, and other tasks. Especially, we emphasized the latest progress and contributions of different methods in medical image analysis, which are summarized base on different application scenarios, including classification, segmentation, detection, and image registration. In addition, the applications of different methods are summarized in different application area, such as pulmonary, brain, digital pathology, brain, skin, lung, renal, breast, neuromyelitis, vertebrae, and musculoskeletal, etc. Critical discussion of open challenges and directions for future research are finally summarized. Especially, excellent algorithms in computer vision, natural language processing, and unmanned driving will be applied to medical image recognition in the future.
Collapse
Affiliation(s)
- Juan Li
- School of Information Engineering, Wuhan Business University, Wuhan, 430056, China; School of Artificial Intelligence, Wuchang University of Technology, Wuhan, 430223, China; Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China
| | - Pan Jiang
- School of Information Engineering, Wuhan Business University, Wuhan, 430056, China
| | - Qing An
- School of Artificial Intelligence, Wuchang University of Technology, Wuhan, 430223, China
| | - Gai-Ge Wang
- School of Computer Science and Technology, Ocean University of China, Qingdao, 266100, China.
| | - Hua-Feng Kong
- School of Information Engineering, Wuhan Business University, Wuhan, 430056, China.
| |
Collapse
|
14
|
Lawley A, Hampson R, Worrall K, Dobie G. Analysis of neural networks for routine classification of sixteen ultrasound upper abdominal cross sections. Abdom Radiol (NY) 2024; 49:651-661. [PMID: 38214722 PMCID: PMC10830611 DOI: 10.1007/s00261-023-04147-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Revised: 11/28/2023] [Accepted: 11/29/2023] [Indexed: 01/13/2024]
Abstract
PURPOSE Abdominal ultrasound screening requires the capture of multiple standardized plane views as per clinical guidelines. Currently, the extent of adherence to such guidelines is dependent entirely on the skills of the sonographer. The use of neural network classification has the potential to better standardize captured plane views and streamline plane capture reducing the time burden on operators by combatting operator variability. METHODS A dataset consisting of 16 routine upper abdominal ultrasound scans from 64 patients was used to test the classification accuracy of 9 neural networks. These networks were tested on both a small, idealised subset of 800 samples as well as full video sweeps of the region of interest using stratified sampling and transfer learning. RESULTS The highest validation accuracy attained by both GoogLeNet and InceptionV3 is 83.9% using transfer learning and the large sample set of 26,294 images. A top-2 accuracy of 95.1% was achieved using InceptionV3. Alexnet attained the highest accuracy of 79.5% (top-2 of 91.5%) for the smaller sample set of 800 images. The neural networks evaluated during this study were also successfully able to identify problematic individual cross sections such as between kidneys, with right and left kidney being accurately identified 78.6% and 89.7%, respectively. CONCLUSION Dataset size proved a more important factor in determining accuracy than network selection with more complex neural networks providing higher accuracy as dataset size increases and simpler linear neural networks providing better results where the dataset is small.
Collapse
Affiliation(s)
- Alistair Lawley
- Faculty Electronic and Electrical Engineering, University of Strathclyde, Glasgow, UK.
| | - Rory Hampson
- Faculty Electronic and Electrical Engineering, University of Strathclyde, Glasgow, UK
| | - Kevin Worrall
- Faculty of Engineering, University of Glasgow, Glasgow, UK
| | - Gordon Dobie
- Faculty Electronic and Electrical Engineering, University of Strathclyde, Glasgow, UK
| |
Collapse
|
15
|
Kaur H, Saini SK, Thakur N, Juneja M. Survey of Denoising, Segmentation and Classification of Pancreatic Cancer Imaging. Curr Med Imaging 2024; 20:e150523216892. [PMID: 37189279 DOI: 10.2174/1573405620666230515090523] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2022] [Revised: 03/10/2023] [Accepted: 04/03/2023] [Indexed: 05/17/2023]
Abstract
BACKGROUND Pancreatic cancer is one of the most serious problems that has taken many lives worldwide. The diagnostic procedure using the traditional approaches was manual by visually analyzing the large volumes of the dataset, making it time-consuming and prone to subjective errors. Hence the need for the computer-aided diagnosis system (CADs) emerged that comprises the machine and deep learning approaches for denoising, segmentation and classification of pancreatic cancer. INTRODUCTION There are different modalities used for the diagnosis of pancreatic cancer, such as Positron Emission Tomography/Computed Tomography (PET/CT), Magnetic Resonance Imaging (MRI), Multiparametric-MRI (Mp-MRI), Radiomics and Radio-genomics. Although these modalities gave remarkable results in diagnosis on the basis of different criteria. CT is the most commonly used modality that produces detailed and fine contrast images of internal organs of the body. However, it may also contain a certain amount of gaussian and rician noise that is necessary to be preprocessed before segmentation of the required region of interest (ROI) from the images and classification of cancer. METHOD This paper analyzes different methodologies used for the complete diagnosis of pancreatic cancer, including the denoising, segmentation and classification, along with the challenges and future scope for the diagnosis of pancreatic cancer. RESULT Various filters are used for denoising and image smoothening and filters as gaussian scale mixture process, non-local means, median filter, adaptive filter and average filter have been used more for better results. CONCLUSION In terms of segmentation, atlas based region-growing method proved to give better results as compared to the state of the art whereas, for the classification, deep learning approaches outperformed other methodologies to classify the images as cancerous and non- cancerous. These methodologies have proved that CAD systems have become a better solution to the ongoing research proposals for the detection of pancreatic cancer worldwide.
Collapse
Affiliation(s)
- Harjinder Kaur
- Department of UIET, University of Punjab, Chandigarh, 160014, India
| | | | - Niharika Thakur
- Department of UIET, University of Punjab, Chandigarh, 160014, India
| | - Mamta Juneja
- Department of UIET, University of Punjab, Chandigarh, 160014, India
| |
Collapse
|
16
|
Zhao D, Wang W, Tang T, Zhang YY, Yu C. Current progress in artificial intelligence-assisted medical image analysis for chronic kidney disease: A literature review. Comput Struct Biotechnol J 2023; 21:3315-3326. [PMID: 37333860 PMCID: PMC10275698 DOI: 10.1016/j.csbj.2023.05.029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 05/28/2023] [Accepted: 05/28/2023] [Indexed: 06/20/2023] Open
Abstract
Chronic kidney disease (CKD) causes irreversible damage to kidney structure and function. Arising from various etiologies, risk factors for CKD include hypertension and diabetes. With a progressively increasing global prevalence, CKD is an important public health problem worldwide. Medical imaging has become an important diagnostic tool for CKD through the non-invasive identification of macroscopic renal structural abnormalities. Artificial intelligence (AI)-assisted medical imaging techniques aid clinicians in the analysis of characteristics that cannot be easily discriminated by the naked eye, providing valuable information for the identification and management of CKD. Recent studies have demonstrated the effectiveness of AI-assisted medical image analysis as a clinical support tool using radiomics- and deep learning-based AI algorithms for improving the early detection, pathological assessment, and prognostic evaluation of various forms of CKD, including autosomal dominant polycystic kidney disease. Herein, we provide an overview of the potential roles of AI-assisted medical image analysis for the diagnosis and management of CKD.
Collapse
Affiliation(s)
- Dan Zhao
- Department of Nephrology, Tongji Hospital, School of Medicine, Tongji University, Shanghai 200065, China
| | - Wei Wang
- Department of Radiology, Tongji Hospital, School of Medicine, Tongji University, Shanghai 200065, China
| | - Tian Tang
- Department of Nephrology, Tongji Hospital, School of Medicine, Tongji University, Shanghai 200065, China
| | - Ying-Ying Zhang
- Department of Nephrology, Tongji Hospital, School of Medicine, Tongji University, Shanghai 200065, China
| | - Chen Yu
- Department of Nephrology, Tongji Hospital, School of Medicine, Tongji University, Shanghai 200065, China
| |
Collapse
|
17
|
Dadoun H, Delingette H, Rousseau AL, de Kerviler E, Ayache N. Deep clustering for abdominal organ classification in ultrasound imaging. J Med Imaging (Bellingham) 2023; 10:034502. [PMID: 37216152 PMCID: PMC10194825 DOI: 10.1117/1.jmi.10.3.034502] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Revised: 02/27/2023] [Accepted: 04/27/2023] [Indexed: 05/24/2023] Open
Abstract
Purpose The purpose of this study is to examine the utilization of unlabeled data for abdominal organ classification in multi-label (non-mutually exclusive classes) ultrasound images, as an alternative to the conventional transfer learning approach. Approach We present a new method for classifying abdominal organs in ultrasound images. Unlike previous approaches that only relied on labeled data, we consider the use of both labeled and unlabeled data. To explore this approach, we first examine the application of deep clustering for pretraining a classification model. We then compare two training methods, fine-tuning with labeled data through supervised learning and fine-tuning with both labeled and unlabeled data using semisupervised learning. All experiments were conducted on a large dataset of unlabeled images (n u = 84967 ) and a small set of labeled images (n s = 2742 ) comprising progressively 10%, 20%, 50%, and 100% of the images. Results We show that for supervised fine-tuning, deep clustering is an effective pre-training method, with performance matching that of ImageNet pre-training using five times less labeled data. For semi-supervised learning, deep clustering pre-training also yields higher performance when the amount of labeled data is limited. Best performance is obtained with deep clustering pre-training combined with semi-supervised learning and 2742 labeled example images with an F 1 -score weighted average of 84.1%. Conclusions This method can be used as a tool to preprocess large unprocessed databases, thus reducing the need for prior annotations of abdominal ultrasound studies for the training of image classification algorithms, which in turn could improve the clinical use of ultrasound images.
Collapse
Affiliation(s)
- Hind Dadoun
- Université Côte d’Azur, Inria Epione Team, Sophia Antipolis, Nice, France
| | - Hervé Delingette
- Université Côte d’Azur, Inria Epione Team, Sophia Antipolis, Nice, France
| | | | | | - Nicholas Ayache
- Université Côte d’Azur, Inria Epione Team, Sophia Antipolis, Nice, France
| |
Collapse
|
18
|
Intharah T, Wiratchawa K, Wanna Y, Junsawang P, Titapun A, Techasen A, Boonrod A, Laopaiboon V, Chamadol N, Khuntikeo N. BiTNet: Hybrid deep convolutional model for ultrasound image analysis of human biliary tract and its applications. Artif Intell Med 2023; 139:102539. [PMID: 37100509 DOI: 10.1016/j.artmed.2023.102539] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2022] [Revised: 03/15/2023] [Accepted: 03/27/2023] [Indexed: 04/03/2023]
Abstract
Certain life-threatening abnormalities, such as cholangiocarcinoma, in the human biliary tract are curable if detected at an early stage, and ultrasonography has been proven to be an effective tool for identifying them. However, the diagnosis often requires a second opinion from experienced radiologists, who are usually overwhelmed by many cases. Therefore, we propose a deep convolutional neural network model, named biliary tract network (BiTNet), developed to solve problems in the current screening system and to avoid overconfidence issues of traditional deep convolutional neural networks. Additionally, we present an ultrasound image dataset for the human biliary tract and demonstrate two artificial intelligence (AI) applications: auto-prescreening and assisting tools. The proposed model is the first AI model to automatically screen and diagnose upper-abdominal abnormalities from ultrasound images in real-world healthcare scenarios. Our experiments suggest that prediction probability has an impact on both applications, and our modifications to EfficientNet solve the overconfidence problem, thereby improving the performance of both applications and of healthcare professionals. The proposed BiTNet can reduce the workload of radiologists by 35% while keeping the false negatives to as low as 1 out of every 455 images. Our experiments involving 11 healthcare professionals with four different levels of experience reveal that BiTNet improves the diagnostic performance of participants of all levels. The mean accuracy and precision of the participants with BiTNet as an assisting tool (0.74 and 0.61, respectively) are statistically higher than those of participants without the assisting tool (0.50 and 0.46, respectively (p<0.001)). These experimental results demonstrate the high potential of BiTNet for use in clinical settings.
Collapse
Affiliation(s)
- Thanapong Intharah
- Visual Intelligence Laboratory, Department of Statistics, Faculty of Science, Khon Kaen University, Khon Kaen, Thailand.
| | - Kannika Wiratchawa
- Visual Intelligence Laboratory, Department of Statistics, Faculty of Science, Khon Kaen University, Khon Kaen, Thailand
| | - Yupaporn Wanna
- Visual Intelligence Laboratory, Department of Statistics, Faculty of Science, Khon Kaen University, Khon Kaen, Thailand
| | - Prem Junsawang
- Visual Intelligence Laboratory, Department of Statistics, Faculty of Science, Khon Kaen University, Khon Kaen, Thailand
| | - Attapol Titapun
- Department of Surgery, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand
| | - Anchalee Techasen
- Faculty of Associated Medical Sciences, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand
| | - Arunnit Boonrod
- Department of Radiology, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand
| | - Vallop Laopaiboon
- Department of Radiology, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand
| | - Nittaya Chamadol
- Department of Radiology, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand
| | - Narong Khuntikeo
- Department of Surgery, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand.
| |
Collapse
|
19
|
Waisberg E, Ong J, Kamran SA, Paladugu P, Zaman N, Lee AG, Tavakkoli A. Transfer learning as an AI-based solution to address limited datasets in space medicine. LIFE SCIENCES IN SPACE RESEARCH 2023; 36:36-38. [PMID: 36682827 DOI: 10.1016/j.lssr.2022.12.002] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Revised: 12/03/2022] [Accepted: 12/30/2022] [Indexed: 06/17/2023]
Abstract
The advent of artificial intelligence (AI) has a promising role in the future long-duration spaceflight missions. Traditional AI algorithms rely on training and testing data from the same domain. However, astronaut medical data is naturally limited to a small sample size and often difficult to collect, leading to extremely limited datasets. This significantly limits the ability of traditional machine learning methodologies. Transfer learning is a potential solution to overcome this dataset size limitation and can help improve training time and performance of a neural networks. We discuss the unique challenges of space medicine in producing datasets and transfer learning as an emerging technique to address these issues.
Collapse
Affiliation(s)
- Ethan Waisberg
- University College Dublin School of Medicine, Belfield, Dublin, Ireland.
| | - Joshua Ong
- Michigan Medicine, University of Michigan, Ann Arbor, United States
| | - Sharif Amit Kamran
- Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, Nevada, United States
| | - Phani Paladugu
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania, United States; Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
| | - Nasif Zaman
- Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, Nevada, United States
| | - Andrew G Lee
- Center for Space Medicine, Baylor College of Medicine, Houston, Texas, United States; Department of Ophthalmology, Blanton Eye Institute, Houston Methodist Hospital, Houston, Texas, United States; The Houston Methodist Research Institute, Houston Methodist Hospital, Houston, Texas, United States; Departments of Ophthalmology, Neurology, and Neurosurgery, Weill Cornell Medicine, New York, New York, United States; Department of Ophthalmology, University of Texas Medical Branch, Galveston, Texas, United States; University of Texas MD Anderson Cancer Center, Houston, Texas, United States; Texas A&M College of Medicine, Texas, United States; Department of Ophthalmology, The University of Iowa Hospitals and Clinics, Iowa City, Iowa, United States
| | - Alireza Tavakkoli
- Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, Nevada, United States
| |
Collapse
|
20
|
Li H, Shen C, Wang G, Sun Q, Yu K, Li Z, Liang X, Chen R, Wu H, Wang F, Wang Z, Lian C. BloodNet: An attention-based deep network for accurate, efficient, and costless bloodstain time since deposition inference. Brief Bioinform 2023; 24:6960974. [PMID: 36572655 DOI: 10.1093/bib/bbac557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2022] [Revised: 10/28/2022] [Indexed: 12/28/2022] Open
Abstract
The time since deposition (TSD) of a bloodstain, i.e., the time of a bloodstain formation is an essential piece of biological evidence in crime scene investigation. The practical usage of some existing microscopic methods (e.g., spectroscopy or RNA analysis technology) is limited, as their performance strongly relies on high-end instrumentation and/or rigorous laboratory conditions. This paper presents a practically applicable deep learning-based method (i.e., BloodNet) for efficient, accurate, and costless TSD inference from a macroscopic view, i.e., by using easily accessible bloodstain photos. To this end, we established a benchmark database containing around 50,000 photos of bloodstains with varying TSDs. Capitalizing on such a large-scale database, BloodNet adopted attention mechanisms to learn from relatively high-resolution input images the localized fine-grained feature representations that were highly discriminative between different TSD periods. Also, the visual analysis of the learned deep networks based on the Smooth Grad-CAM tool demonstrated that our BloodNet can stably capture the unique local patterns of bloodstains with specific TSDs, suggesting the efficacy of the utilized attention mechanism in learning fine-grained representations for TSD inference. As a paired study for BloodNet, we further conducted a microscopic analysis using Raman spectroscopic data and a machine learning method based on Bayesian optimization. Although the experimental results show that such a new microscopic-level approach outperformed the state-of-the-art by a large margin, its inference accuracy is significantly lower than BloodNet, which further justifies the efficacy of deep learning techniques in the challenging task of bloodstain TSD inference. Our code is publically accessible via https://github.com/shenxiaochenn/BloodNet. Our datasets and pre-trained models can be freely accessed via https://figshare.com/articles/dataset/21291825.
Collapse
Affiliation(s)
- Huiyu Li
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Chen Shen
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Gongji Wang
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Qinru Sun
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Kai Yu
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Zefeng Li
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - XingGong Liang
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Run Chen
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Hao Wu
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Fan Wang
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Zhenyuan Wang
- Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine & Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Chunfeng Lian
- School of Mathematics and Statistics, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| |
Collapse
|
21
|
Hao D, Ahsan M, Salim T, Duarte-Rojo A, Esmaeel D, Zhang Y, Arefan D, Wu S. A self-training teacher-student model with an automatic label grader for abdominal skeletal muscle segmentation. Artif Intell Med 2022; 132:102366. [DOI: 10.1016/j.artmed.2022.102366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Revised: 06/15/2022] [Accepted: 07/14/2022] [Indexed: 11/02/2022]
|
22
|
The Fusion Application of Deep Learning Biological Image Visualization Technology and Human-Computer Interaction Intelligent Robot in Dance Movements. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:2538896. [PMID: 36177314 PMCID: PMC9514919 DOI: 10.1155/2022/2538896] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Accepted: 09/06/2022] [Indexed: 11/18/2022]
Abstract
The paper aims to apply the deep learning-based image visualization technology to extract, recognize, and analyze human skeleton movements and evaluate the effect of the deep learning-based human-computer interaction (HCI) system. Dance education is researched. Firstly, the Visual Geometry Group Network (VGGNet) is optimized using Convolutional Neural Network (CNN). Then, the VGGNet extracts the human skeleton movements in the OpenPose database. Secondly, the Long Short-Term Memory (LSTM) network is optimized and recognizes human skeleton movements. Finally, an HCI system for dance education is designed based on the extraction and recognition methods of human skeleton movements. Results demonstrate that the highest extraction accuracy is 96%, and the average recognition accuracy of different dance movements is stable. The effectiveness of the proposed model is verified. The recognition accuracy of the optimized F-Multiple LSTMs is increased to 88.9%, suitable for recognizing human skeleton movements. The dance education HCI system’s interactive accuracy built by deep learning-based visualization technology reaches 92%; the overall response time is distributed between 5.1 s and 5.9 s. Hence, the proposed model has excellent instantaneity. Therefore, the deep learning-based image visualization technology has enormous potential in human movement recognition, and combining deep learning and HCI plays a significant role.
Collapse
|
23
|
Xia D, Chen G, Wu K, Yu M, Zhang Z, Lu Y, Xu L, Wang Y. Research progress and hotspot of the artificial intelligence application in the ultrasound during 2011-2021: A bibliometric analysis. Front Public Health 2022; 10:990708. [PMID: 36187670 PMCID: PMC9520910 DOI: 10.3389/fpubh.2022.990708] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2022] [Accepted: 08/26/2022] [Indexed: 01/26/2023] Open
Abstract
Ultrasound, as a common clinical examination tool, inevitably has human errors due to the limitations of manual operation. Artificial intelligence is an advanced computer program that can solve this problem. Therefore, the relevant literature on the application of artificial intelligence in the ultrasonic field from 2011 to 2021 was screened by authors from the Web of Science Core Collection, which aims to summarize the trend of artificial intelligence application in the field of ultrasound, meanwhile, visualize and predict research hotspots. A total of 908 publications were included in the study. Overall, the number of global publications is on the rise, and studies on the application of artificial intelligence in the field of ultrasound continue to increase. China has made the largest contribution in this field. In terms of institutions, Fudan University has the most number of publications. Recently, IEEE Access is the most published journal. Suri J. S. published most of the articles and had the highest number of citations in this field (29 articles). It's worth noting that, convolutional neural networks (CNN), as a kind of deep learning algorithm, was considered to bring better image analysis and processing ability in recent most-cited articles. According to the analysis of keywords, the latest keyword is "COVID-19" (2020.8). The co-occurrence analysis of keywords by VOSviewer visually presented four clusters which consisted of "deep learning," "machine learning," "application in the field of visceral organs," and "application in the field of cardiovascular". The latest hot words of these clusters were "COVID-19; neural-network; hepatocellular carcinoma; atherosclerotic plaques". This study reveals the importance of multi-institutional and multi-field collaboration in promoting research progress.
Collapse
Affiliation(s)
- Demeng Xia
- Luodian Clinical Drug Research Center, Shanghai Baoshan Luodian Hospital, Shanghai University, Shanghai, China
| | - Gaoqi Chen
- Department of Pancreatic Hepatobiliary Surgery, Changhai Hospital, Naval Medical University, Shanghai, China
| | - Kaiwen Wu
- Department of Gastroenterology, The Third People's Hospital of Chengdu, The Affiliated Hospital of Southwest Jiaotong University, Chengdu, China
| | - Mengxin Yu
- Department of Ultrasound, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Zhentao Zhang
- Department of Clinical Medicine, The Naval Medical University, Shanghai, China
| | - Yixian Lu
- Department of Clinical Medicine, The Naval Medical University, Shanghai, China
| | - Lisha Xu
- Department of Ultrasound, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China,Lisha Xu
| | - Yin Wang
- Department of Ultrasound, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China,*Correspondence: Yin Wang
| |
Collapse
|
24
|
Ong J, Tavakkoli A, Zaman N, Kamran SA, Waisberg E, Gautam N, Lee AG. Terrestrial health applications of visual assessment technology and machine learning in spaceflight associated neuro-ocular syndrome. NPJ Microgravity 2022; 8:37. [PMID: 36008494 PMCID: PMC9411571 DOI: 10.1038/s41526-022-00222-7] [Citation(s) in RCA: 43] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2022] [Accepted: 08/01/2022] [Indexed: 02/05/2023] Open
Abstract
The neuro-ocular effects of long-duration spaceflight have been termed Spaceflight Associated Neuro-Ocular Syndrome (SANS) and are a potential challenge for future, human space exploration. The underlying pathogenesis of SANS remains ill-defined, but several emerging translational applications of terrestrial head-mounted, visual assessment technology and machine learning frameworks are being studied for potential use in SANS. To develop such technology requires close consideration of the spaceflight environment which is limited in medical resources and imaging modalities. This austere environment necessitates the utilization of low mass, low footprint technology to build a visual assessment system that is comprehensive, accessible, and efficient. In this paper, we discuss the unique considerations for developing this technology for SANS and translational applications on Earth. Several key limitations observed in the austere spaceflight environment share similarities to barriers to care for underserved areas on Earth. We discuss common terrestrial ophthalmic diseases and how machine learning and visual assessment technology for SANS can help increase screening for early intervention. The foundational developments with this novel system may help protect the visual health of both astronauts and individuals on Earth.
Collapse
Affiliation(s)
- Joshua Ong
- University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | - Alireza Tavakkoli
- Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, NV, USA
| | - Nasif Zaman
- Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, NV, USA
| | - Sharif Amit Kamran
- Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, NV, USA
| | - Ethan Waisberg
- University College Dublin School of Medicine, Belfield, Dublin, Ireland
| | - Nikhil Gautam
- Department of Computer Science, Rice University, Houston, TX, USA
| | - Andrew G Lee
- Center for Space Medicine, Baylor College of Medicine, Houston, TX, USA. .,Department of Ophthalmology, Blanton Eye Institute, Houston Methodist Hospital, Houston, TX, USA. .,The Houston Methodist Research Institute, Houston Methodist Hospital, Houston, TX, USA. .,Departments of Ophthalmology, Neurology, and Neurosurgery, Weill Cornell Medicine, New York, NY, USA. .,Department of Ophthalmology, University of Texas Medical Branch, Galveston, TX, USA. .,University of Texas MD Anderson Cancer Center, Houston, TX, USA. .,Texas A&M College of Medicine, Bryan, TX, USA. .,Department of Ophthalmology, The University of Iowa Hospitals and Clinics, Iowa City, IA, USA.
| |
Collapse
|
25
|
Kornblith AE, Addo N, Dong R, Rogers R, Grupp-Phelan J, Butte A, Gupta P, Callcut RA, Arnaout R. Development and Validation of a Deep Learning Strategy for Automated View Classification of Pediatric Focused Assessment With Sonography for Trauma. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2022; 41:1915-1924. [PMID: 34741469 PMCID: PMC9072593 DOI: 10.1002/jum.15868] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Revised: 10/14/2021] [Accepted: 10/19/2021] [Indexed: 06/09/2023]
Abstract
OBJECTIVE Pediatric focused assessment with sonography for trauma (FAST) is a sequence of ultrasound views rapidly performed by clinicians to diagnose hemorrhage. A technical limitation of FAST is the lack of expertise to consistently acquire all required views. We sought to develop an accurate deep learning view classifier using a large heterogeneous dataset of clinician-performed pediatric FAST. METHODS We developed and conducted a retrospective cohort analysis of a deep learning view classifier on real-world FAST studies performed on injured children less than 18 years old in two pediatric emergency departments by 30 different clinicians. FAST was randomly distributed to training, validation, and test datasets, 70:20:10; each child was represented in only one dataset. The primary outcome was view classifier accuracy for video clips and still frames. RESULTS There were 699 FAST studies, representing 4925 video clips and 1,062,612 still frames, performed by 30 different clinicians. The overall classification accuracy was 97.8% (95% confidence interval [CI]: 96.0-99.0) for video clips and 93.4% (95% CI: 93.3-93.6) for still frames. Per view still frames were classified with an accuracy: 96.0% (95% CI: 95.9-96.1) cardiac, 99.8% (95% CI: 99.8-99.8) pleural, 95.2% (95% CI: 95.0-95.3) abdominal upper quadrants, and 95.9% (95% CI: 95.8-96.0) suprapubic. CONCLUSION A deep learning classifier can accurately predict pediatric FAST views. Accurate view classification is important for quality assurance and feasibility of a multi-stage deep learning FAST model to enhance the evaluation of injured children.
Collapse
Affiliation(s)
- Aaron E Kornblith
- Department of Emergency Medicine, University of California, San Francisco, CA, USA
- Department of Pediatrics, University of California, San Francisco, CA, USA
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, USA
| | - Newton Addo
- Department of Emergency Medicine, University of California, San Francisco, CA, USA
- Department of Medicine, Division of Cardiology, University of California, San Francisco, CA, USA
| | - Ruolei Dong
- Department of Bioengineering, University of California, Berkeley, CA, USA
- Department of Bioengineering and Therapeutic Sciences, University of California, San Francisco, CA, USA
| | - Robert Rogers
- Center for Digital Health Innovation, University of California, San Francisco, CA, USA
| | - Jacqueline Grupp-Phelan
- Department of Emergency Medicine, University of California, San Francisco, CA, USA
- Department of Pediatrics, University of California, San Francisco, CA, USA
| | - Atul Butte
- Department of Pediatrics, University of California, San Francisco, CA, USA
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, USA
| | - Pavan Gupta
- Center for Digital Health Innovation, University of California, San Francisco, CA, USA
| | - Rachael A Callcut
- Center for Digital Health Innovation, University of California, San Francisco, CA, USA
- Department of Surgery, University of California, Davis, CA, USA
| | - Rima Arnaout
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, USA
- Department of Medicine, Division of Cardiology, University of California, San Francisco, CA, USA
| |
Collapse
|
26
|
Malik N, Bzdok D. From YouTube to the brain: Transfer learning can improve brain-imaging predictions with deep learning. Neural Netw 2022; 153:325-338. [PMID: 35777174 DOI: 10.1016/j.neunet.2022.06.014] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Revised: 04/20/2022] [Accepted: 06/09/2022] [Indexed: 12/01/2022]
Abstract
Deep learning has recently achieved best-in-class performance in several fields, including biomedical domains such as X-ray images. Yet, data scarcity poses a strict limit on training successful deep learning systems in many, if not most, biomedical applications, including those involving brain images. In this study, we translate state-of-the-art transfer learning techniques for single-subject prediction of simpler (sex and age) and more complex phenotypes (number of people in household, household income, fluid intelligence and smoking behavior). We fine-tuned 2D and 3D ResNet-18 convolutional neural networks for target phenotype predictions from brain images of ∼40,000 UK Biobank participants, after pretraining on YouTube videos from the Kinetics dataset and natural images from the ImageNet dataset. Transfer learning was effective on several phenotypes, especially sex and age classification. Additionally, transfer learning in particular outperformed deep learning models trained from scratch especially on smaller sample sizes. The out-of-sample performance using transfer learning from previously learned knowledge based on real-world images and videos could unlock the potential in many areas of imaging neuroscience where deep learning solutions are currently infeasible.
Collapse
Affiliation(s)
- Nahiyan Malik
- School of Computer Science, McGill University, Montreal, QC, Canada; Mila - Quebec Artificial Intelligence Institute, Montreal, QC, Canada.
| | - Danilo Bzdok
- School of Computer Science, McGill University, Montreal, QC, Canada; Mila - Quebec Artificial Intelligence Institute, Montreal, QC, Canada; McConnell Brain Imaging Centre, Montreal Neurological Institute (MNI), McGill University, Montreal, QC, Canada; Department of Biomedical Engineering, Faculty of Medicine, McGill University, Montreal, QC, Canada.
| |
Collapse
|
27
|
Bonmati E, Hu Y, Grimwood A, Johnson GJ, Goodchild G, Keane MG, Gurusamy K, Davidson B, Clarkson MJ, Pereira SP, Barratt DC. Voice-Assisted Image Labeling for Endoscopic Ultrasound Classification Using Neural Networks. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1311-1319. [PMID: 34962866 DOI: 10.1109/tmi.2021.3139023] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Ultrasound imaging is a commonly used technology for visualising patient anatomy in real-time during diagnostic and therapeutic procedures. High operator dependency and low reproducibility make ultrasound imaging and interpretation challenging with a steep learning curve. Automatic image classification using deep learning has the potential to overcome some of these challenges by supporting ultrasound training in novices, as well as aiding ultrasound image interpretation in patient with complex pathology for more experienced practitioners. However, the use of deep learning methods requires a large amount of data in order to provide accurate results. Labelling large ultrasound datasets is a challenging task because labels are retrospectively assigned to 2D images without the 3D spatial context available in vivo or that would be inferred while visually tracking structures between frames during the procedure. In this work, we propose a multi-modal convolutional neural network (CNN) architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. We use a CNN composed of two branches, one for voice data and another for image data, which are joined to predict image labels from the spoken names of anatomical landmarks. The network was trained using recorded verbal comments from expert operators. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels. We conclude that the addition of spoken commentaries can increase the performance of ultrasound image classification, and eliminate the burden of manually labelling large EUS datasets necessary for deep learning applications.
Collapse
|
28
|
Song Y, Zheng J, Lei L, Ni Z, Zhao B, Hu Y. CT2US: Cross-modal transfer learning for kidney segmentation in ultrasound images with synthesized data. ULTRASONICS 2022; 122:106706. [PMID: 35149255 DOI: 10.1016/j.ultras.2022.106706] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2021] [Revised: 01/16/2022] [Accepted: 02/04/2022] [Indexed: 06/14/2023]
Abstract
Accurate segmentation of kidney in ultrasound images is a vital procedure in clinical diagnosis and interventional operation. In recent years, deep learning technology has demonstrated promising prospects in medical image analysis. However, due to the inherent problems of ultrasound images, data with annotations are scarce and arduous to acquire, hampering the application of data-hungry deep learning methods. In this paper, we propose cross-modal transfer learning from computerized tomography (CT) to ultrasound (US) by leveraging annotated data in the CT modality. In particular, we adopt cycle generative adversarial network (CycleGAN) to synthesize US images from CT data and construct a transition dataset to mitigate the immense domain discrepancy between US and CT. Mainstream convolutional neural networks such as U-Net, U-Res, PSPNet, and DeepLab v3+ are pretrained on the transition dataset and then transferred to real US images. We first trained CNN models on a data set composed of 50 ultrasound images and validated them on a validation set composed of 30 ultrasound images. In addition, we selected 82 ultrasound images from another hospital to construct a cross-site data set to verify the generalization performance of the models. The experimental results show that with our proposed transfer learning strategy, the segmentation accuracy in dice similarity coefficient (DSC) reaches 0.853 for U-Net, 0.850 for U-Res, 0.826 for PSPNet and 0.827 for DeepLab v3+ on the cross-site test set. Compared with training from scratch, the accuracy improvement was 0.127, 0.097, 0.105 and 0.036 respectively. Our transfer learning strategy effectively improves the accuracy and generalization ability of ultrasound image segmentation model with limited training data.
Collapse
Affiliation(s)
- Yuxin Song
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China and University of Chinese Academy of Sciences, Beijing 100039, China.
| | - Jing Zheng
- Department of Ultrasound, First Affiliated Hospital of Southern University of Science and Technology, Second Clinical College of Jinan University, Shenzhen People's Hospital, Shenzhen, 518020, China.
| | - Long Lei
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China.
| | - Zhipeng Ni
- Department of Ultrasound, First Affiliated Hospital of Southern University of Science and Technology, Second Clinical College of Jinan University, Shenzhen People's Hospital, Shenzhen, 518020, China.
| | - Baoliang Zhao
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China; Pazhou Lab, Guangzhou 510320, China.
| | - Ying Hu
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China; Pazhou Lab, Guangzhou 510320, China.
| |
Collapse
|
29
|
Object-Specific Four-Path Network for Stroke Risk Stratification of Carotid Arteries in Ultrasound Images. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2022; 2022:2014349. [PMID: 35509862 PMCID: PMC9061007 DOI: 10.1155/2022/2014349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Accepted: 03/31/2022] [Indexed: 11/18/2022]
Abstract
Atherosclerotic carotid plaques have been shown to be closely associated with the risk of stroke. Since patients with symptomatic carotid plaques have a greater risk for stroke, stroke risk stratification based on the classification of carotid plaques into symptomatic or asymptomatic types is crucial in diagnosis, treatment planning, and medical treatment monitoring. A deep learning technique would be a good choice for implementing classification. Usually, to acquire a high-accuracy classification, a specific network architecture needs to be designed for a given classification task. In this study, we propose an object-specific four-path network (OSFP-Net) for stroke risk assessment by integrating ultrasound carotid plaques in both transverse and longitudinal sections of the bilateral carotid arteries. Each path of the OSFP-Net comprises of a feature extraction subnetwork (FE) and a feature downsampling subnetwork (FD). The FEs in the four paths use the same network structure to automatically extract features from ultrasound images of carotid plaques. The FDs use different object-specific pooling strategies for feature downsampling based on the observation that the sizes and shapes in the feature maps obtained from FEs should be different. The object-specific pooling strategies enable the network to accept arbitrarily sized carotid plaques as input and to capture a more informative context for improving the classification accuracy. Extensive experimental studies on a clinical dataset consisting of 333 subjects with 1332 carotid plaques show the superiority of our OSFP-Net against several state-of-the-art deep learning-based methods. The experimental results demonstrate better clinical agreement between the ground truth and the prediction, which indicates its great potential for use as a risk stratification and as a monitoring tool in the management of patients at risk for stroke.
Collapse
|
30
|
Sivanandan R, Jayakumari J. Development of a Novel CNN Architecture for Improved Diagnosis from Liver Ultrasound Tumor Images. INT J UNCERTAIN FUZZ 2022. [DOI: 10.1142/s0218488522500088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Malignant liver tumors are considered as one of the most common cancers and a leading cause of cancer death worldwide. While using convolutional neural networks (CNNs) for feature extraction from ultrasound (US) images and tasks thereafter, most works focus on pre-trained architectures using transfer learning which can sometimes cause negative transfer and reduced performance in medical domain. A new method based on Pascal’s Triangle was developed for feature extraction using CNN. The convolutions and the kernels in Pascal’s Triangle based CNN (PT-CNN) are according to the coefficients at each level of Pascal’s Triangle. Due to the fuzzy nature of US images, the input layer takes a combination of the image and its neutrosophically pre-processed components as a single unit to improve noise robustness. The proposed CNN when implemented as a backbone for feature extraction for binary classification, gave validation accuracy > 90% and test accuracy >95% than other state-of-the-art CNN architectures. For tumor segmentation using Mask R-CNN framework, the aggregated feature maps from all convolutional layers were given as the input to the region proposal network for multiscale region proposals and to facilitate the detection of tumors of varying sizes. This gave an F1 score > 0.95 when compared with other architectures as backbone on Mask R-CNN framework and simple U-Net based segmentation. It is suggested that the promising results of PT-CNN as a feature extraction backbone could be further investigated in other domains.
Collapse
Affiliation(s)
- Revathy Sivanandan
- Department of Electronics and Communication Engineering, Mar Baselios College of Engineering and Technology, Kerala, India
| | - J. Jayakumari
- Department of Electronics and Communication Engineering, Mar Baselios College of Engineering and Technology, Kerala, India
| |
Collapse
|
31
|
Kim HE, Cosa-Linan A, Santhanam N, Jannesari M, Maros ME, Ganslandt T. Transfer learning for medical image classification: a literature review. BMC Med Imaging 2022; 22:69. [PMID: 35418051 PMCID: PMC9007400 DOI: 10.1186/s12880-022-00793-7] [Citation(s) in RCA: 174] [Impact Index Per Article: 58.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Accepted: 03/30/2022] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND Transfer learning (TL) with convolutional neural networks aims to improve performances on a new task by leveraging the knowledge of similar tasks learned in advance. It has made a major contribution to medical image analysis as it overcomes the data scarcity problem as well as it saves time and hardware resources. However, transfer learning has been arbitrarily configured in the majority of studies. This review paper attempts to provide guidance for selecting a model and TL approaches for the medical image classification task. METHODS 425 peer-reviewed articles were retrieved from two databases, PubMed and Web of Science, published in English, up until December 31, 2020. Articles were assessed by two independent reviewers, with the aid of a third reviewer in the case of discrepancies. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. We investigated articles focused on selecting backbone models and TL approaches including feature extractor, feature extractor hybrid, fine-tuning and fine-tuning from scratch. RESULTS The majority of studies (n = 57) empirically evaluated multiple models followed by deep models (n = 33) and shallow (n = 24) models. Inception, one of the deep models, was the most employed in literature (n = 26). With respect to the TL, the majority of studies (n = 46) empirically benchmarked multiple approaches to identify the optimal configuration. The rest of the studies applied only a single approach for which feature extractor (n = 38) and fine-tuning from scratch (n = 27) were the two most favored approaches. Only a few studies applied feature extractor hybrid (n = 7) and fine-tuning (n = 3) with pretrained models. CONCLUSION The investigated studies demonstrated the efficacy of transfer learning despite the data scarcity. We encourage data scientists and practitioners to use deep models (e.g. ResNet or Inception) as feature extractors, which can save computational costs and time without degrading the predictive power.
Collapse
Affiliation(s)
- Hee E Kim
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany.
| | - Alejandro Cosa-Linan
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Nandhini Santhanam
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Mahboubeh Jannesari
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Mate E Maros
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Thomas Ganslandt
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
- Chair of Medical Informatics, Friedrich-Alexander-Universität Erlangen-Nürnberg, Wetterkreuz 15, 91058, Erlangen, Germany
| |
Collapse
|
32
|
Abdalla HB, Ahmed AM, Zeebaree SR, Alkhayyat A, Ihnaini B. Rider weed deep residual network-based incremental model for text classification using multidimensional features and MapReduce. PeerJ Comput Sci 2022; 8:e937. [PMID: 35494853 PMCID: PMC9044237 DOI: 10.7717/peerj-cs.937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Accepted: 03/08/2022] [Indexed: 06/14/2023]
Abstract
Increasing demands for information and the rapid growth of big data have dramatically increased the amount of textual data. In order to obtain useful text information, the classification of texts is considered an imperative task. Accordingly, this article will describe the development of a hybrid optimization algorithm for classifying text. Here, pre-processing was done using the stemming process and stop word removal. Additionally, we performed the extraction of imperative features and the selection of optimal features using the Tanimoto similarity, which estimates the similarity between features and selects the relevant features with higher feature selection accuracy. Following that, a deep residual network trained by the Adam algorithm was utilized for dynamic text classification. Dynamic learning was performed using the proposed Rider invasive weed optimization (RIWO)-based deep residual network along with fuzzy theory. The proposed RIWO algorithm combines invasive weed optimization (IWO) and the Rider optimization algorithm (ROA). These processes are carried out under the MapReduce framework. Our analysis revealed that the proposed RIWO-based deep residual network outperformed other techniques with the highest true positive rate (TPR) of 85%, true negative rate (TNR) of 94%, and accuracy of 88.7%.
Collapse
Affiliation(s)
- Hemn Barzan Abdalla
- Department of Computer Science, Wenzhou-Kean University, Wenzhou, Zhejiang, China
| | - Awder M. Ahmed
- Department of Communication Engineering, Technical College of Engineering, Sulaimani Polytechnic University, Sulaymaniyah, Iraq
| | - Subhi R.M. Zeebaree
- Energy Department, Technical Collage Engineering, Duhok Polytechnic University, Duhok, Iraq
| | - Ahmed Alkhayyat
- Department of Computer Technical Engineering, College of Technical Engineering, Islamic University, Najaf, Iraq
| | - Baha Ihnaini
- Department of Computer Science, Wenzhou-Kean University, Wenzhou, Zhejiang, China
| |
Collapse
|
33
|
Tsuneki M. Deep learning models in medical image analysis. J Oral Biosci 2022; 64:312-320. [PMID: 35306172 DOI: 10.1016/j.job.2022.03.003] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 03/08/2022] [Accepted: 03/09/2022] [Indexed: 02/07/2023]
Abstract
BACKGROUND Deep learning is a state-of-the-art technology that has rapidly become the method of choice for medical image analysis. Its fast and robust object detection, segmentation, tracking, and classification of pathophysiological anatomical structures can support medical practitioners during routine clinical workflow. Thus, deep learning-based applications for diseases diagnosis will empower physicians and allow fast decision-making in clinical practice. HIGHLIGHT Deep learning can be more robust with various features for differentiating classes, provided the training set is large and diverse for analysis. However, sufficient medical images for training sets are not always available from medical institutions, which is one of the major limitations of deep learning in medical image analysis. This review article presents some solutions for this issue and discusses efforts needed to develop robust deep learning-based computer-aided diagnosis applications for better clinical workflow in endoscopy, radiology, pathology, and dentistry. CONCLUSION The introduction of deep learning-based applications will enhance the traditional role of medical practitioners in ensuring accurate diagnoses and treatment in terms of precision, reproducibility, and scalability.
Collapse
Affiliation(s)
- Masayuki Tsuneki
- Medmain Research, Medmain Inc., Fukuoka, Japan; Division of Anatomy and Cell Biology of the Hard Tissue, Department of Tissue Regeneration and Reconstruction, Niigata University Graduate School of Medical and Dental Sciences, Niigata, Japan.
| |
Collapse
|
34
|
The Application of Wavelet Transform of Raman Spectra to Facilitate Transfer Learning for Gasoline Detection and Classification. TALANTA OPEN 2022. [DOI: 10.1016/j.talo.2022.100106] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023] Open
|
35
|
Lin CY, Chou CC, Chen LR, Wu WT, Hsu PC, Yang TH, Chang KV. Quantitative Analysis of Dynamic Subacromial Ultrasonography: Reliability and Influencing Factors. Front Bioeng Biotechnol 2022; 10:830508. [PMID: 35242751 PMCID: PMC8886165 DOI: 10.3389/fbioe.2022.830508] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 01/26/2022] [Indexed: 11/25/2022] Open
Abstract
Objective: Current imaging methods used to examine patients with subacromial impingement syndrome (SIS) are limited by their semi-quantitative nature and their capability of capturing dynamic movements. This study aimed to develop a quantitative analytic model to assess subacromial motions using dynamic ultrasound and to examine their reliability and potential influencing factors. Method: We included 48 healthy volunteers and examined their subacromial motions with dynamic ultrasound imaging. The parameters were the minimal vertical acromiohumeral distance, rotation radius, and degrees of the humeral head. The generalized estimating equation (GEE) was used to investigate the impact of different shoulder laterality, postures, and motion phases on the outcome. Result: Using the data of the minimal vertical acromiohumeral distance, the intra-rater and inter-rater reliabilities (intra-class correlation coefficient) were determined as 0.94 and 0.88, respectively. In the GEE analysis, a decrease in the minimal vertical acromiohumeral distance was associated with the abduction phase and full-can posture, with a beta coefficient of −0.02 cm [95% confidence interval (CI), −0.03 to −0.01] and −0.07 cm (95% CI, −0.11 to −0.02), respectively. The abduction phase led to a decrease in the radius of humeral rotation and an increase in the angle of humeral rotation, with a beta coefficient of −1.28 cm (95% CI, −2.16 to −0.40) and 6.60° (95% CI, 3.54–9.67), respectively. A significant negative correlation was observed between the rotation angle and radius of the humeral head and between the rotation angle and the minimal vertical acromiohumeral distance. Conclusion: Quantitative analysis of dynamic ultrasound imaging enables the delineation of subacromial motion with good reliability. The vertical acromiohumeral distance is the lowest in the abduction phase and full-can posture, and the rotation angle of the humeral head has the potential to serve as a new parameter for the evaluation of SIS.
Collapse
Affiliation(s)
- Che-Yu Lin
- Institute of Applied Mechanics, College of Engineering, National Taiwan University, Taipei, Taiwan
| | - Chia-Ching Chou
- Institute of Applied Mechanics, College of Engineering, National Taiwan University, Taipei, Taiwan
| | - Lan-Rong Chen
- Department of Physical Medicine and Rehabilitation and Community and Geriatric Research Center, National Taiwan University Hospital, Bei-Hu Branch, Taipei, Taiwan
| | - Wei-Ting Wu
- Department of Physical Medicine and Rehabilitation and Community and Geriatric Research Center, National Taiwan University Hospital, Bei-Hu Branch, Taipei, Taiwan
- Department of Physical Medicine and Rehabilitation, National Taiwan University College of Medicine, Taipei, Taiwan
| | - Po-Cheng Hsu
- Department of Physical Medicine and Rehabilitation, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Tung-Han Yang
- Institute of Applied Mechanics, College of Engineering, National Taiwan University, Taipei, Taiwan
| | - Ke-Vin Chang
- Department of Physical Medicine and Rehabilitation and Community and Geriatric Research Center, National Taiwan University Hospital, Bei-Hu Branch, Taipei, Taiwan
- Department of Physical Medicine and Rehabilitation, National Taiwan University College of Medicine, Taipei, Taiwan
- Center for Regional Anesthesia and Pain Medicine, Wang-Fang Hospital, Taipei Medical University, Taipei, Taiwan
- *Correspondence: Ke-Vin Chang,
| |
Collapse
|
36
|
Luo W, Feng P, Zhang J, Yu D, Wu Z. A novel deep convolution generative adversarial transfer learning model for data-driven assembly quality diagnosis. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2022. [DOI: 10.3233/jifs-211860] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
As the service life of the assembly equipment are short, the tightening data it produces are very limited. Therefore, data-driven assembly quality diagnosis is still a challenge task in industries. Transfer learning can be used to address small data problems. However, transfer learning has strict requirements on the training dataset, which is hard to satisfy. To solve the above problem, an Improved Deep Convolution Generative Adversarial Transfer Learning Model (IDCGAN-TM) is proposed, which integrates three modules: The generative learning module automatically produces source datasets based on small target datasets by using the improved generative-adversarial theory. The feature learning module improves the feature extraction ability by building a lightweight deep learning model (DL). The transfer learning module consists of a pre-trained DL and a one fully connected layer to better perform the intelligent quality diagnosis on the training small sample data. A parallel computing method is adopted to obtain produced source data efficiently. Real assembly quality diagnosis cases are designed and discussed to validate the advance of the proposed model. In addition, the comparison experiments are designed to show that the proposed approach holds the better transfer diagnosis performance compared with the existing three state-of-art approaches.
Collapse
Affiliation(s)
- Wentao Luo
- Beijing Key Lab of Precision/Ultra-Precision Manufacturing Equipments and Control, Department of Mechanical Engineering, Tsinghua University, Beijing, China
- Division of Advanced Manufacturing, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
| | - Pingfa Feng
- Beijing Key Lab of Precision/Ultra-Precision Manufacturing Equipments and Control, Department of Mechanical Engineering, Tsinghua University, Beijing, China
- State Key Laboratory of Tribology, Department of Mechanical Engineering, Tsinghua University, Beijing, China
- Division of Advanced Manufacturing, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
| | - Jianfu Zhang
- Beijing Key Lab of Precision/Ultra-Precision Manufacturing Equipments and Control, Department of Mechanical Engineering, Tsinghua University, Beijing, China
- State Key Laboratory of Tribology, Department of Mechanical Engineering, Tsinghua University, Beijing, China
| | - Dingwen Yu
- Beijing Key Lab of Precision/Ultra-Precision Manufacturing Equipments and Control, Department of Mechanical Engineering, Tsinghua University, Beijing, China
| | - Zhijun Wu
- Beijing Key Lab of Precision/Ultra-Precision Manufacturing Equipments and Control, Department of Mechanical Engineering, Tsinghua University, Beijing, China
| |
Collapse
|
37
|
Zhou Q, Zhu W, Li F, Yuan M, Zheng L, Liu X. Transfer Learning of the ResNet-18 and DenseNet-121 Model Used to Diagnose Intracranial Hemorrhage in CT Scanning. Curr Pharm Des 2022; 28:287-295. [PMID: 34961458 DOI: 10.2174/1381612827666211213143357] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2021] [Revised: 09/24/2021] [Accepted: 10/08/2021] [Indexed: 02/05/2023]
Abstract
OBJECTIVE The aim of the study was to verify the ability of the deep learning model to identify five subtypes and normal images in non-contrast enhancement CT of intracranial hemorrhage. METHODS A total of 351 patients (39 patients in the normal group, 312 patients in the intracranial hemorrhage group) who underwent intracranial hemorrhage noncontrast enhanced CT were selected, obtaining 2768 images in total (514 images for the normal group, 398 images for the epidural hemorrhage group, 501 images for the subdural hemorrhage group, 497 images for the intraventricular hemorrhage group, 415 images for the cerebral parenchymal hemorrhage group, and 443 images for the subarachnoid hemorrhage group). Based on the diagnostic reports of two radiologists with more than 10 years of experience, the ResNet-18 and DenseNet-121 deep learning models were selected. Transfer learning was used. 80% of the data was used for training models, 10% was used for validating model performance against overfitting, and the last 10% was used for the final evaluation of the model. Assessment indicators included accuracy, sensitivity, specificity, and AUC values. RESULTS The overall accuracy of ResNet-18 and DenseNet-121 models was obtained as 89.64% and 82.5%, respectively. The sensitivity and specificity of identifying five subtypes and normal images were above 0.80. The sensitivity of the DenseNet-121 model to recognize intraventricular hemorrhage and cerebral parenchymal hemorrhage was lower than 0.80, 0.73, and 0.76, respectively. The AUC values of the two deep learning models were found to be above 0.9. CONCLUSION The deep learning model can accurately identify the five subtypes of intracranial hemorrhage and normal images, and it can be used as a new tool for clinical diagnosis in the future.
Collapse
Affiliation(s)
- Qi Zhou
- Medical College of Guangxi University, Nanning, Guangxi,China
| | - Wenjie Zhu
- Department of Emergency, The First Affiliated Hospital of Shantou University Medical College, Shantou 515041,China
| | - Fuchen Li
- College of Art and Science, Vanderbilt University, Nashville, Tennessee 37212, USA
| | - Mingqing Yuan
- Medical College of Guangxi University, Nanning, Guangxi,China
| | - Linfeng Zheng
- Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai, 200042,China
| | - Xu Liu
- Medical College of Guangxi University, Nanning, Guangxi,China
| |
Collapse
|
38
|
Yi PH, Arun A, Hafezi-Nejad N, Choy G, Sair HI, Hui FK, Fritz J. Can AI distinguish a bone radiograph from photos of flowers or cars? Evaluation of bone age deep learning model on inappropriate data inputs. Skeletal Radiol 2022; 51:401-406. [PMID: 34351456 PMCID: PMC8339162 DOI: 10.1007/s00256-021-03880-y] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Revised: 07/15/2021] [Accepted: 07/25/2021] [Indexed: 02/02/2023]
Abstract
OBJECTIVE To evaluate the behavior of a publicly available deep convolutional neural network (DCNN) bone age algorithm when presented with inappropriate data inputs in both radiological and non-radiological domains. METHODS We evaluated a publicly available DCNN-based bone age application. The DCNN was trained on 12,612 pediatric hand radiographs and won the 2017 RSNA Pediatric Bone Age Challenge (concordance of 0.991 with radiologist ground-truth). We used the application to analyze 50 left-hand radiographs (appropriate data inputs) and seven classes of inappropriate data inputs in radiological (i.e., chest radiographs) and non-radiological (i.e., image of street numbers) domains. For each image, we noted if (1) the application distinguished between appropriate and inappropriate data inputs and (2) inference time per image. Mean inference times were compared using ANOVA. RESULTS The 16Bit Bone Age application calculated bone age for all pediatric hand radiographs with mean inference time of 1.1 s. The application did not distinguish between pediatric hand radiographs and inappropriate image types, including radiological and non-radiological domains. The application inappropriately calculated bone age for all inappropriate image types, with mean inference time of 1.1 s for all categories (p = 1). CONCLUSION A publicly available DCNN-based bone age application failed to distinguish between appropriate and inappropriate data inputs and calculated bone age for inappropriate images. The awareness of inappropriate outputs based on inappropriate DCNN input is important if tasks such as bone age determination are automated, emphasizing the need for appropriate oversight at the data input and verification stage to avoid unrecognized erroneous results.
Collapse
Affiliation(s)
- Paul H. Yi
- University of Maryland Intelligent Imaging (UMII) Center, Department of Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, MD USA
| | - Anirudh Arun
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD USA
| | - Nima Hafezi-Nejad
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD USA
| | - Garry Choy
- Department of Radiology, Veterans Affairs Palo Alto Health Care System, Palo Alto, CA USA
| | - Haris I. Sair
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD USA
| | - Ferdinand K. Hui
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD USA
| | - Jan Fritz
- Department of Radiology, New York University Grossman School of Medicine, New York, NY USA
| |
Collapse
|
39
|
Li J, Wang P, Zhou Y, Liang H, Lu Y, Luan K. A novel classification method of lymph node metastasis in colorectal cancer. Bioengineered 2021; 12:2007-2021. [PMID: 34024255 PMCID: PMC8806456 DOI: 10.1080/21655979.2021.1930333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Revised: 05/07/2021] [Accepted: 05/08/2021] [Indexed: 11/21/2022] Open
Abstract
Colorectal cancer lymph node metastasis, which is highly associated with the patient's cancer recurrence and survival rate, has been the focus of many therapeutic strategies that are highly associated with the patient's cancer recurrence and survival rate. The popular methods for classification of lymph node metastasis by neural networks, however, show limitations as the available low-level features are inadequate for classification, and the radiologists are unable to quickly review the images. Identifying lymph node metastasis in colorectal cancer is a key factor in the treatment of patients with colorectal cancer. In the present work, an automatic classification method based on deep transfer learning was proposed. Specifically, the method resolved the problem of repetition of low-level features and combined these features with high-level features into a new feature map for classification; and a merged layer which merges all transmitted features from previous layers into a map of the first full connection layer. With a dataset collected from Harbin Medical University Cancer Hospital, the experiment involved a sample of 3,364 patients. Among these samples, 1,646 were positive, and 1,718 were negative. The experiment results showed the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were 0.8732, 0.8746, 0.8746 and 0.8728, respectively, and the accuracy and AUC were 0.8358 and 0.8569, respectively. These demonstrated that our method significantly outperformed the previous classification methods for colorectal cancer lymph node metastasis without increasing the depth and width of the model.
Collapse
Affiliation(s)
- Jin Li
- College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin, Heilongjiang Province, China
| | - Peng Wang
- College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin, Heilongjiang Province, China
| | - Yang Zhou
- College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin, Heilongjiang Province, China
- Department of Radiology, Harbin Medical University Cancer Hospital, Harbin, Heilongjiang Province, China
| | - Hong Liang
- College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin, Heilongjiang Province, China
| | - Yang Lu
- College of Information and Electrical Engineering, Heilongjiang Bayi Agricultural University, Daqing, Heilongjiang Province, China
| | - Kuan Luan
- College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin, Heilongjiang Province, China
| |
Collapse
|
40
|
A fundamental study assessing the generalized fitting method in conjunction with every possible coalition of N-combinations (G-EPOC) using the appendicitis detection task of computed tomography. Pol J Radiol 2021; 86:e532-e541. [PMID: 34820029 PMCID: PMC8607834 DOI: 10.5114/pjr.2021.110309] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2020] [Accepted: 07/31/2020] [Indexed: 11/28/2022] Open
Abstract
Purpose Increased use of deep learning (DL) in medical imaging diagnoses has led to more frequent use of 10-fold cross-validation (10-CV) for the evaluation of the performance of DL. To eliminate some of the (10-fold) repetitive processing in 10-CV, we proposed a “generalized fitting method in conjunction with every possible coalition of N-combinations (G-EPOC)”, to estimate the range of the mean accuracy of 10-CV using less than 10 results of 10-CV. Material and methods G-EPOC was executed as follows. We first provided (2N-1) coalition subsets using a specified N, which was 9 or less, out of 10 result datasets of 10-CV. We then obtained the estimation range of the accuracy by applying those subsets to the distribution fitting twice using a combination of normal, binominal, or Poisson distributions. Using datasets of 10-CVs acquired from the practical detection task of the appendicitis on CT by DL, we scored the estimation success rates if the range provided by G-EPOC included the true accuracy. Results G-EPOC successfully estimated the range of the mean accuracy by 10-CV at over 95% rates for datasets with N assigned as 2 to 9. Conclusions G-EPOC will help lessen the consumption of time and computer resources in the development of computerbased diagnoses in medical imaging and could become an option for the selection of a reasonable K value in K-CV.
Collapse
|
41
|
Cheng CY, Chiu IM, Hsu MY, Pan HY, Tsai CM, Lin CHR. Deep Learning Assisted Detection of Abdominal Free Fluid in Morison's Pouch During Focused Assessment With Sonography in Trauma. Front Med (Lausanne) 2021; 8:707437. [PMID: 34631730 PMCID: PMC8494971 DOI: 10.3389/fmed.2021.707437] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2021] [Accepted: 08/25/2021] [Indexed: 12/03/2022] Open
Abstract
Background: The use of focused assessment with sonography in trauma (FAST) enables clinicians to rapidly screen for injury at the bedsides of patients. Pre-hospital FAST improves diagnostic accuracy and streamlines patient care, leading to dispositions to appropriate treatment centers. In this study, we determine the accuracy of artificial intelligence model-assisted free-fluid detection in FAST examinations, and subsequently establish an automated feedback system, which can help inexperienced sonographers improve their interpretation ability and image acquisition skills. Methods: This is a single-center study of patients admitted to the emergency room from January 2020 to March 2021. We collected 324 patient records for the training model, 36 patient records for validation, and another 36 patient records for testing. We balanced positive and negative Morison's pouch free-fluid detection groups in a 1:1 ratio. The deep learning (DL) model Residual Networks 50-Version 2 (ResNet50-V2) was used for training and validation. Results: The accuracy, sensitivity, and specificity of the model performance for ascites prediction were 0.961, 0.976, and 0.947, respectively, in the validation set and 0.967, 0.985, and 0.913, respectively, in the test set. Regarding feedback prediction, the model correctly classified qualified and non-qualified images with an accuracy of 0.941 in both the validation and test sets. Conclusions: The DL algorithm in ResNet50-V2 is able to detect free fluid in Morison's pouch with high accuracy. The automated feedback and instruction system could help inexperienced sonographers improve their interpretation ability and image acquisition skills.
Collapse
Affiliation(s)
- Chi-Yung Cheng
- Department of Emergency Medicine, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan.,Department of Computer Science and Engineering, National Sun Yat-sen University, Kaohsiung, Taiwan
| | - I-Min Chiu
- Department of Emergency Medicine, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan.,Department of Computer Science and Engineering, National Sun Yat-sen University, Kaohsiung, Taiwan
| | - Ming-Ya Hsu
- Department of Computer Science and Engineering, National Sun Yat-sen University, Kaohsiung, Taiwan
| | - Hsiu-Yung Pan
- Department of Emergency Medicine, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan
| | - Chih-Min Tsai
- Department of Pediatrics, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan
| | - Chun-Hung Richard Lin
- Department of Computer Science and Engineering, National Sun Yat-sen University, Kaohsiung, Taiwan
| |
Collapse
|
42
|
Chen X, Zhong X. A Convolutional Neural Network Algorithm for the Optimization of Emergency Nursing Rescue Efficiency for Critical Patients. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:1034972. [PMID: 34659675 PMCID: PMC8514904 DOI: 10.1155/2021/1034972] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Accepted: 09/02/2021] [Indexed: 11/30/2022]
Abstract
In order to help pathologists quickly locate the lesion area, improve the diagnostic efficiency, and reduce missed diagnosis, a convolutional neural network algorithm for the optimization of emergency nursing rescue efficiency of critical patients was proposed. Specifically, three convolution layers and convolution kernels of different sizes are used to extract the features of patients' posture behavior, and the classifier of patients' posture behavior recognition system is used to learn the feature information by capturing the nonlinear relationship between the features to achieve accurate classification. By testing the accuracy of patient posture behavior feature extraction, the recognition rate of a certain action, and the average recognition rate of all actions in the patient body behavior recognition system, it is proved that the convolution neural network algorithm can greatly improve the efficiency of emergency nursing. The algorithm is applied to the patient posture behavior detection system, so as to realize the identification and monitoring of patients and improve the level of intelligent medical care. Finally, the open source framework platform is used to test the patient behavior detection system. The experimental results show that the larger the test data set is, the higher the accuracy of patient posture behavior feature extraction is, and the average recognition rate of patient posture behavior category is 97.6%, thus verifying the effectiveness and correctness of the system, to prove that the convolutional neural network algorithm has a very large improvement of emergency nursing rescue efficiency.
Collapse
Affiliation(s)
- Xueyan Chen
- Emergency Department, Zhejiang Hospital, Hangzhou, Zhejiang 310030, China
| | - Xiaofei Zhong
- Neurosurgery Department, Zhejiang Hospital, Hangzhou, Zhejiang 310030, China
| |
Collapse
|
43
|
|
44
|
Multilevel Strip Pooling-Based Convolutional Neural Network for the Classification of Carotid Plaque Echogenicity. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2021; 2021:3425893. [PMID: 34457035 PMCID: PMC8390163 DOI: 10.1155/2021/3425893] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Revised: 07/12/2021] [Accepted: 08/02/2021] [Indexed: 12/29/2022]
Abstract
Carotid plaque echogenicity in ultrasound images has been found to be closely correlated with the risk of stroke in atherosclerotic patients. The automatic and accurate classification of carotid plaque echogenicity is of great significance for clinically estimating the stability of carotid plaques and predicting cardiovascular events. Existing convolutional neural networks (CNNs) can provide an automatic carotid plaque echogenicity classification; however, they require a fixed-size input image, while the carotid plaques are of varying sizes. Although cropping and scaling the input carotid plaque images is promising, it will cause content loss or distortion and hence reduce the classification accuracy. In this study, we redesign the spatial pyramid pooling (SPP) and propose multilevel strip pooling (MSP) for the automatic and accurate classification of carotid plaque echogenicity in the longitudinal section. The proposed MSP module can accept arbitrarily sized carotid plaques as input and capture a long-range informative context to improve the accuracy of classification. In our experiments, we implement an MSP-based CNN by using the visual geometry group (VGG) network as the backbone. A total of 1463 carotid plaques (335 echo-rich plaques, 405 intermediate plaques, and 723 echolucent plaques) were collected from Zhongnan Hospital of Wuhan University. The 5-fold cross-validation results show that the proposed MSP-based VGGNet achieves a sensitivity of 92.1%, specificity of 95.6%, accuracy of 92.1%, and F1-score of 92.1%. These results demonstrate that our approach provides a way to enhance the applicability of CNN by enabling the acceptance of arbitrary input sizes and improving the classification accuracy of carotid plaque echogenicity, which has a great potential for an efficient and objective risk assessment of carotid plaques in the clinic.
Collapse
|
45
|
Hybrid Transfer Learning for Classification of Uterine Cervix Images for Cervical Cancer Screening. J Digit Imaging 2021; 33:619-631. [PMID: 31848896 DOI: 10.1007/s10278-019-00269-1] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022] Open
Abstract
Transfer learning using deep pre-trained convolutional neural networks is increasingly used to solve a large number of problems in the medical field. In spite of being trained using images with entirely different domain, these networks are flexible to adapt to solve a problem in a different domain too. Transfer learning involves fine-tuning a pre-trained network with optimal values of hyperparameters such as learning rate, batch size, and number of training epochs. The process of training the network identifies the relevant features for solving a specific problem. Adapting the pre-trained network to solve a different problem requires fine-tuning until relevant features are obtained. This is facilitated through the use of large number of filters present in the convolutional layers of pre-trained network. A very few features out of these features are useful for solving the problem in a different domain, while others are irrelevant, use of which may only reduce the efficacy of the network. However, by minimizing the number of filters required to solve the problem, the efficiency of the training the network can be improved. In this study, we consider identification of relevant filters using the pre-trained networks namely AlexNet and VGG-16 net to detect cervical cancer from cervix images. This paper presents a novel hybrid transfer learning technique, in which a CNN is built and trained from scratch, with initial weights of only those filters which were identified as relevant using AlexNet and VGG-16 net. This study used 2198 cervix images with 1090 belonging to negative class and 1108 to positive class. Our experiment using hybrid transfer learning achieved an accuracy of 91.46%.
Collapse
|
46
|
Alawad M, Yoon HJ, Gao S, Mumphrey B, Wu XC, Durbin EB, Jeong JC, Hands I, Rust D, Coyle L, Penberthy L, Tourassi G. Privacy-Preserving Deep Learning NLP Models for Cancer Registries. IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTING 2021; 9:1219-1230. [PMID: 36117774 PMCID: PMC9481201 DOI: 10.1109/tetc.2020.2983404] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Population cancer registries can benefit from Deep Learning (DL) to automatically extract cancer characteristics from the high volume of unstructured pathology text reports they process annually. The success of DL to tackle this and other real-world problems is proportional to the availability of large labeled datasets for model training. Although collaboration among cancer registries is essential to fully exploit the promise of DL, privacy and confidentiality concerns are main obstacles for data sharing across cancer registries. Moreover, DL for natural language processing (NLP) requires sharing a vocabulary dictionary for the embedding layer which may contain patient identifiers. Thus, even distributing the trained models across cancer registries causes a privacy violation issue. In this paper, we propose DL NLP model distribution via privacy-preserving transfer learning approaches without sharing sensitive data. These approaches are used to distribute a multitask convolutional neural network (MT-CNN) NLP model among cancer registries. The model is trained to extract six key cancer characteristics - tumor site, subsite, laterality, behavior, histology, and grade - from cancer pathology reports. Using 410,064 pathology documents from two cancer registries, we compare our proposed approach to conventional transfer learning without privacy-preserving, single-registry models, and a model trained on centrally hosted data. The results show that transfer learning approaches including data sharing and model distribution outperform significantly the single-registry model. In addition, the best performing privacy-preserving model distribution approach achieves statistically indistinguishable average micro- and macro-F1 scores across all extraction tasks (0.823,0.580) as compared to the centralized model (0.827,0.585).
Collapse
Affiliation(s)
- Mohammed Alawad
- Computational Sciences and Engineering Division, Health Data Sciences Institute, Oak Ridge National Laboratory, Oak Ridge, TN, 37831
| | - Hong-Jun Yoon
- Computational Sciences and Engineering Division, Health Data Sciences Institute, Oak Ridge National Laboratory, Oak Ridge, TN, 37831
| | - Shang Gao
- Computational Sciences and Engineering Division, Health Data Sciences Institute, Oak Ridge National Laboratory, Oak Ridge, TN, 37831
| | - Brent Mumphrey
- Louisiana Tumor Registry, Louisiana State University Health Sciences Center School of Public Health, New Orleans, LA, 70112
| | - Xiao-Cheng Wu
- Louisiana Tumor Registry, Louisiana State University Health Sciences Center School of Public Health, New Orleans, LA, 70112
| | - Eric B Durbin
- Kentucky Cancer Registry, University of Kentucky, Lexington, KY, 40506.; Division of Biomedical Informatics, College of Medicine, University of Kentucky, Lexington, KY, 40506.; Cancer Research Informatics Shared Resource Facility, Markey Cancer Center, University of Kentucky, Lexington, KY, 40506
| | - Jong Cheol Jeong
- Division of Biomedical Informatics, College of Medicine, University of Kentucky, Lexington, KY, 40506.; Cancer Research Informatics Shared Resource Facility, Markey Cancer Center, University of Kentucky, Lexington, KY, 40506
| | - Isaac Hands
- Kentucky Cancer Registry, University of Kentucky, Lexington, KY, 40506.; Cancer Research Informatics Shared Resource Facility, Markey Cancer Center, University of Kentucky, Lexington, KY, 40506
| | - David Rust
- Kentucky Cancer Registry, University of Kentucky, Lexington, KY, 40506
| | - Linda Coyle
- Information Management Services Inc, Calverton, MD, 20705
| | - Lynne Penberthy
- Surveillance Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD, USA
| | - Georgia Tourassi
- Computational Sciences and Engineering Division, Health Data Sciences Institute, Oak Ridge National Laboratory, Oak Ridge, TN, 37831
| |
Collapse
|
47
|
Tewary S, Mukhopadhyay S. HER2 Molecular Marker Scoring Using Transfer Learning and Decision Level Fusion. J Digit Imaging 2021; 34:667-677. [PMID: 33742331 PMCID: PMC8329150 DOI: 10.1007/s10278-021-00442-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2020] [Revised: 01/13/2021] [Accepted: 03/01/2021] [Indexed: 01/28/2023] Open
Abstract
In prognostic evaluation of breast cancer, immunohistochemical (IHC) marker human epidermal growth factor receptor 2 (HER2) is used for prognostic evaluation. Accurate assessment of HER2-stained tissue sample is essential in therapeutic decision making for the patients. In regular clinical settings, expert pathologists assess the HER2-stained tissue slide under microscope for manual scoring based on prior experience. Manual scoring is time consuming, tedious, and often prone to inter-observer variation among group of pathologists. With the recent advancement in the area of computer vision and deep learning, medical image analysis has got significant attention. A number of deep learning architectures have been proposed for classification of different image groups. These networks are also used for transfer learning to classify other image classes. In the presented study, a number of transfer learning architectures are used for HER2 scoring. Five pre-trained architectures viz. VGG16, VGG19, ResNet50, MobileNetV2, and NASNetMobile with decimating the fully connected layers to get 3-class classification have been used for the comparative assessment of the networks as well as further scoring of stained tissue sample image based on statistical voting using mode operator. HER2 Challenge dataset from Warwick University is used in this study. A total of 2130 image patches were extracted to generate the training dataset from 300 training images corresponding to 30 training cases. The output model is then tested on 800 new test image patches from 100 test images acquired from 10 test cases (different from training cases) to report the outcome results. The transfer learning models have shown significant accuracy with VGG19 showing the best accuracy for the test images. The accuracy is found to be 93%, which increases to 98% on the image-based scoring using statistical voting mechanism. The output shows a capable quantification pipeline in automated HER2 score generation.
Collapse
Affiliation(s)
- Suman Tewary
- School of Medical Science and Technology, Indian Institute of Technology Kharagpur, Kharagpur, India
- Computational Instrumentation, CSIR-Central Scientific Instruments Organisation, Chandigarh, India
| | - Sudipta Mukhopadhyay
- Department of Electronics and Electrical Communication Engineering, Indian Institute of Technology Kharagpur, Kharagpur, India.
| |
Collapse
|
48
|
Rapid Assessment of Acute Ischemic Stroke by Computed Tomography Using Deep Convolutional Neural Networks. J Digit Imaging 2021; 34:637-646. [PMID: 33963421 DOI: 10.1007/s10278-021-00457-y] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2020] [Revised: 03/10/2021] [Accepted: 04/27/2021] [Indexed: 01/01/2023] Open
Abstract
Acute stroke is one of the leading causes of disability and death worldwide. Regarding clinical diagnoses, a rapid and accurate procedure is necessary for patients suffering from acute stroke. This study proposes an automatic identification scheme for acute ischemic stroke using deep convolutional neural networks (DCNNs) based on non-contrast computed tomographic (NCCT) images. Our image database for the classification model was composed of 1254 grayscale NCCT images from 96 patients (573 images) with acute ischemic stroke and 121 normal controls (681 images). According to the consensus of critical stroke findings by two neuroradiologists, a gold standard was established and used to train the proposed DCNN using machine-generated image features. Including the earliest DCNN, AlexNet, the popular Inception-v3, and ResNet-101 were proposed. To train the limited data size, transfer learning with ImageNet parameters was also used. The established models were evaluated by tenfold cross-validation and tested on an independent dataset containing 50 patients with acute ischemic stroke (108 images) and 58 normal controls (117 images) from another institution. AlexNet without pretrained parameters achieved an accuracy of 97.12%, a sensitivity of 98.11%, a specificity of 96.08%, and an area under the receiver operating characteristic curve (AUC) of 0.9927. Using transfer learning, transferred AlexNet, transferred Inception-v3, and transferred ResNet-101 achieved accuracies between 90.49 and 95.49%. Tested with a dataset from another institution, AlexNet showed an accuracy of 60.89%, a sensitivity of 18.52%, and a specificity of 100%. Transferred AlexNet, Inception-v3, and ResNet-101 achieved accuracies of 81.77%, 85.78%, and 80.89%, respectively. The proposed DCNN architecture as a computer-aided diagnosis system showed that training from scratch can generate a customized model for a specific scanner, and transfer learning can generate a more generalized model to provide diagnostic suggestions of acute ischemic stroke to radiologists.
Collapse
|
49
|
Chang KP, Lin SH, Chu YW. Artificial intelligence in gastrointestinal radiology: A review with special focus on recent development of magnetic resonance and computed tomography. Artif Intell Gastroenterol 2021; 2:27-41. [DOI: 10.35712/aig.v2.i2.27] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/27/2021] [Revised: 03/21/2021] [Accepted: 04/20/2021] [Indexed: 02/06/2023] Open
Abstract
Artificial intelligence (AI), particularly the deep learning technology, have been proven influential to radiology in the recent decade. Its ability in image classification, segmentation, detection and reconstruction tasks have substantially assisted diagnostic radiology, and has even been viewed as having the potential to perform better than radiologists in some tasks. Gastrointestinal radiology, an important subspecialty dealing with complex anatomy and various modalities including endoscopy, have especially attracted the attention of AI researchers and engineers worldwide. Consequently, recently many tools have been developed for lesion detection and image construction in gastrointestinal radiology, particularly in the fields for which public databases are available, such as diagnostic abdominal magnetic resonance imaging (MRI) and computed tomography (CT). This review will provide a framework for understanding recent advancements of AI in gastrointestinal radiology, with a special focus on hepatic and pancreatobiliary diagnostic radiology with MRI and CT. For fields where AI is less developed, this review will also explain the difficulty in AI model training and possible strategies to overcome the technical issues. The authors’ insights of possible future development will be addressed in the last section.
Collapse
Affiliation(s)
- Kai-Po Chang
- PhD Program in Medical Biotechnology, National Chung Hsing University, Taichung 40227, Taiwan
- Department of Pathology, China Medical University Hospital, Taichung 40447, Taiwan
| | - Shih-Huan Lin
- PhD Program in Medical Biotechnology, National Chung Hsing University, Taichung 40227, Taiwan
| | - Yen-Wei Chu
- PhD Program in Medical Biotechnology, National Chung Hsing University, Taichung 40227, Taiwan
- Institute of Genomics and Bioinformatics, National Chung Hsing University, Taichung 40227, Taiwan
- Institute of Molecular Biology, National Chung Hsing University, Taichung 40227, Taiwan
- Agricultural Biotechnology Center, National Chung Hsing University, Taichung 40227, Taiwan
- Biotechnology Center, National Chung Hsing University, Taichung 40227, Taiwan
- PhD Program in Translational Medicine, National Chung Hsing University, Taichung 40227, Taiwan
- Rong Hsing Research Center for Translational Medicine, Taichung 40227, Taiwan
| |
Collapse
|
50
|
Reverse Scan Conversion and Efficient Deep Learning Network Architecture for Ultrasound Imaging on a Mobile Device. SENSORS 2021; 21:s21082629. [PMID: 33918047 PMCID: PMC8070375 DOI: 10.3390/s21082629] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/24/2021] [Revised: 04/01/2021] [Accepted: 04/06/2021] [Indexed: 11/20/2022]
Abstract
Point-of-care ultrasound (POCUS), realized by recent developments in portable ultrasound imaging systems for prompt diagnosis and treatment, has become a major tool in accidents or emergencies. Concomitantly, the number of untrained/unskilled staff not familiar with the operation of the ultrasound system for diagnosis is increasing. By providing an imaging guide to assist clinical decisions and support diagnosis, the risk brought by inexperienced users can be managed. Recently, deep learning has been employed to guide users in ultrasound scanning and diagnosis. However, in a cloud-based ultrasonic artificial intelligence system, the use of POCUS is limited due to information security, network integrity, and significant energy consumption. To address this, we propose (1) a structure that simultaneously provides ultrasound imaging and a mobile device-based ultrasound image guide using deep learning, and (2) a reverse scan conversion (RSC) method for building an ultrasound training dataset to increase the accuracy of the deep learning model. Experimental results show that the proposed structure can achieve ultrasound imaging and deep learning simultaneously at a maximum rate of 42.9 frames per second, and that the RSC method improves the image classification accuracy by more than 3%.
Collapse
|