Deep learning techniques for using computed tomography imaging for hepatocellular carcinoma diagnosis, treatment and prognosis

Abstract

Hepatocellular carcinoma (HCC), the predominant form of primary liver cancer, significantly threatens to global health. Despite considerable advances in diagnostic and therapeutic approaches in recent years, the prognosis for patients with HCC remains unsatisfactory. The emergence of artificial intelligence (AI), particularly deep learning technologies, offers new hope for improving the diagnosis and treatment of HCC. Researchers have extensively explored ways to integrate deep learning models into the clinical management of HCC patients, which provides a valuable foundation for developing more personalized treatment strategies. Compared with other detection methods, computed tomography (CT) has attracted significant research interest because of its comprehensive advantages, including wide availability and high resolution, making it well suited for AI-powered analysis. This review systematically integrates deep learning technologies for HCC based on CT imaging, while focusing primarily on tumor diagnosis, segmentation, treatment response prediction, and patient prognosis prediction. Moreover, we review popular deep learning networks in various fields and describe the advantages of these prevalent deep learning models for different applications. Furthermore, we discuss the outstanding challenges in applying deep learning to extract information from CT images for the diagnosis and treatment of HCC patients. These insights could provide guidance for subsequent studies.

Key Words: Hepatocellular carcinoma; Computed tomography; Deep learning; Diagnosis; Treatment; Prognosis

Core Tip: This review systematically integrates deep learning technologies for hepatocellular carcinoma (HCC) based on computed tomography (CT) imaging, with a primary focus on tumor diagnosis, segmentation, predicting treatment response, and forecasting patient prognosis. Moreover, we reviewed popular deep learning networks in various fields and described the advantages of these prevalent deep learning models for different applications. Furthermore, we discussed the outstanding challenges in applying deep learning to extract information from CT images for the diagnosis and treatment of HCC patients. These insights could provide guidance for subsequent studies.

INTRODUCTION

Liver cancer ranks as the sixth most prevalent malignancy and the third leading cause of cancer-related death globally, thus posing a significant threat to human health[1]. According to projections by the American Cancer Society, approximately 42240 new liver cancer cases and 30090 cancer-attributable deaths are anticipated in the United States during 2025[2]. Hepatocellular carcinoma (HCC) is the predominant histopathological subtype, and accounts for 90% of primary liver cancer cases[3]. Multiple etiological factors contribute to HCC, including chronic hepatitis B/C viral infection, heavy alcohol consumption, obesity, and nonalcoholic fatty liver disease[4] (Figure 1). HCC frequently presents with nonspecific clinical manifestations in early stages, thus delaying diagnosis for most patients until advanced phases with concomitant intrahepatic or extrahepatic metastases, and thus hindering treatment options[5-7]. Current therapeutic strategies for treating HCC include surgical resection[7], liver transplantation (for selected patients)[7], radiotherapy[5], immunotherapy[8], transarterial chemoembolization (TACE)[9], targeted therapy[10] and systemic chemotherapy[11] (Figure 2). Although numerous treatment methods are available for HCC, the prognosis remains unsatisfactory with up to 70% of patients experiencing recurrence within 5 years after treatment. Promoting early diagnosis of liver cancer, optimizing treatment strategies, and enhancing recurrence surveillance would improve the prognosis of patients with HCC.

Figure 1 Risk factors for hepatocellular carcinoma. AFC: Advanced fibrosis cirrhosis; GPD: Genetic predisposition; HBV: Hepatitis B virus; HCV: Hepatitis C virus; MASLD: Metabolic associated steatotic liver disease. The image materials are sourced from Biorender (https://biorender.com) and Figdraw (https://www.figdraw.com) (Supplementary material).

Figure 2 Diagnostic and therapeutic approaches for hepatocellular carcinoma. CT: Computed tomography; MRI: Magnetic resonance imaging; cf-DNAs: Cell free DNAs; CTCs: Circulating tumor cells; ctDNA: Circulating tumor DNAs; circRNAs: Circular RNAs; mRNA: Messenger RNA; lncRNAs: Long non-coding RNAs; miRNAs: MicroRNAs; EVs: Extracellular vesicles; IHC: Immunohistochemistry; HE: Hematoxylin and eosin staining; CTLA-4: Cytotoxic T lymphocyte-associated antigen-4; PD-1: Programmed death-1; PD-L1: Programmed death ligand 1; TCR: T cell antigen receptor; MHC I: Major histocompatibility complex class I; CD80: Cluster of differentiation 80. The image materials are sourced from Biorender (https://biorender.com) and The Human Protein Atlas (https://www.proteinatlas.org) (Supplementary material).

Imaging techniques [computed tomography (CT), magnetic resonance imaging (MRI), and ultrasonography (US)] are pivotal in the diagnosis, formulation of treatment strategies, and prognostic assessment of HCC[12] (Figure 2). US provides inexpensive, real-time, and noninvasive detection for patients with HCC, thereby establishing it as the primary screening modality[13]. However, its operator dependence and limited sensitivity constrain diagnostic accuracy[14]. MRI is the gold standard for radiological imaging of HCC and can significantly improve early-stage HCC detection efficacy[15]. Although MRI has superior diagnostic performance[16,17], its prolonged acquisition times due to enhanced soft-tissue contrast[18,19], combined with operator dependence and elevated costs, limit clinical applicability. CT is another gold standard for radiological imaging of HCC and has a high spatial resolution and a short scan duration; thus, CT has emerged as a more pragmatic option for routine clinical practice[18].

However, conventional CT image analysis relies on manual interpretation, which suffers from high subjectivity and poor reproducibility and fails to meet the growing demands for precision medicine in clinical practice[20]. In recent years, deep learning (DL), a prominent subfield of artificial intelligence (AI), has propelled the intelligent transformation of medical image analysis through its exceptional capabilities in image recognition, feature extraction, and classification prediction. DL models have been extensively implemented across liver imaging processing workflows. These implementations encompass HCC segmentation, detection and prognostic prediction, while also providing clinical decision support for therapeutic interventions, thereby significantly increasing both precision and efficiency across the diagnostic, therapeutic management, and prognostic assessment continuum in HCC care.

This review systematically integrates DL technologies based on CT imaging for detecting HCC, with a primary focus on tumor diagnosis[21-31], segmentation[32-49], treatment response prediction[50-56], and patient prognosis prediction[57-70]. Moreover, we review popular DL networks in various fields and describe their advantages for different applications. Furthermore, we discuss the outstanding challenges in applying DL to extract information from CT images for diagnosing and treating of HCC patients. These insights could provide guidance for subsequent studies.

CONVOLUTIONAL NEURAL NETWORKS AND ITS DERIVATIVE MODELS

Classical convolutional neural networks

Convolutional neural networks (CNNs) are among the most important neural network architectures in DL and are well suited for image data processing and analysis[71]. The design of CNNs is inspired by the receptive field mechanism in biological neural systems, where each neuron connects only to a local region of the feature map from the previous layer[72]. A typical CNN consists of multiple convolutional layers, activation functions [such as rectified linear unit (ReLU)], pooling layers, normalization layers, and fully connected layers, thus forming an end-to-end automated learning system[73] (Figure 3A)[74]. Multiple convolutional layers in the early part of the network progressively capture low-, mid-, and high-level features of images, such as edges, textures, and shapes[75]. The subsequent pooling layers compress the feature maps by down sampling, thereby improving the model’s robustness to transformations such as translation and scaling[76].

Figure 3 Deep learning networks architecture diagram. A: The architecture of convolutional neural network[74]; B: The architecture of U-shaped network[79]; C: The architecture of residual network[84]; D: The architecture of densely connected convolutional network[84]; E: The architecture of visual geometry group network[93]; F: The architecture of Transformer network[96]; G: The architecture of Efficient network[101]. ReLU: Rectified linear unit; conv: Convolution; MBconv: Mobile inverted bottleneck convolution; BN: Batch nomalization; SE: Squeeze and Excitation. Citation for Figure 3A: Zhang J, Yang Y, Feng X, Xu H, Chen J, He Y. Identification of Bacterial Blight Resistant Rice Seeds Using Terahertz Imaging and Hyperspectral Imaging Combined With Convolutional Neural Network. Front Plant Sci 2020; 11: 821. Copyright ©The Author(s) 2020. Published by Frontiers Media SA; Citation for Figure 3B: Pravitasari A, Asnawi M, Nugraha F, Darmawan G, Hendrawati T. Enhancing 3D Lung Infection Segmentation with 2D U-Shaped Deep Learning Variants. Appl Sci 2023; 13: 11640. Copyright ©The Author(s) 2023. Published by Multidisciplinary Digital Publishing Institute; Citation for Figure 3C and D: Wang X, Mi Y, Zhang X. 3D human pose data augmentation using Generative Adversarial Networks for robotic-assisted movement quality assessment. Front Neurorobot 2024; 18: 1371385. Copyright ©The Author(s) 2024. Published by Frontiers Media SA; Citation for Figure 3E: Zhang X, Feng Q, Zhu D, Liang X, Zhang J. Compressing recognition network of cotton disease with spot-adaptive knowledge distillation. Front Plant Sci 2024; 15: 1433543. Copyright ©The Author(s) 2024. Published by Frontiers Media SA; Citation for Figure 3F: Choi SR, Lee M. Transformer Architecture and Attention Mechanisms in Genome Data Analysis: A Comprehensive Review. Biology (Basel) 2023; 12: 1033. Copyright ©The Author(s) 2023. Published by Multidisciplinary Digital Publishing Institute; Citation for Figure 3G: Nigam S, Jain R, Singh VK, Singh AK, Krishna H. Automated severity level estimation of wheat rust using an EfficientNet-CBAM hybrid model. Front Plant Sci 2025; 16: 1540642. Copyright ©The Author(s) 2025. Published by Frontiers Media SA. The authors have obtained the permission for figure using (Supplementary material).

U shaped network

U shaped network (UNet) is a classical CNN architecture specifically designed for medical image segmentation and has been widely applied in the automated identification and localization of lesions[77]. Its structure consists of a symmetric encoder and a decoder path: The encoder extracts deep semantic features through multilayer convolution and pooling operations, while the decoder gradually restores spatial resolution by up sampling[78] (Figure 3B)[79]. Skip connections between the encoder and decoder integrate shallow spatial information with deep semantics, thereby significantly improving segmentation accuracy and preserving boundary details[80].

Residual network

Residual network (ResNet) was first proposed by He et al[81] to address the issues of vanishing gradients and degradation in deep network training, thereby enabling effective training of much deeper networks. The core innovation of ResNet lies in its introduction of the residual block[82], which consists of a stack of convolutional layers, batch normalization layers, and a nonlinear activation function (ReLU)[83] (Figure 3C)[84]. A substantial body of empirical evidence demonstrates that residual connections can significantly alleviate the difficulty of fitting training samples when training deep neural networks, while maintaining strong generalization performance on test samples[85].

Densely connected convolutional network

Densely connected convolutional network (DenseNet) was proposed by Huang et al[86]. The main advantage of this network is that it has fewer parameters than a standard convolutional network does[87]. Each convolutional layer in the network receives additional input from all preceding layers and then passes its feature maps to all subsequent layers[88] (Figure 3D)[84]. Each layer of DenseNet can directly receive gradients from the loss function and the original input signal, thereby achieving implicit deep supervision and alleviating the vanishing gradient problem, thereby facilitating the training of deeper network architectures[89].

Visual geometry group network

The visual geometry group (VGG) network (VGGNet) architecture, which has a moderately deep yet uniform structure, was designed to achieve high accuracy in image classification tasks[90]. Its design is similar to that of AlexNet, thus implying that VGGNet possesses many parameters[91]. The network comprises six distinct CNN configurations, namely VGG11, VGG11 (local response normalization), VGG13, VGG16 (convolutional 1), VGG16, and VGG19[90,91]. Taking VGG16 as an example, its 16 convolutional layers are divided into two 3 × 3 convolutional layers, accompanied by 2 × 2 max pooling and two blocks, including three layers of 3 × 3 convolutions succeeded by 2 × 2 max pooling[92] (Figure 3E)[93]. Owing to the nonlinearities in the blocks, this network provides significantly enhanced discriminative capability, which is one of its key advantages[92]. Another advantage of the block design is that it allows for two ReLU activations to be performed after each convolution[92].

Transformer network

The transformer network is a DL architecture based on the attention mechanism[94]. The model is structured as a stack of multiple identical layers, each consisting of two key components: A multi-head self-attention (MSA) module and a position wise feedforward network[95] (Figure 3F)[96]. The MSA mechanism, which is central to the transformer’s operation[97], computes the response at a given pixel position by weighting information from all other positions, thereby enabling it to dynamic measurement of the correlation of pixels at all other locations in the image[98]. This model boasts several advantages, including a strong ability to learning long-range dependencies, powerful multimodal fusion, and good interpretability[89]. It has demonstrated remarkable achievements in tasks such as image classification and segmentation.

Efficient network

The efficient network (EfficientNet) architecture family can provide an appropriate method to scale CNNs for improved accuracy and efficiency[99]. The first model introduced was EfficientNet-B0, which employs a compound scaling method that uses a fixed set of coefficients to uniformly scale network width, depth, and resolution; this model contains 5.3 million parameters and takes 224 × 224 images as input[100] (Figure 3G)[101]. This approach allowed the authors to generate a highly efficient CNN architecture. Furthermore, they applied the same compound scaling method to EfficientNet-B0 to obtain the scaled variants EfficientNet-B1 to B7[102]. Compared with traditional models, EfficientNets are smaller in size, faster in inference speed, and highly accurate[103].

AIDING HCC DIAGNOSIS VIA CT-BASED DL

CT is widely used in the diagnosis of HCC[104,105]. However, conventional CT image analysis techniques are time-consuming and subjective[106]. Distinguishing HCC from hepatic lesions such as intrahepatic cholangiocarcinoma, focal nodular hyperplasia, and metastases can also be challenging[106]. Recently, DL models have been widely employed in HCC diagnosis because they can automatically detect subtle morphological alterations in hepatic lesions and enhance image resolution, thereby significantly improving the sensitivity and accuracy of HCC diagnosis[26,107-110].

In this review, modified ResNet and its integrations with other networks were widely used DL approaches in HCC diagnosis based on CT imaging[21-24] (Table 1). Among all the network models, the AI system proposed by Wang et al[21] stands out because of the many patients in its training set. This AI system consists of NoduleNet and HCCNet, and both networks are built upon the ResNet architecture[21]. Moreover, NoduleNet which can identify nodule images, was designed to function as an auxiliary model[21]. These advantages contribute to the system’s robust performance; for example, this system achieved area under the curves (AUCs) of 0.88 and 0.89 on internal and external validation sets, respectively[21]. To fully use the spatial information in CT images, some researchers have used three-dimensional (3D) ResNet architectures to extract features from these images[22,23]. To compare the performance between nonenhanced phase CT and contrast-enhanced phase CT scans within a 3D ResNet-based image analysis model, Ling et al[22] proposed both a base model and an enhanced model built upon the 3D ResNet architecture. In the base model, the authors used nonenhanced phase CT, whereas in the enhanced model, they employed contrast-enhanced phase CT scans[22]. The results demonstrated that the enhanced model significantly outperformed the base model in terms of execution performance[22]. To improve diagnostic performance, the authors introduced patient sex and age into the model and proposed the minimum extra information about patients and lesions model[22]. Achieving an AUC of 0.96, the model demonstrated significantly improved sensitivity and specificity[22]. Moreover, other researchers have explored alternative approaches to enhance the performance of 3D ResNet models in extracting and diagnosing HCC-related information from CT scans. Guo et al[23] proposed ALARM which integrates DL with the aMAP HCC risk score. This risk score, which is calculated using five common clinical variables (age, male sex, albumin-bilirubin concentration and platelet count), was developed to predict the likelihood of HCC occurrence in patients with chronic hepatitis[111]. Achieving AUCs of 0.90 and 0.92 on the internal and external validation sets, respectively, ALARM can effectively predict the short-term development of HCC in patients with liver cirrhosis[23].

Table 1 Characteristics of deep learning networks for diagnosis of hepatocellular carcinoma from computed tomography images, mean ± SD.

Ref.	Method	Base network	Type of CT	Train set	Test set	Validation set	ACC	Sen	Spe	AUC	DICE
Wang et al[21]	HCCNet and NoduleNet	ResNet	NCCT/CECT	7512	385/556		0.81/0.81	0.78/0.89	0.84/0.74	0.89/0.88
Ling et al[22]	3D ResNet	ResNet	CECT	480 or 481	120 or 121		0.92 (0.87, 0.96)	0.95 (0.90, 1)	0.88 (0.82, 0.91)	0.96 (0.93, 0.98)
Guo et al[23]	ALARM	3D ResNet50 and nnUNet	CECT	924		231/703		0.92/0.94	0.89/0.93	0.90/0.92
Kim et al[24]	MASK R-NN	ResNet101 and UNet and FPN and RPN	CECT	568	589			0.85			0.96
Shan et al[25]	3D ResUNet	ResUNet	CECT								0.88
Zossou et al[26]	RA-UNet	UNet	CT	4536 slices	315 slices	1134 slices	0.94	0.94			0.88
Chen et al[27]	SED	UNet and DenseUNet	CT	4000 slices	300 slices	800 slices	0.99			0.95	0.75
Gao et al[28]	STIC	CNN and gated RNN	CECT	499	113/111		0.93 ± 0.04	0.93 ± 0.10	0.94 ± 0.04	0.99 ± 0.01
Rocha et al[29]	CNN	CNN	CECT	317	79		0.95	0.92 ± 0.01		0.99 ± 0.00
Khan et al[30]	Multi-modal deep neural network	AlexNet	CECT	248	75	75	0.96	> 0.94	> 0.98	0.83
Balagourouchetty et al[31]	FCNet	GoogleNet	CECT	444	190		0.97			0.995

CT: Computed tomography; AUC: Area under curve; ACC: Accuracy; Sen: Sensitivity; Spe: Specificity; DICE: Dice coefficient; ResNet: Residual network; 3D: Three-dimensional; ALARM: Alarm; MASK R-NN: Mask region-based convolutional neural network; ResUNet: Residual U shaped network; RA-UNet: Residual attention U shaped network; SED: Successive encoder-decoder; CNN: Convolutional neural network; FCNet: Fully connected neural network; RNN: Recurrent neural network; UNet: U shaped network; FPN: Feature pyramid network; RPN: Region proposal network; nnUNet: Neural network U shaped network; NCCT: Non-contrast computed tomography; CECT: Contrast enhanced computed tomography; STIC: Spatial extractor temporal encoder integration classifier.

Modified UNet and its integration with other networks are also frequently used for diagnosing of HCC on the basis of CT images[23,24,26,27] (Table 1). Differentiating between primary HCC and metastatic cancer is particularly critical for determining a patient’s subsequent treatment plan. However, as mentioned previously, distinguishing HCC from other liver lesions poses certain challenges[106]. To address this challenge, Zossou et al[26] proposed a model that connects a residual attention UNet (RA-UNet) and a 9-layer CNN classifier in series. RA-UNet is used for segmenting of CT images, and both the segmented results and the original images are fed into a CNN[26]. Finally, RA-UNet achieved a classification accuracy of 93.97%[26]. Similarly employing a cascaded network architecture, Chen et al[27] proposed a successive encoder-decoder (SED) model to detect HCC. The difference was that they employed a cascaded framework of UNet and Dense UNet[27]. SED-1 (UNet) is primarily used to remove unwanted voxels and organs and extract the liver’s location from CT images[27]. SED-2 (Dense UNet) uses the output from SED-1 to further segment the lesions[27]. This model achieved an accuracy of 0.99 and an AUC of 0.95 in extracting HCC from CT images[27]. To provide a more intuitive visualization of the shape and location of the HCC, the authors also reconstructed the two-dimensional (2D) segmentation results from the SED model into a 3D format[27], which is highly beneficial for clinical practice.

AIDING HCC THERAPY PLANNING VIA CT-BASED DL

Pretreatment HCC segmentation

Accurate HCC segmentation prior to treatment is critical in HCC therapy. However, HCC segmentation is particularly challenging because of the similar tissue intensity between the liver and adjacent organs and the high variability in tumor morphology and size[48,112,113]. Current HCC segmentation is accomplished primarily through three technical approaches: Manual, semiautomatic, and automatic[114]. Because automatic segmentation reduces interobserver variability and decreases reliance on operator experience, it is gaining increasing attention in clinical practice. Automatic segmentation is achieved primarily through two technical approaches: Machine learning and DL[115]. Although machine learning has substantially increased the precision of automatic segmentation, the continued need for manual feature engineering compromises system robustness[116]. Owing to its superior feature learning capabilities, DL is being increasingly adopted by researchers for automated medical image analysis, including liver tumor segmentation.

In this review, modified UNets are the most prevalent DL approaches for HCC segmentation based on CT imaging (Table 2). To further improve the segmentation efficiency of UNet, researchers have enhanced to its architecture, either through cascaded UNet networks or the integration of specialized modules[34,35,37-41]. Ouhmich et al[34] designed a three-stage cascaded neural network system based on UNet to perform liver tissue segmentation. The basic process is as follows: The initial network module processes input CT scans to segment liver regions; The second network uses the segmentation results from the first network to segment liver tumor lesion segmentation; And the third network segments the necrotic tissue within the lesion[34]. This cascaded architecture performed exceptionally in automated liver segmentation[34]. Moreover, researchers have explored the integration of specialized modules into the UNet architecture to increase its segmentation accuracy[35,39]. Residual and dense blocks are the most frequently used blocks in modified UNet[35,37,38,40]. The former can enhance detailed feature extraction from images while effectively addressing segmentation deficiencies in low-contrast regions and weakly defined boundaries[117]. Additionally, it can effectively address the vanishing gradient problem that occurs with increasing network depth[117]. The latter can concatenate feature maps and perform dimension-level merging thereby reducing the number of required input feature maps[37]. It not only reduces computational complexity but also mitigates gradient vanishing[37]. To reduce training losses and improve segmentation performance, Khan et al[35] proposed the residual multiscale UNet (RMS-UNet) network. They incorporated dilated convolution and residual modules into the UNet architecture. Dilated convolution with multiple dilations can improve the receptive view of input CT images without any change in kernel size[35]. However, adding more convolutional layers could introduce training errors; thus, a ResNet was introduced to avoid overfitting[35]. Both qualitative and quantitative results demonstrate the superior performance of RMS-UNet[35]. To obtain sufficient image features and parameters, reduce runtime, and minimize interference from irrelevant regions, Chen et al[37] designed a novel residual-dense-attention (RDA) UNet network by introducing the following into a UNet network: A residual block from a residual neural network, a dense block from DenseNet and an attention gates module from an attention UNet model. The proposed RDA UNet achieved high accuracy while reducing runtime by 28%[37]. Nevertheless, the forementioned studies focused primarily on 2D UNet architectures, thus inherently limiting the full use of spatial information. To fully exploit spatial information during image segmentation process, Çiçek et al[118] proposed the 3D UNet architecture. Regardless, compared with 2D UNet, 3D UNet requires more memory and has a longer computational time[119]. To address these challenges, Li et al[38] formulated a novel hybrid densely connected UNet, which integrated 3D DenseUNet with 2D DenseUNet via an auto-context mechanism; CT intra-slice and inter-slice features were jointly optimized through a hybrid feature fusion layer[38]. Experimental results on public medical imaging datasets demonstrated that the proposed method performs competently in HCC segmentation[38]. Wang et al[39] also designed a hybrid 3D UNet network to segment HCC. In designing the network, which they designated as a multiscale attention and deep supervision-based 3D UNet, the authors incorporated multiscale attention and deep supervision mechanisms into the 3D UNet architecture[39]. The introduction of the attention mechanism increased the network model’s ability to use multiscale contextual spatial information[39]. The incorporation of supervision mechanisms improved the model’s overall accuracy[39]. The proposed model, which the authors evaluated on public datasets, demonstrated superior computational efficiency and performance in liver cancer segmentation[39].

Table 2 Characteristics of deep learning networks for segmentation of hepatocellular carcinoma from computed tomography images, mean ± SD.

Ref.	Method	Combine	Type of CT	Train set	Test set	Validation set	VOE (%)	RVD (%)	ASD (mm)	RMSD (mm)	DICE (%)
Nakai et al[32]	CNN		CECT	493	62	62
Shah et al[33]	MDL-CNN	Cascade	CECT	2948 slice	1264 slice		9.8	8.2			95.7
Ouhmich et al[34]	UNet	Cascade	CECT	Cross-validation (1:6)							68.1 ± 23.2
Khan et al[35]	RMS-UNet	Residual multi-scale	CT	101	21	21	14.95 ± 9.40	-0.7 ± 1.3	3.06 ± 3.13	1.60 ± 0.72	91.92 ± 0.05
Gong et al[36]	UNet-DRLSEIC	DRLSEIC	CT	110	40						95.2 ± 1.7
Chen et al[37]	RDA-UNet	ResNet DenseNet and UNet	CT	15611 slice		3903 slice					87.03
Li et al[38]	H-DenseUnNet	DenseNet and UNet	CECT	131	70	20	11.68 ± 4.33	-0.01 ± 0.05	0.58 ± 0.46	1.87 ± 2.33	93.7 ± 2
Wang et al[39]	MAD-Unet	Multi-scale attention and deep supervision	CECT	116	15		6.83 ± 2.31	0.34 ± 0.19	1.03 ± 0.37	3.74 ± 3.58	97.27 ± 1.22
Lee et al[40]	HFS-Net	DenseUNet and UNet	DNCT	298	179	118					82.8
Ou et al[41]	ResTransUNet	Transformer and UNet	CECT				8.04 ± 6.8	-0.07 ± 9.5			95.35 ± 4.5
Jiang et al[42]	Swin-UNet	SFTB and LCAB	CECT	104	26		37.38	-0.1577	5.1433		76.14
Clinton Atabansi et al[43]	ICT-Net	Transformer and convolution	CECT	1789	225	223					90.91
d’Albenzi et al[44]	DEDC-Net	ResNet and VGG-19	CECT	101	15	15			12.17 ± 12.67		46.1 ± 27.4
Singh et al[45]	FasNet	ResNet-50 and VGG-16	CECT								87.66
Guo et al[46]	FCN and ACM	FCN and ACM	CT	42	16	19			1.6 ± 0.5	3.5 ± 1.2	95.8 ± 1.4
Zhang et al[47]	DeepRecS	RMP-Net and CGBS-Net	CECT	139	46	46	15.88 ± 3.79	0.32 ± 6.45	0.47 ± 0.45	1.57 ± 1.46	91.32 ± 2.30
Balasubramanian et al[48]	APESTNet and Mask R-CNN	APESTNet and Mask R-CNN	CECT	121	10	20	5.37 ± 3.27	-1.08 ± 2.06	1.85 ± 0.30		97.31 ± 1.49
Liu et al[49]	S2DANet	FSMF and MAHA and GMCA	CECT	92	26	13	43.95	0.3861	10.80		69.51

VOE: Volumetric overlap error; CT: Computed tomography; RVD: Relative volume difference; ASD: Average symmetric surface distance; RMSD: Root mean square symmetric surface distance; DICE: Dice coefficient; DNCT: Dynamic computed tomography; CECT: Contrast-enhanced computed tomography; MDL: Multi-channel deep learning; CNN: Convolutional neural network; UNet: U shaped network; RMS: Residual multi scale; DRLSEIC: Distance regularized level set evolution D constrained; RDA-Unet: Residual-dense-attention-U shaped network; H-DenseUnNet: Hybrid densely connected U shaped network; MAD: Multi-scale attention and deep supervision; HFS: Hierarchical fusion strategy; ResTransUnet: An effective network combined with transformer and U shaped network for liver segmentation in computed tomography scans; ICT-Net: Integrated convolution and transformer-based network; DEDC-Net: The dual-encoder double concatenation network; FCN: Fully convolutional network; ACM: Active contour model; DeepRecS: A novel deep learning based interactive method for liver tumor segmentation, which incorporated RECIST-marks as guiding signals; APESTNet: Enhanced Swin transformer network with adversarial propagation; Mask R-CNN: Mask region-based convolutional neural network; S2DANet: Spatial and spectral-learning double-branched aggregation network; ResNet: Residual network; DenseNet: Densely connected convolutional network; SFTB: Swin-neighborhood fusion transformer block; LCAB: Large-kernel convolutional attention block; VGG: Visual geometry group; RMP-Net: RECIST mark propagation network; CGBS-Net: Context-guided boundary-sensitive network; FSMF: Fourier spectral-learning multi-scale fusion; MAHA: Multi-axis aggregation Hadamard attention; GMCA: Group multi-head cross-attention aggregation.

Beyond UNet, various other CNN architectures have been successfully employed for HCC segmentation tasks (Table 2). The hybrid ResNet and VGGNet configuration represents a frequently used approach[44,45]. ResNet can extract deep features and VGGNet can effectively utilize spatial information[44,45]. On the basis of this theoretical framework, d’Albenzio et al[44] proposed a novel dual-encoder double concatenation network in which two parallel encoders (VGG19 and ResNet) were used to extract features[44]. These features were passed to a single decoder[44]. This network demonstrated superior HCC segmentation capability. Subsequently, Singh et al[45] designed the FasNet, which integrates VGG16 and ResNet50. Similarly, this model performed exceptionally well in HCC segmentation[45].

Treatment response prediction

Surgical resection and liver transplantation remain the primary potentially curative modalities for treating HCC[120]. However, some patients have already lost the opportunity for surgical treatment by the time they are diagnosed[121]. In cases of unresectable HCC at the early to intermediate stages, locoregional therapies now constitute the primary treatment modality[122,123]. Locoregional therapies primarily consist of the following three modalities: Radiofrequency ablation, TACE, and transarterial radioembolization. For patients with advanced-stage HCC, systemic therapy may provide clinical benefit[124]. Systemic therapies for HCC include chemotherapy, targeted therapy, and immunotherapy[125,126]. For decades, chemotherapy has remained the standard systemic treatment option for advanced HCC, but patient benefits remain limited[127]. In recent decades, novel molecular-targeted and immunotherapy drugs have been developed[126], these drugs have revolutionized the management of advanced HCC and offered new hope where limited options previously had existed[128]. Although both locoregional therapies and emerging targeted immunotherapies provide varying degrees of clinical benefit, therapeutic responses exhibit significant interpatient heterogeneity, particularly for targeted agents and immunotherapies, where treatment efficacy often depends on the patient’s genetic profile[129-131]. Accurate prediction of patient treatment responsiveness will facilitate optimal therapeutic strategy development by clinicians and promote personalized medicine. Imaging techniques have consistently served as cornerstone modalities for assessing treatment response in patients with HCC[132-134]. Manual analysis of medical images relies on experienced radiologists and is susceptible to subjective variability and other limitations. Consequently, Lambin et al[135] pioneered the concept of radiomics to employ high-throughput feature extraction algorithms to derive quantitative characteristics from medical images. The emergence of machine learning and DL has further accelerated the advancement of medical imaging analysis technologies[136]. The integration of radiomics, machine learning, and DL has substantially increased the utilization efficiency of medical imaging information[137]. In predicting treatment response in patients with HCC, researchers frequently integrate DL, machine learning, and radiomics to increase predictive efficiency[51,53]. Some investigators have further incorporated clinical data into analytical networks to further improve performance[54,56].

For predicting treatment response in patients with HCC, modified ResNet is the most widely adopted CNN architecture and is frequently integrated with radiomics data and clinical variables in this review[50,52-56] (Table 3). Liao et al[54] and Lin et al[55] used the ResNet18-based models to predict treatment response in patients with HCC receive immune checkpoint inhibitor (ICI) therapy and combination lenvatinib and ICI therapy, respectively. Three additional studies utilized ResNet50-based models to predict treatment response in patients with HCC who receive either TACE therapy or TACE-hepatic arterial infusion chemotherapy (HAIC) combined with a programmed cell death 1 (PD-1) inhibitor and tyrosine kinase inhibitor (TKI) therapy[50,53,56]. ResNet18 is an 18-layer architecture that employs residual connections to mitigate the vanishing gradient problem[138]. It has lower computational costs while maintaining the benefits of deeper models[81]. Liao et al[54] used ResNet18 to integrate information from multiphase CT images to predict treatment response in patients with HCC who receive combination lenvatinib and ICI therapy, which provided it superior predictive accuracy, achieving an AUC of 0.802[54]. To improve accuracy, Lin et al[55] employed an integrated model that combined features from ResNet18, radiomics, and clinical data to predict treatment response in patients with HCC receive ICI therapy[55]. Predictions made by this model were clearly superior to those made using radiomics or clinical data alone[55]. Compared with ResNet18, ResNet50 comprises more layers, thus enabling it to capture more complex features[81]. The incorporation of residual connections enhances trainability at such depths[81]. ResNet50 delivers significantly superior performance, particularly on large datasets and more complex image processing tasks, thereby enhancing the classification accuracy of DL models[81]. Owing to these advantages, several studies have used it to predict the treatment response in patients with HCC[50,53,56]. Peng et al[50] used ResNet50 to predict treatment response patients with HCC who receive TACE therapy. This network achieved accuracies of 85.1% and 82.8% in the two validation sets[50]. With respect to predicting the response of patients with HCC to TACE-HAIC combined with PD-1 inhibitors and TKI therapy, Yin et al[56] proposed a combined model that integrates ResNet50, radiomics, and clinical data to improve accuracy. This model achieved a remarkable accuracy of 89.5% in the training cohort, which is highly encouraging[56].

Table 3 Characteristics of deep learning networks for treatment response of hepatocellular carcinoma from computed tomography images.

Ref.	Therapy	Base network	Combine model			Type of CT	Train set	Test set	Validation set	CR and PR (%)	SD and PD (%)	AUC (%)	ACC (%)
Peng et al[50]	TACE	ResNet50			DL	CECT	562		89/138	41.58/42.03	58.42/57.97	96 (94-97); 97 (96-98)	85.1/82.8
Peng et al[51]	TACE	DL	R		DL	CECT	139		171	60.82	39.18	99.4 (98.7-100)
Sun et al[52]	TACE	ResNet18	R	C	DL	CECT	299		100	43	57	0.91 (0.85-0.97)
Lin et al[53]	TACE	ResNet50	ML (SVC)		DL	CECT	42	26				92 (90-94) (SVC)	81 (80-82)
Liao et al[54]	CLICI	ResNet18			DL	CECT	72		48	27.1	72.9	80.2 (78.0-82.4)	72.5
Lin et al[55]	ICI	ResNet18	R	C	DL	CECT	153	50		70	30	0.88 (0.77-0.99)
Yin et al[56]	TACE-HAIC and ICI and TKI	RseNet50	R	C	DL	CECT	92	30	50	24	76	0.85	79.1

CR: Complete response; CT: Computed tomography; PD: Progressive disease; PR: Partial response; SD: Stable disease; ACC: Accuracy; AUC: Area under the receiver operating characteristics curve; CECT: Contrast-enhanced computed tomography; C: Clinical data; DL: Deep learning; R: Radiomics; ML: Machine learning; SVC: Support vector classifier; ResNet: Residual network; CLICI: Combination therapy of lenvatinib and immune checkpoint inhibitor; TACE: Trans-arterial chemoembolization; HAIC: Hepatic arterial infusion chemotherapy; ICI: Immune checkpoint inhibitor; TKI: Tyrosine kinase inhibitor.

CT-BASED DL TO PREDICT HCC PROGNOSIS

As previously stated, a variety of therapeutic approaches exist for curing and palliating of HCC. Despite this, the prognosis for HCC patients remains poor[139]. Curative treatment options for early-stage HCC provide a 5-year survival rate of more than 70%[4]. However, up to 70% of patients with HCC experience tumor recurrence within 5 years after undergoing resection or ablation therapy[140]. For patients with advanced liver cancer who have undergone systemic therapy, the median survival is only 1 year to 1.5 years[4]. Monitoring a patient’s disease status and identifying those with a poor prognosis inform clinical decision-making and enable timely intervention. The molecular and pathological profiles of HCC are strongly correlated with biological aggressiveness and have significant prognostic value[141]. However, assessing these factors requires tissue samples, which are not readily available for most HCC patients[139]. Fortunately, imaging techniques can depict entire tumors and offer noninvasive insights into their biology and heterogeneity[139]. Advances in radiomics and DL have greatly enhanced the use of CT imaging data, improved the accuracy of prognosis prediction, and provided valuable insights for clinical treatment[142].

Recurrence

Tumor recurrence is a major determinant of overall survival (OS) in patients with HCC[143]. HCC recurrence is classified into early and late recurrence. Early recurrence, defined as recurrence occurring within two years of HCC treatment, is primarily attributed to occult intrahepatic dissemination from the original tumor and is correlated with a high initial tumor burden[144]. In contrast, late recurrence (≥ 2 years) is not related to the primary lesion but rather to the development of de novo HCC[144]. Compared with patients with late recurrence, patients with early recurrent HCC have significantly shorter post-recurrence survival[145]. Therefore, predicting HCC recurrence and its pattern (early vs late) will assist clinicians in formulating both post-treatment surveillance strategies and corresponding treatment plans. In using CT imaging to develop prognostic models for HCC recurrence, researchers have increasingly integrated clinical variables with DL-based radiographic analysis within a multimodal framework[57-62].

In this review, modified ResNet is the most frequently employed CNN architecture that uses CT imaging to predict HCC recurrence[57,58,62] (Table 4). Furthermore, all researchers integrated the modified ResNet model with clinical data to improve its predictive accuracy[57,58,62]. To maximize the use of information from every phase of contrast enhanced CT and improve predictive accuracy, Wang et al[57] embedded an attention mechanism into the ResNet network. Intra-phase attention focuses on important information across different channels and spatial locations within a single phase, whereas inter-phase attention focuses on salient features across different contrast phases[57]. When combined with clinical data, the model achieved an accuracy of 81.2% in predicting HCC recurrence[57]. To increase prediction accuracy, Lv et al[58] adopted a multimodal fusion approach that integrates radiomic data, clinical variables, and the ResNet50 model, thus resulting in an AUC of 0.83. Notably, to select the optimal network for constructing a prediction model, Zhang et al[59] evaluated and compared the predictive performance of four architectures DenseNet121, ResNet101, ResNet50, and VGG19 on separate training, validation, and test sets. Owing to its superior stability and ability to minimize data loss, VGG19 was ultimately selected by researchers to construct a prediction model. Their results demonstrated that the 2.5D DL model was superior to the 3D model[59]. Subsequently, the integration of the 2.5D model with clinical data further enhanced the predictive performance, thereby achieving an AUC of 0.804 in the external validation set[59].

Table 4 Characteristics of deep learning networks for prognosis of hepatocellular carcinoma from computed tomography images, mean ± SD.

Ref.	Therapy	Predict aim	Base network	Combine model			Type of CT	Train set	Test set	Validation set	C-index	ACC	AUC
Wang et al[57]	SR	Recurrence	ResNet18	C		DL	CECT	Ten-fold cross-validation (167)				0.81 ± 0.01	0.87 ± 0.03
Lv et al[58]	SR	Recurrence	ResNet50	C	R	DL	CECT	156	68				0.83 (0.80-0.87)
Zhang et al[59]	SR	Recurrence	VGG19	C		DL	CECT	162		70/91		0.714	0.80
Gao et al[60]	LT or SR	Recurrence	DSViT			DL	CECT	5-fold cross validation (204)				0.76 ± 0.06	0.80 ± 0.04
Yao et al[61]	SR	Recurrence	DenseNet			DL	CECT	180		122	0.78 (0.76-0.79)		0.80 (2 years)
Hui et al[62]	SR	Recurrence	ResNet	C		DL	CECT	536	560	153			0.86 (0.78-0.92)
Sun et al[63]	TACE	Survival	ResNet101	C		DL	CECT	241		60	0.88 (0.80-0.96)	0.93	0.96 (0.88-1.00)
Dai et al[64]	TACE	Survival	ResNeXt	C	R	DL	CECT	115	165	30		0.80	0.89
Wei et al[65]	SBRT	Survival	CNN-survNet	C	R	DL	CECT	100	33	34	0.65 (0.64-0.68)
Chen et al[66]	SBRT	Survival	ResNet50	C	R	DL	CECT	Nested cross-validation					0.86 (0.80-0.93)
Ren et al[67]	TACE and TKI	Survival	Resnet50			DL	CECT	10-fold cross validation (103)				0.92	0.94
Xia et al[68]	IT	Survival	EfficientNet	C		DL	CECT	129	46	32	0.75 (0.66-0.84)		0.839
Xu et al[69]	IT	Survival	EfficientNet; Semisupervised; CNN-Transformer	C		DL	CECT	520	209	130	0.74 (0.70-0.78)		0.84 (0.80-0.88) (2 years)
Lee et al[70]	M	Survival	DenseNet121	C		DL	CECT	507	146		0.821 ± 0.022		0.89 ± 0.02

CT: Computed tomography; ACC: Accuracy; AUC: Area under the receiver operating characteristics curve; CECT: Contrast-enhanced computed tomography; DL: Deep learning; C: Clinical data; R: Radiomics; ResNet: Residual network; VGG: Visual geometry group; DSViT: Dual-style vision transformer; ResNet: Residual network; DenseNet: Densely connected convolutional network; CNN-survNet: Convolutional neural network-survival network; EfficientNet: Efficient network; CNN: Convolutional neural network; LT: Liver transplantation; SR: Surgical resection; TACE: Trans-arterial chemoembolization; SBRT: Stereotactic body radiotherapy; TKI: Tyrosine kinase inhibitor; IT: Immunotherapy; M: Multiple treatment modalities; C-index: Concordance index.

Survival

OS and progression-free survival (PFS) are the primary endpoints for measuring the efficacy of HCC treatment[146]. As previously mentioned, a variety of treatment modalities are currently available for HCC, and patient survival has achieved improved markedly in some respects[147]. However, patients with HCC undergoing the same treatment show significant variation in the improvement of their survival[148]. The inherent heterogeneity of HCC itself dictates divergent responses to therapy and ultimately affects patient survival[149]. Therefore, pretreatment evaluation of post-therapeutic survival for individual patients by using diverse information contributes to the formulation of more personalized treatment approaches and ultimately leads to prolonged patient survival. Imaging information has long been an instrumental tool for assessing patient prognosis[139,150]. With the integration of machine learning and DL into radiology, a wealth of information concealed within medical images is being uncovered, thus dramatically increasing the predictive accuracy for patient survival[151].

In the context of CT-based survival prediction, modified ResNet remained the predominant model in this review[63,64,66,67] (Table 4). Notably, prior to constructing the combined model, both Chen et al[66] and Ren et al[67] compared the performance of ResNet50 with that of other convolutional models, and their results consistently demonstrated the superiority of ResNet50 over alternative architectures. In the study by Chen et al[66], hand-crafted radiomic features, DL features, and clinical data were integrated to predict the 2-year survival of HCC patients after stereotactic body radiation therapy; their model achieved an AUC of 0.86[66]. Ren et al[67] developed another model that integrated ResNet50, a feature selector based on mutual information maximization, and the nearest neighbors machine learning algorithm to predict the OS of HCC patients receiving a combination therapy of TACE and TKI; the model achieved an accuracy of 0.92[67]. It is noteworthy that Dai et al[64] used a 3D ResNeXt architecture in their study. ResNeXt, an architecture that builds upon the foundation of ResNet, integrates the “split-transform-merge” strategy of the inception network with a stacked building block approach[152]. ResNeXt has garnered significant attention because of its performance improvements on image classification tasks[153]. Dai et al[64] integrated hepatitis B surface antigen, five radiomics signatures, and a 3D ResNeXt architecture into a novel predictive model, which achieved an AUC of 0.89 in the validation cohort[64].

In addition to modified ResNet, modified EfficientNet is a widely adopted model for predicting the survival period of patients with HCC after treatment by using CT imaging information[68,69] (Table 4). The scalability and efficiency of EfficientNet render it highly suitable for medical image analysis applications, particularly those requiring high precision in resource-limited environments[154]. Notably, EfficientNet has been applied to predict the prognosis of patients with HCC after immunotherapy[68,69]. Building upon EfficientNet, Xia et al[68] proposed a convolutional-recurrent neural network architecture, which can decode spatial features from CT images and capture the temporal dynamics of tumor evolution between baseline and follow-up scans. When integrated with clinical data, the proposed model achieved an AUC of 0.839 for predicting 2-year survival in HCC patients treated with immunotherapy[68]. To improve the prediction of OS and PFS in HCC patients receiving ICI therapy, Xu et al[69] developed an integrated system that combined EfficientNet, a semi-supervised learning framework, a CNN-Transformer model, and clinical data. By fully using diverse data types, the model achieved an AUC of 0.84 in predicting 2-year patient survival[69].

FUTURE

As previously mentioned, DL has significantly improved the utilization efficiency of CT images for early diagnosis, treatment decision-making, and prognostic assessment of patients with HCC. Nevertheless, many challenges remain to be addressed. First, publication bias should be noted. Studies with negative or suboptimal results are rarely published. This imbalance may lead to an overly optimistic assessment of the potential for DL in clinical applications. Moving forward, it is paramount to encourage researchers to preregister their studies, which enables the discovery of all research outcomes, irrespective of their publication status. Furthermore, to further mitigate publication bias, specialized journals for publishing negative results should be established. Second, in terms of experimental design, most current studies are single-center and retrospective. Single-center studies carry the risk of significant regional bias, and a retrospective design precludes the possibility of real-time validation. Therefore, there is a clear need to initiate multicenter prospective studies, even randomized controlled trials. Third, as mentioned earlier, researchers have used either contrast-enhanced CT or non-contrast CT scans. The substantial difference in image quality and characteristics between these protocols presents a significant preprocessing challenge. Effectively mitigating this heterogeneity to ensure model generalizability remains a critical and unresolved issue in current research. Fourth, even within the same domain, multiple network models or combined network models are used for extracting information from CT images. Especially within the same application domain, different researchers have proposed a variety of fusion models with varying performance. However, as these models are trained and evaluated on non-standardized, heterogeneous datasets, their results are often not directly comparable. This lack of consistency inevitably creates significant challenges for clinical practitioners in selecting the optimal model for deployment. Sharing image data or using public datasets for model validation is imperative. Fifth, AI tools are often described as black boxes[155]. As a type of AI tool, DL also suffers from the black box problem[156]. This leads to poor interpretability of the results. The interpretability of a model’s output is crucial for clinicians to make corresponding decisions regarding diagnosis, treatment, and monitoring. Designing models with inherent interpretability, rather than creating methods to explain these black-box models, might be a more reliable approach to mitigate these issues[157]. However, this still requires further exploration from researchers. In addition to the technical issues mentioned above, the application of DL presents ethical and regulatory challenges. Adherence to regulatory and ethical frameworks is an essential prerequisite for integrating DL-based imaging into HCC clinical workflows. This process is inherently resource intensive, and requires considerable effort from researchers. However, currently, no applications of DL models have been approved by the United States Food and Drug Administration for image-guided treatments of HCC[158]. There are currently some Food and Drug Administration-approved radiology AI-based software devices for thoracic radiology[159]. This breakthrough paves the way for the much-anticipated integration of DL-based imaging into clinical management of HCC. That a growing number of DL-based imaging technologies will be integrated into HCC clinical practice soon is anticipated.

CONCLUSION

Over the past few years, DL has made remarkable progress in extracting information from CT images to aid clinicians in the early diagnosis, treatment response monitoring, and prognosis assessment of patients with HCC. Researchers often integrate multiple DL models to extract information from CT images. This approach compensates for the drawbacks of individual networks and increases their ability to extract information. In addition, researchers have integrated traditional radiomic and clinical data with DL models. This combined approach further enhances the model’s performance. All these advancements demonstrate that DL holds great promise for extracting information from CT images and advancing health care development. These promising application prospects, however, present several significant challenges. These include how to reduce CT image heterogeneity during preprocessing, enable data sharing, how to compare the performance of different methods, and interpret the output results. Therefore, the true integration of DL into clinical practice will require researchers’ ongoing and dedicated efforts.