Multimodal artificial intelligence technology in the precision diagnosis and treatment of gastroenterology and hepatology: Innovative applications and challenges

Abstract

With the rapid development of artificial intelligence (AI) technology, multimodal data integration has become an important means to improve the accuracy of diagnosis and treatment in gastroenterology and hepatology. This article systematically reviews the latest progress of multimodal AI technology in the diagnosis, treatment, and decision-making for gastrointestinal tumors, functional gastrointestinal diseases, and liver diseases, focusing on the innovative applications of endoscopic image AI, pathological section AI, multi-omics data fusion models, and wearable devices combined with natural language processing. Multimodal AI can significantly improve the accuracy of early diagnosis and the efficiency of individualized treatment planning by integrating imaging, pathological data, molecular, and clinical phenotypic data. However, current AI technologies still face challenges such as insufficient data standardization, limited generalization of models, and ethical compliance. This paper proposes solutions, such as the establishment of cross-center data sharing platform, the development of federated learning framework, and the formulation of ethical norms, and looks forward to the application prospect of multimodal large-scale models in the disease management process. This review provides theoretical basis and practical guidance for promoting the clinical translation of AI technology in the field of gastroenterology and hepatology.

Key Words: Artificial intelligence; Multimodal data; Gastroenterology; Hepatology; Precision medicine; Challenges and countermeasures

Core Tip: This manuscript systematically reviews the latest progress in multimodal artificial intelligence (AI) in gastroenterology and hepatology, focusing on innovative applications, including endoscopic AI (89% sensitivity for early gastric cancer detection), multi-omics models (42% objective response rate for programmed cell death 1-sensitive gastric cancer subtypes), magnetic resonance imaging-based liver fibrosis staging [area under the curve (AUC) = 0.89], and AI-driven hepatocellular carcinoma recurrence prediction (AUC = 0.91). It analyzes critical challenges (data standardization gaps, model “black boxes”) and proposes solutions (federated learning, interpretable frameworks), providing vital guidance for AI’s clinical translation to advance precision medicine.

INTRODUCTION

The revolutionary progress of artificial intelligence (AI) technology is profoundly reshaping the cognitive paradigm and practice of modern medicine, especially in clinical fields that deal with high-dimensional heterogeneous data[1]. As disciplines involving complex pathophysiological mechanisms, gastroenterology and hepatology are undergoing a critical transformation from empirical medicine to data-driven precision medicine[2]. The unique multi-level complexity of digestive system diseases, covering molecular pathway abnormalities, dynamic evolution of microenvironment, organ function compensation and systemic response, provides a unique application scenario technology. According to statistics[3], deep learning-based image analysis systems have increased the detection rate of early gastric cancer (EGC) to 92.7%[4], and the area under the curve (AUC) of the hepatocellular carcinoma (HCC) recurrence prediction model can reach 0.91[5]. These advances indicate that AI is becoming the core driving force to break through the bottleneck of traditional diagnosis and treatment (Supplementary Table 1).

Multimodal data integration is the most prominent feature of multimodal AI technology[6]. Compared with traditional methods that rely on a single data source, multimodal AI constructs a disease panorama from molecular to organ scale by jointly analyzing multi-dimensional information, such as endoscopic images, radiomics, pathological sections, genomic variants and real-time physiological monitoring[7]. For example, in the field of colorectal cancer, the fusion model of computed tomography (CT) radiomics features and circulating tumor DNA (ctDNA) methylation profiles improved the accuracy of lymph node metastasis (LNM) prediction by 23%[8]. The diagnostic consistency (kappa value) of ultrasound elastography combined with serum metabolomics analysis for non-invasive staging of liver fibrosis reaches 0.82[9]. This cross-scale data fusion not only overcomes the fragmentation defect of traditional biomarkers, but more importantly reveals the dynamic correlation mechanism among tumor heterogeneity, treatment resistance and microenvironment remodeling[10].

In the field of precision diagnosis and treatment of gastrointestinal cancer, multimodal AI has achieved three core breakthroughs. First, an intelligent analysis system for endoscopic images identified mucosal surface microstructural changes via convolutional neural network (CNN), resulting in an early diagnostic sensitivity of 89% for Barrett's esophagogastric-associated dysplasia[11]. Second, a deep-learning model of pathological sections could predict microsatellite instability (MSI) status directly from hematoxylin & eosin (H&E)-stained images with a specificity (87%) comparable to that of PCR, but with a 72-hour shorter test cycle[12]. More importantly, the multi-omics integration platform analyzed the interaction network between gene mutations and the spatial distribution of immune cells by graph neural network and successfully identified the molecular subtypes of gastric cancer sensitive to programmed cell death 1 (PD-1) inhibitors [objective response rate (ORR) increased to 42%][13]. These technological advances promote the transformation of individualized treatment strategies from the "one-size-fits-all" model to the "dynamic adaptation" model[14].

AI applications in liver disease management also show paradigm-shifting potential. Aiming at the clinical challenge of liver fibrosis staging, the FIB-Net model based on deep learning of ultrasound images achieved non-invasive diagnosis of F3-4 fibrosis by quantifying the liver capsule ripple characteristics and hemodynamic parameters (AUC = 0.89)[15]. In the process of liver cancer diagnosis and treatment, the multimodal AI system integrated three innovative dimensions: (1) Preoperative planning: The 3D U-Net model automatically segmented the tumor volume and calculated total tumor burden (TTB) by magnetic resonance imaging (MRI) images, which improved the prognosis prediction efficiency of the Barcelona Clinic Liver Cancer (BCLC) staging system by 18%[16]; (2) Treatment response prediction: The deep learning algorithm based on spatial transcriptomics could locate distribution hotspots of tertiary lymphatic structures (TLS) in the tumor and accurately screen the population of patients that would benefit from immunotherapy [hazard ratio (HR) = 0.32][17]; and (3) Recurrence monitoring: Liquid biopsy-driven LSTM model can identify early warnings of subclinical recurrence by tracking dynamic changes of ctDNA mutation allele frequency (sensitivity 93%)[18]. These advances mark the evolution of liver disease management from static staging to real-time adaptive intervention.

Despite the remarkable gains, the clinical translation of multimodal AI faces structural challenges. Lack of data standardization is the primary obstacle: Differences in image acquisition protocols between medical centers can shift radiomics feature values by up to 37%, resulting in a decrease in AUC of 0.15 in cross-institutional validation of the model[19]. The lack of algorithmic interpretability restricts clinical trust: Up to 68% of gastroenterologists reported that they could not understand the decision-making basis of the attention mechanism-based pathological diagnostic model[20]. In addition, ethical and regulatory frameworks lag behind technological innovation: The current medical AI approval system lacks evaluation criteria for multimodal products, resulting in 46% of innovative technologies being stuck in the "last mile"[21]. In view of these bottlenecks, the introduction of a federated learning (FL) framework provides new ideas for breaking down data siloing. For example, the gastric cancer recurrence prediction model integrates heterogeneous data from 23 centers worldwide through distributed training, improving the generalization of the model by 29%[19]. The development of interpretable AI technology has also made an important breakthrough. The hybrid architecture based on neural-symbolic systems can generate diagnostic reports that conform to pathological logic, with the adoption rate among doctors increasing from 58% to 82%[1].

From the perspective of technology evolution, AI applications in gastroenterology and hepatology are moving in four development directions: (1) Full-cycle closed-loop management: Digital twin technology integrates patients' genomic, metabolomic, and real-time imaging data to dynamically simulate disease progression trajectories and optimize intervention timing[6]; (2) Multimodal real-time interaction: The augmented reality endoscopy system can overlay lesion boundaries and AI-identified molecular typing information, enabling endoscopic treatment accuracy to reach the sub-millimeter level[22]; (3) Cross-scale mechanism analysis: The combination of spatial multi-omics and causal inference models is revealing the molecular coupling mechanism between hepatic stellate cells (HSC) activation and portal hypertension in the process of liver fibrosis[23]; and (4) Universal healthcare: The lightweight AI model reduced the number of parameters by 90% through knowledge distillation technology, such that the liver cancer screening accuracy of ultrasound equipment in primary hospitals was close to that in tertiary hospitals[24]. These innovations not only redefine the diagnosis and treatment standards of digestive system diseases but also give rise to the whole chain of "prevention-diagnosis-treatment-rehabilitation" intelligent medical ecology[11].

This review systematically summarizes the innovative application map of multimodal AI in gastrointestinal and liver diseases, and analyzes the solutions to key challenges such as data heterogeneity, model generalization, and clinical interpretability. By integrating the evidence chain of 128 cutting-edge studies, a transformation framework of "technology-clinical-supervision" co-evolution is proposed, which will provide theoretical support and practical guidelines[21] technology from laboratory to bedside[21].

APPLICATION OF MULTIMODAL AI TECHNOLOGY IN GASTROENTEROLOGY

Precise diagnosis and treatment of gastrointestinal tumors

AI analysis of endoscopic images: The rapid development of AI technology has significantly improved the accuracy in identifying EGC and predicting LNM. Endoscopic image analysis systems based on deep learning can identify the differentiation status and invasion depth of gastric cancer in real time. For example, the model developed by Liu et al[25], which analyzes gastric mucosal characteristics using the Kyoto gastritis score, achieves high sensitivity (92.7%) and specificity (87.7%) for early lesions, providing support for accurate endoscopic biopsy. In terms of LNM prediction, multimodal data analysis combined with AI (including tumor surface microstructure, vascular pattern, and depth of invasion) can significantly improve the accuracy of preoperative evaluation. For example, a user study proposed a combined prediction model based on absence of ulceration, high differentiation, and young age or proximal tumor location, with a negative predictive value (NPV) of 100%, providing a reliable basis for function-preserving surgery[26]. Furthermore, the enhanced Vision Transformer (ViT) developed by Ayana et al[27] has shown excellent classification performance in the early detection of colorectal cancer by fusing pathological features and imaging data. If this technology is transferred to gastric cancer, it is expected to further optimize the risk stratification of LNM. The combination of AI[4] and clinical models improves diagnostic efficiency, provides new ideas for individualized treatment decisions, and promotes the application of precision medicine in EGC management[4]. The process is shown in Figure 1.

Figure 1 Artificial intelligence-assisted diagnostic and clinical decision integration workflow for gastrointestinal cancers. Step 1: Input data layer: Integration of multi-dimensional data including molecular biomarkers (circulating tumor DNA, microsatellite instability status, CpG island methylation), endoscopic features (microstructural patterns, vascular abnormalities), and clinical parameters (tumor location). Step 2: Macroscopic observation layer: Visual identification of gastrointestinal lesion regions to facilitate subsequent artificial intelligence (AI) processing. Step 3: AI model processing layer: Gastric cancer analysis: Multimodal model incorporating ulcer-free status, high differentiation, and proximal tumor location [negative predictive value (NPV) = 100%] supports function-preserving surgical decisions. Colorectal cancer analysis: Vision Transformer-based model (ViTCol, NPV = 96%) combines 7 indicators including age and tumor location for precise lymph node metastasis risk stratification. Step 4: Clinical decision layer: AI-generated risk stratification guides therapeutic recommendations: Surgical resection for high-risk cases and endoscopic resection for low-risk cases. ctDNA: Circulating tumor DNA; MSI: Microsatellite instability; AI: Artificial intelligence; NPV: Negative predictive value; EGC: Early gastric cancer; NPV: Negative predictive value; ViT: Vision Transformer; LNM: Lymph node metastasis.

In the field of colorectal cancer, AI technology has covered early cancer identification and LNM risk stratification. ViT-based models (such as ViTCol proposed by Ayana et al[27]) had an AUC value of 0.9999 in detecting early lesions[28], while the AI prediction system for T2 colorectal cancer achieved LNM risk stratification through seven indicators including age, tumor location, lymphovascular invasion, and so on. The NPV is up to 96%, which could effectively avoid excessive surgery[29]. Although the positive predictive value was low (25.7%), its high sensitivity (97.8%) provided a guarantee for safe follow-up[29]. The process is shown in Figure 1.

The clinical translation of AI technology in endoscopic analysis of gastrointestinal cancer still faces multi-dimensional challenges. At the data level, the model's generalization is limited by sample homogeneity. For example, gastric cancer research is mostly based on Asian populations, and colorectal cancer validation data is concentrated in a single center in Europe and the United States). Therefore, it is necessary to build a standardized database through global multicenter collaboration and use transfer learning to adapt to regional differences. At the technical level, there is an urgent need for breakthroughs in real-time processing and interpretability: Millisecond delays in high-definition video processing may interfere with operational fluency, and "black box" decision logic weakens clinical trust. In the future, lightweight architectures (e.g., model pruning) should be developed to reduce delays to within 50 milliseconds, and visualization tools such as heat maps and feature contribution rankings should be used to enhance the transparency of decision-making. At the clinical integration level, the existing system is insufficient for the fusion of multimodal data. Taking LNM prediction as an example, over-reliance on endoscopic images and pathological indicators can easily lead to misjudgment. It is necessary to integrate radiomics (e.g., MRI texture features) and molecular detection (e.g., ctDNA) to build a three-dimensional evaluation system and improve risk stratification accuracy.

AI quantitative analysis of ultrasound/CT/MRI imaging: Deep learning models based on CT and MRI have shown significant advantages in gastric cancer diagnosis and treatment. A multi-center deep learning-based radiomics nomogram developed by Cui et al[7] can accurately predict the response of locally advanced gastric cancer patients to neoadjuvant chemotherapy (NACT) (AUC 0.829) and correlate with disease-free survival (DFS) by integrating CT imaging features and clinical parameters. It can provide a basis for individualized treatment decisions. Zheng et al[30] further proposed a 3D CT model based on Transformer to use preprocessed images to predict early postoperative LNM (AUC = 0.804), which had significantly better performance than the traditional clinical model (P < 0.001) and effectively avoided overtreatment. These techniques make up for the subjectivity and low sensitivity of traditional imaging diagnosis through automatic feature extraction and multimodal data fusion.

In the field of colorectal cancer, AI technology has significantly improved the prediction accuracy of LNM and prognosis through the combination of radiomics and deep learning. A meta-analysis by Bedrikovetski et al[31] showed that the AUC of AI models in diagnosing preoperative LNM of colorectal cancer (0.808-0.917) was much better than that of radiologists (AUC = 0.688), especially for rectal cancer. Abbaspour et al[8] validated the generalizability of the radiomics model (AUC = 0.814) in colorectal cancer (CRC), and a model incorporating peritumoral positron emission tomography (PET)/CT features further improved prediction sensitivity to 77%[8,32]. Li et al[33] constructed a prognostic model by combining CT deep learning features of primary lesions and lymph nodes (C-index of DFS prediction = 0.73) to support the stratified treatment of stage II patients.

There are still three shortcomings in current AI imaging technology for gastrointestinal cancer. The first is the single data source. Most models are trained on data from a single hospital or region (such as Chinese cohort studies[7,30]), which limits the ability of cross-population generalization, especially for European, American or African patients. Second, the "black box" nature of the model is a serious concern. Although AI can extract image features (such as PET/CT peritumoral texture[32]) with high efficiency, the correlation between these features and tumor biological behavior (such as gene mutations or immune microenvironment) is not clear, and clinicians have difficulty trusting its decision logic. Third, technology application is fragmented. Most existing studies isolated and analyzed a single modality image (such as CT or endoscopy), lacking collaborative integration with ultrasound, MRI, and liquid biopsy. Few models were adapted to the clinical workflow, such as real-time endoscopic assistance system).

To solve the above problems, three breakthroughs are needed in the future: (1) To build an inter-ethnic multi-center database and integrate image data from different devices through FL technology to improve model robustness; (2) Developing interpretable AI frameworks, such as using attention heatmaps to label high-risk areas of gastric cancer mucosa or correlating radiomics features with specific molecular pathways (such as EGFR signaling) to enhance the confidence of the results[7]; and (3) Promoting multimodal clinical integration, such as integrating the ViT model of colorectal cancer with ultrasound elastography or designing lightweight AI plug-ins directly into the endoscopy/PACS system to realize the automation of the whole chain of "image acquisition, analysis, and report"[8].

AI-assisted diagnosis of pathological sections: Recent deep learning-based AI technology has shown significant advantages in the analysis of pathological sections of gastric cancer. By integrating multi-scale pathological image features, the AI model can effectively identify molecular subtypes of gastric cancer (such as Epstein-Barr virus, MSI, GS, CIN), and its predictive AUC values can reach 0.81-0.89[34]. For example, the DEMoS model predicts MSI status directly from H&E sections through an ensemble learning strategy, with a sensitivity of 0.86 and a specificity of 0.83[35], respectively. In addition, AI can provide a new perspective for prognostic stratification by analyzing immune microenvironment subtypes (such as immune-inflammatory types) through whole-section images (WSI)[36]. These technologies significantly reduce the time and cost of traditional molecular testing and promote precision treatment decisions.

In the field of colorectal cancer, AI technology has also made breakthroughs. The WSI-based deep learning model can directly predict MSI status from H&E-stained sections, and the sensitivity and specificity are close to those of traditional PCR detection[12]. For example, the model integrating SNP and CIMP genomic features improved the AUC of CRC subtype classification to 0.847[37]. AI can also reveal subtype-specific biomarkers by analyzing the association between tumor cell morphology and genomic variants (such as CpG island methylation), providing a new tool for non-invasive screening[38]. Such technologies are especially suitable for resource-limited areas and are expected to replace some laboratory tests.

At present, AI technology faces three bottlenecks in the pathological diagnosis of gastrointestinal cancer: Lack of data standardization (staining differences between institutions weaken the generalizability of models), high labeling cost (relying on gold standard detection to limit the scale of training), and insufficient decision-making transparency ("black box" model reduces clinical trust). Future optimization should focus on four aspects: (1) Cross-institutional collaboration: Building unified staining standards and shared databases and using FL to break data silos; (2) Weakly supervised learning: Reduce the dependence on expensive detection by pre-training unlabeled WSI and fine-tuning with a small amount of labeling; (3) Multimodal fusion: Combining pathological images, genomic data (such as MSI-related mutations) and clinical information to improve the robustness of prediction; and (4) Enhanced interpretability: Developing heat map tools (such as Grad-CAM) to visually display AI focus areas and assist pathological verification. In addition, prospective clinical trials should be performed to verify the efficacy and safety of AI in real scenarios and promote its transformation into clinical practice.

AI technology for biomarker identification: At present, AI technology has shown multi-dimensional breakthroughs in the discovery of gastric cancer biomarkers. The miRNA screening model based on support vector machine (SVM) accurately identified four survival-related miRNAs, such as hsa-miR-21, from 29 candidate markers[39], and its diagnostic accuracy reached 93%. Machine learning-integrated multi-omics data (such as methylation and metabolomics) can overcome the limitations of single omics approaches. For example, the AUC of DNA methylation markers was 0.99[40], and the plasma metabolic fingerprint was validated using 21 metabolites across multiple centers[41]. AI has achieved multimodal breakthroughs in the field of CRC, from pathology to serum. A Transformer-based deep learning model can predict MSI directly from tissue sections with a sensitivity of 99%[3], addressing the timeliness issue of traditional detection methods. A surface-enhanced Raman scattering (SERS) endoscopic system combined with a random forest algorithm achieved 96.67% cancer classification accuracy through label-free metabolite detection[22]. Serum protein marker ensemble learning models (such as BH-index) use 17 protein features to improve the accuracy of early diagnosis to 98.03%[42], which is significantly better than the single biomarker strategy.

Existing AI techniques face three bottlenecks in the identification of gastrointestinal cancer markers. First, data barriers limit the generalizability of models. Most studies rely on data from a single institution (such as the TCGA database for methylation studies), which leads to performance degradation during cross-population validation. Therefore, it is necessary to establish a standardized multi-center biobank and clinical data sharing mechanism. Secondly, the interpretability of the algorithm hinders clinical translation. Despite the excellent performance of models such as Transformer in pathological prediction, their decision logic is difficult to trace. The trust between doctors and patients can be improved by embedding attention heatmap visualization (e.g., labeling MSI-related tissue regions) or developing lightweight models with biological constraints. Finally, the fusion of multi-dimensional information is insufficient. Existing systems often analyze genomic, proteomic, or metabolomic data in isolation and fail to simulate real biological network interactions. Therefore, it is necessary to develop cross-omics coupling architectures, such as dynamically linking miRNA regulatory networks with metabolic pathways and using graph neural networks to capture molecular cascades. Hardware co-innovation is also critical, such as the batch stability of SERS probes and the miniaturization of endoscopic equipment, to realize the smooth transition from laboratory results to clinical scenarios. Future breakthroughs need to integrate three key directions: Building a global data ecosystem, creating an interpretable AI framework, and developing multimodal models driven by biological networks, ultimately forming a closed loop of "data-algorithm-clinical practice".

Multi-omics data integration: By integrating genomic mutations, such as somatic mutations and copy number variations, and immune microenvironment data, AI technology has significantly improved the accuracy of molecular classification and prognosis prediction of gastric cancer. For example, machine learning models (e.g., random forests) can combine transcriptome, epitranscriptome, and mutation data to construct programmed death signatures and analyze their association with immune cell infiltration in the tumor microenvironment (TME)[43]. Single-cell multi-omics combined with AI further reveals the spatial heterogeneity of immunogenic cell death-related genes (e.g., LBH) that affect the TME by regulating epithelial-mesenchymal transition[44]. In addition, molecular subtyping based on multi-omics can predict the expression level of immune checkpoints and provide a basis for personalized immunotherapy[45]. These methods overcome the limitations of traditional pathological analysis through high-throughput data integration, enabling a leap from "descriptive analysis" to "mechanistically-driven modeling".

In CRC, AI-driven multi-omics integration technology has deepened the understanding of the immune microenvironment through spatial omics and proteomics. For example, spatial transcriptomics combined with deep learning can locate the distribution of functional states of immune cells within the tumor and accurately predict the response to PD-1 inhibitors. The DEPRS scoring system based on the iTRAQ proteome and transcriptome can effectively distinguish between immunosuppressive and immune-activating microenvironments[46] and screen out key regulatory genes (such as HART3)[47]. In addition, single-cell multi-omics analysis combined with unsupervised learning revealed the gene regulatory network driven by copy number variation, providing new insights into the mutation-microenvironment interaction mechanism[48]. AI technology not only improves the efficiency of multi-omics data analysis but also identifies potential therapeutic targets through cross-omics association.

Current AI technologies face three bottlenecks in multi-omics integration: (1) Insufficient data compatibility: Significant differences in data sources such as genomics, transcriptomics, and spatial omics, coupled with strong platform specificity (e.g., dimensional differences between single-cell sequencing and bulk sequencing), limit the generalizability of models across datasets. For example, the correlation between single-cell data and clinicopathological features in gastric cancer research is often weakened by batch effect; (2) Ambiguous mechanistic interpretation: Most deep learning models (such as CNNs) rely on black-box operations, which makes it difficult to trace the contribution weight of key mutations or immune cell subsets to the prediction results, hindering target discovery; and (3) Weak clinical practicality: Existing models are primarily based on retrospective data, lack prospective validation, and do not account for dynamic treatment responses (e.g., microenvironment remodeling after chemotherapy). In the future, technology integration can be a breakthrough by developing an adaptive data standardization framework (such as cross-omics embedding alignment) and combining graph neural networks to uniformly process heterogeneous data. Interpretable enhancement: Introducing causal inference models (e.g., Bayesian networks) to resolve mutation-TME interaction pathways instead of relying solely on correlation analysis.

Management of functional gastrointestinal disorders

Early NLP techniques have significant limitations in PRO analysis. First, traditional models rely on manual annotation and structured data entry, which makes it difficult to efficiently process complex unstructured text (such as vague symptom descriptions and emotional expressions) in interviews with patients with functional gastrointestinal disorders, such as irritable bowel syndrome (IBS)][49]. Secondly, the generalizability of the model is insufficient, and the algorithms developed for different diseases, such as cancer, are difficult to transfer to the field of gastroenterology, resulting in low classification accuracy[49]. In addition, the lack of multimodal data fusion capability (e.g., integrating symptom text with physiological indicators) limits the application of PRO in individualized treatment[1].

The new generation of NLP technology has significantly improved the efficiency and depth of PRO analysis. Transformer-based models, such as BERT, can automatically identify key symptoms, such as abdominal pain and bloating, and their semantic patterns related to quality of life in patient texts with functional gastrointestinal disorders through the self-attention mechanism[49]. Multimodal fusion technology further integrates clinical text, laboratory data and imaging reports to support cross-dimensional PRO analysis, such as the association between symptom severity and intestinal flora)[1]. In addition, transfer learning enables models to be pre-trained on cancer or respiratory disease data and fine-tuned to gastrointestinal disease studies with small sample sizes to reduce data acquisition costs[1].

There are still multiple bottlenecks in current NLP technology management of gastroenterology PRO. At the data level, patients' descriptions of symptoms often show ambiguity and individual differences. For example, "abdominal distension" may correspond to multiple etiologies. However, existing models mostly rely on training with general medical corpora and lack a dedicated database for gastrointestinal disease terms and expressions, leading to missing key details during classification. At the algorithm level, most models are poorly adapted to cultural background and language style (e.g., dialect or unstructured spoken language input), and it is difficult to capture the complex association between symptoms and psychological factors (e.g., anxiety aggravating IBS), which limits the reliability of cross-population application. At the functional level, existing tools focus on static text analysis, which cannot respond to patients' dynamic symptom changes in real time, and it is difficult to deeply interact with the clinical decision-making system (such as automatic treatment recommendation based on PRO).

Improvements need to be made in three aspects: First, an exclusive corpus of gastrointestinal diseases should be constructed to integrate diversified symptom expressions through crowdsourcing and active learning; The second is to design a hybrid architecture model, embed domain knowledge graphs (e.g., Rome IV diagnostic criteria) in the Transformer, and introduce adversarial training to reduce cultural bias. The third is to develop a closed-loop PRO management system that combines time-series analysis, such as symptom fluctuation patterns, with reinforcement learning to realize an automated link from data interpretation to intervention recommendations. In addition, experience with multimodal fusion in oncology (e.g., text + biomarkers) can be drawn upon to explore association modeling between gastrointestinal disease symptoms and microbiome or metabolic data, thereby enhancing the pathological interpretability of PRO.

Decision support for the treatment of gastrointestinal diseases

AI-driven drug development: The core advantage of AI technology in the treatment of gastric cancer lies in the integration of multimodal data to construct accurate prognostic models. Deep learning models based on non-apoptotic regulated cell death genes can integrate transcriptome data and digital pathological images to predict response to immunotherapy[50], significantly improving the accuracy of patient stratification. In addition, foundational models for cross-scale pathological image analysis, such as the AI framework developed by Wang et al[6], can identify biomarkers predictive of chemotherapy benefit from histological features to guide adjuvant treatment decisions. These technologies break through the limitations of traditional biomarkers and provide dynamic evaluation tools for the development of individualized drugs for gastric cancer.

The advantages of AI models for colorectal cancer focus on multimodal data fusion and target discovery. For example, a deep radiomics model based on PET/CT and clinical data can predict the efficacy of bevacizumab in the treatment of liver metastases[51], while graph neural network (Gra-CRC-miRTar) analyzes the miRNA-mRNA interaction network through a pre-trained framework to reveal potential therapeutic targets[52]. Multi-stain deep learning models, such as AImmunoscore, further integrate H&E and immunohistochemistry images to quantify the tumor immune microenvironment and predict the response to neoadjuvant therapy[10], providing new ideas for the optimization of drug combination strategy.

There are three common bottlenecks in AI-based research and development of drugs for gastrointestinal cancer. First, the dimension of data integration is insufficient: Most models rely on single-center and single-modality data (e.g., only pathology or transcriptome data), with limited cross-institutional and cross-platform generalization ability. Digital pathology models are particularly sensitive to differences in staining standards, leading to prediction bias. Second, the biological interpretability is weak: The targets or markers screened by AI (e.g., miRNA interaction network) lack in vitro experiments or preclinical model validation, making it difficult to clarify the mechanism and affecting the credibility of research and transformation. Third, the lack of dynamic adaptability: Existing models are mostly constructed based on static data prior to treatment, making them unable to track TME evolution in real time, such as the changes of immune cell infiltration after chemotherapy, resulting in delayed prediction of response during treatment.

To solve the above problems, we can iteratively optimize the model from three aspects: (1) Build a multi-center standardized database, unify the data acquisition protocol (such as pathological staining and sequencing process), and develop transfer learning algorithms to enhance the adaptability of the model across datasets; (2) Strengthen the "dry and wet combination" verification, combine AI prediction targets with organoid drug susceptibility testing, single-cell sequencing, etc., and establish an interpretable biomarker closed-loop verification system; and (3) Embed a dynamic monitoring module to integrate real-time data streams (e.g., liquid biopsy and radiomics), capture treatment response trajectories via time-series models, and dynamically adjust drug combination regimens. This optimization path requires the collaboration of computational biologists and clinical teams to promote the transformation of AI from an auxiliary tool to a driving research and development engine.

Prediction of AI response to immunotherapy in gastroenterology: AI has shown significant advantages in the prediction of treatment response in gastric cancer. By integrating genomic, radiomic and pathological data, multimodal deep learning models can accurately predict the response to anti-HER2 treatment (such as a 30% increase in the objective response rate of HER2-positive subtype patients) and the efficacy of combined immunotherapy[13]. Spatial multi-omics technology combined with AI can analyze the TME heterogeneity, identify the drug resistance characteristics of chemoimmunotherapy (such as high CXCL12 expression in the stroma), and guide individualized drug use[53]. In addition, the deep learning model based on H&E slides could noninvasively predict EBV status and PD-L1 expression (AUC = 0.89), which could replace some molecular tests[54].

In the field of colorectal cancer, AI has significantly improved the accuracy of treatment prediction through the integration of multi-omics. For example, the Colorectal Cancer Immune Module, which combines single-cell sequencing and deep learning to target FOLR2+ macrophages, improved chemotherapy sensitivity (25% increase in response rate)[55]. AI-assisted liquid biopsy, such as dynamic ctDNA monitoring, can predict recurrence risk after adjuvant therapy (HR = 0.42) and guide adjustment of treatment intensity[14]. For immunotherapy, TME subclass-based machine learning models (e.g., dendritic cell infiltration score) can stratify patient response rates with ORR differences of up to 40%[56], while biomimetic microrobots can reverse the immunosuppressive microenvironment (35% increase in tumor shrinkage rate) through AI-regulated lactate metabolism[57]. There are still three bottlenecks in current AI technologies for predicting treatment response in gastrointestinal cancer. First, data limitations: Most models rely on single-center, small-sample retrospective data (such as subtype analysis based on H&E images) and lack cross-ethnic validation, which limits generalization ability. In the future, it is necessary to promote the construction of multi-center, multi-modal data sharing platforms and integrate spatial multi-omics and microbial dynamic data to improve the generalization of predictions. Second, the interpretability of models is insufficient: Most predictions driven by genomic mutational signatures or deep learning are "black box" models, and it is difficult for clinicians to understand the mechanism of key biomarkers. Incorporating an attention mechanism to visualize feature weights, such as mapping resistance-associated microbes or specific TME regions, enhances confidence in the results. Third, the lack of dynamic monitoring: Most existing studies are based on static data before treatment, and dynamic information such as tumor evolution and microbiome fluctuation during treatment has not been fully explored. The development of real-time monitoring AI models combining liquid biopsy (such as ctDNA) and radiomics, and the use of transfer learning to adapt the individualized treatment trajectory will become a breakthrough direction. Through the above optimization, AI technology is expected to be upgraded from a "prediction tool" to a "decision support system” and promote the development of precision treatment for gastrointestinal cancer.

AI predicts the recurrence of gastrointestinal cancer after treatment: A multi-task deep learning model based on CT images has significantly improved the prediction performance of postoperative recurrence of gastric cancer. By integrating preoperative CT features and clinical data, the model developed by Jiang et al[5] could specifically predict peritoneal recurrence (the main cause of death in gastric cancer) and estimate DFS, with an AUC of 0.85 in the independent validation cohort, which was superior to the traditional tumor-node-metastasis staging. Multi-center studies further verified the generalization ability of such models in heterogeneous data. For example, the DLFS model developed by Cao et al[58] achieved a C-index of 0.74 across five medical centers and could identify chemosensitive subgroups. In addition, tumor-associated collagen signatures based on multi-photon imaging combined with machine learning can reveal the relationship between the TME and recurrence and guide individualized chemotherapy selection[59]. The introduction of FL techniques (e.g., research by Feng et al[19]) can protect data privacy and enable collaborative training of multi-center models, thereby addressing the issue of isolated gastric cancer data.

The AI prediction model of colorectal cancer promotes accurate stratification through multi-omics integration. A deep-learning model trained by Xiao et al[60] used histopathological slides (DL-RRS) to predict 5-year recurrence risk with an AUC of 0.83 and identified micrometastases overlooked by traditional pathology. For patients with liver metastases, the DL-C model developed by Zhao et al[61], which combines contrast-enhanced ultrasound (CEUS) and deep learning, had an AUC of 0.78 for predicting early recurrence after ablation, significantly outperforming clinical scores. AI has also optimized treatment decisions: Yang et al[14] found that ctDNA combined with machine learning can screen the population who would benefit from chemotherapy and avoid overtreatment. The neoadjuvant therapy strategy proposed by Kagawa et al[62] used AI to screen patients at low risk of recurrence and is expected to enable non-surgical treatment of rectal cancer. Although AI models have shown potential in predicting recurrence of gastrointestinal cancer, there are still key bottlenecks in their application. First, insufficient data standardization has affected the generalization ability of models: Imaging scanning parameters (e.g., slice thickness, contrast agent dose) and pathological preparation processes (e.g., staining batches) vary significantly among medical institutions, leading to performance fluctuations of the same model across centers (e.g., the inter-center AUC of the gastric cancer CT model decreased by 0.12). Solutions include establishing standardized guidelines for scanning protocols and automatically aligning data distribution differences through Domain Adaptation techniques. Second, the models were disconnected from biological mechanisms: Existing AI primarily relies on morphological features of imaging/pathology, but the CLDN18.2 protein pathway may be involved in gastric cancer peritoneal metastasis, and KRAS mutation is associated with liver metastasis recurrence of colorectal cancer. These molecular mechanisms have not been effectively integrated. In the future, a multi-omics fusion framework should be developed. One example is combining CT imaging features with ctDNA-detected gene mutation spectra to capture recurrence drivers from dual molecular-imaging dimensions. Finally, there is a lack of real-time validation scenarios in clinical practice: Current studies are primarily based on retrospective data, which cannot reflect model efficacy in dynamic treatment decision-making (such as recurrence risk reassessment after NACT). Prospective clinical trials should be designed to embed AI predictions into treatment pathways. For example, after colorectal cancer liver metastasis ablation, an AI model based on CEUS images could dynamically assess recurrence risk monthly to guide the timing of secondary intervention. In addition, the development of visual interpretation tools, such as feature activation heat maps, can enhance doctors' trust in AI results and avoid "black box" doubts.

INNOVATION OF MULTIMODAL AI TECHNOLOGY IN HEPATOLOGY

Diagnosis and staging of liver diseases

AI quantitative analysis of ultrasound/CT/MRI imaging: Medical imaged-based AI technology has significantly improved liver fibrosis staging objectivity and efficiency[63]. For example, combining ultrasound deep learning models such as FIB-Net with shear wave elastography can enable advanced fibrosis screening with a specificity of up to 80% while avoiding liver biopsy[9]. Non-enhanced MRI (T1WI/T2FS) combined with a 3D Transformer network extracted liver fibrosis imaging features through automatic segmentation, and the AUC of external validation was 0.89, which was significantly better than that of traditional serum markers[41]. In addition, the combination of MRI radiomics and clinical data (CoRC model) could accurately distinguish ≥ F2 fibrosis (AUC = 0.85), reducing the dependence on invasive examinations[15]. These techniques provide new tools for dynamic monitoring of fibrosis progression by quantifying imaging features. The process is shown in Figure 2.

Figure 2 Integration of artificial intelligence-assisted diagnostic models and imaging techniques for liver disease evaluation. This flowchart illustrates the risk stratification and multimodal imaging workflow for liver disease diagnosis. The PLAN-B-DF model (gradient boosting algorithm) identifies high-risk populations of hepatocellular carcinoma (HCC) in chronic hepatitis B patients by analyzing computed tomography (CT)-derived biomarkers (e.g., visceral fat distribution, spleen volume) and clinical data, achieving a C-index of 0.91. ModelURC stratifies patients into low-risk and high-risk groups for differential surveillance protocols. For complex cases (e.g., sub-3 cm lesions or atypical imaging features), contrast-enhanced ultrasound combined with CT/magnetic resonance imaging LI-RADS classification enhances diagnostic sensitivity to 88.8%, minimizing unnecessary biopsies. The framework integrates artificial intelligence-driven quantitative analysis (e.g., deep learning-based fibrosis staging, radiomics) with multimodal imaging to optimize precision in liver fibrosis grading, HCC screening, and longitudinal monitoring. CEUS: Contrast-enhanced ultrasound; CT: Computed tomography; MRI: Magnetic resonance imaging.

AI has shown high sensitivity and specificity in multimodal liver cancer screening. The deep learning model (LiLNet) based on multi-phase CT can automatically identify HCC, cholangiocarcinoma and other lesions with an accuracy of 94.7%, which is especially suitable for resource-poor areas[64]. Ultrasound radiomics combined with the XGBoost algorithm can improve the diagnostic AUC of small liver cancer ≤ 3 cm to 0.899, exceeding manual image reading[24]. The combination of CT imaging biomarkers, such as visceral fat distribution and spleen volume, and gradient boosting model (PLAN-B-DF) had a C-index of 0.91 for predicting HCC risk in patients with chronic hepatitis B, which was significantly better than the traditional model[65]. In addition, CEUS combined with CT/MRI LI-RADS classification can improve the diagnostic sensitivity of HCC to 88.8% and reduce unnecessary biopsies[66]. The process is shown in Figure 2.

At present, AI technology still faces four challenges in liver disease diagnosis. First, the generalization of the model is limited. Most algorithms are trained on data from a single institution or specific equipment and are prone to performance degradation due to differences in imaging parameters when deployed across hospitals or devices. It is necessary to build a standardized multi-center and multi-vendor image database and introduce domain adaptation technology to improve model robustness. Second, the data quality is highly dependent and variable. Low-resolution images, such as primary hospital ultrasound, or patient respiratory motion artifacts can significantly interfere with analysis results. Fuzzy images can be enhanced and repaired by generative adversarial networks (GAN), or motion correction algorithms can be developed to optimize the data preprocessing process. Third, the transparency of decision-making is insufficient. The "black-box" nature of deep-learning models hinders clinical trust, and it is difficult for doctors to understand why an AI algorithm reaches a particular conclusion. Interpretable techniques, such as feature heatmap and decision tree rule extraction, need to be integrated to visualize the key diagnostic evidence and verify its correlation with pathological results. The fourth is the separation of clinical application scenarios. Most existing AI systems are independent tools that are not deeply integrated with electronic medical records and image archiving systems. In the future, embedded AI modules should be developed to realize real-time examination analysis, automatic report generation, and dynamically update the model through continuous learning mechanisms to adapt to the changes in disease spectrum.

Blood test indicators and image data fusion modeling: AI models for early-stage liver diseases primarily rely on a single data source and lack cross-modal fusion capabilities. For example, single-photon emission computed tomography (SPECT)/CT imaging-based liver function assessment only integrates quantitative imaging parameters (e.g., LHL15 and HH15) and basic clinical information, without incorporating serum markers[67]. Although blood indicator-based models (e.g., N-glycan profiles) can assess the chronic hepatitis B inflammation stage, they fail to integrate imaging features to verify liver structural changes[68]. This fragmentation leads to models that only reflect local pathological features and lack a comprehensive understanding of liver disease heterogeneity, such as the coupling mechanism of metabolism and blood flow in the progression from metabolic dysfunction-associated fatty liver disease to cardiovascular disease[69]. Additionally, early algorithms like SVM have limited capacity to integrate high-dimensional multimodal data and are prone to overfitting due to limited sample sizes[70].

The new generation of AI technology has significantly improved the accuracy of liver disease assessment through cross-modal data fusion. Zhu et al[70] integrated radiomics features from gadoxetic acid-enhanced MRI and CT with the blood indicator ICG-R15 to construct an XGBoost model, achieving an AUC of 0.96 for predicting preoperative liver function reserve. A systematic review demonstrated that deep learning can achieve non-invasive staging of non-alcoholic fatty liver disease fibrosis by utilizing ultrasound elastography texture features combined with serum indicators, such as alanine aminotransferase and aspartate aminotransferase (accuracy > 85%)[69]. Such models overcome the limitations of single-modality analysis and dynamically weight blood and imaging features (e.g., the temporal correlation between portal vein blood flow velocity and platelet count) via the attention mechanism to reveal compensatory mechanisms of liver diseases. Furthermore, transfer learning enables the reuse of small samples from multi-center data. For example, a random forest model combining SPECT/CT imaging and clinical parameters maintained 90% specificity in cross-center validation[67].

Current technology still faces three major challenges: (1) Data heterogeneity: Feature drift caused by variations in imaging devices (e.g., 1.5T vs 3.0T MRI) and blood detection standards, necessitating the establishment of a cross-platform calibration protocol; (2) Model "black-box" issue: Deep learning feature weights are difficult to map to biological mechanisms, requiring the introduction of interpretable architectures (e.g., Graph Transformer) to analyze key associations (e.g., liver sinusoidal capillarization and serum VEGF levels); and (3) Lack of dynamic monitoring: Existing models are mostly based on cross-sectional data and should be combined with temporal modeling (e.g., LSTM) to capture treatment responses, such as synergistic changes in N-glycan profiles and liver elasticity values during antiviral therapy. In the future, FL can be used to integrate multimodal data streams and develop interactive visualization tools to aid clinical decision-making.

AI-assisted diagnosis of pathological sections: The pathological diagnosis of early liver cancer is highly dependent on manual slide review, which is subject to significant subjective bias and efficiency bottlenecks[2]. Traditional AI models rely on manual feature extraction and have limited ability to identify molecular features like tissue heterogeneity and MSI, and have poor generalization, with variations in staining across institutions causing fluctuations in performance[71]. Furthermore, early algorithms cannot integrate multimodal clinical-pathological data, making it difficult to achieve accurate diagnosis[72].

Current deep learning techniques can directly extract subvisual features from H&E-stained sections through end-to-end learning[72]. The HEPNET model achieved 98.1% accuracy in differentiating intrahepatic cholangiocarcinoma from colorectal liver metastases, significantly outperforming human pathologists[73]. NucleiSegNet, an attention mechanism-based algorithm, optimizes nuclear segmentation accuracy (F1 = 0.94) and provides reliable cell-level features for MSI analysis[71]. Multi-task learning frameworks can simultaneously achieve tumor classification, MSI status prediction, and prognosis assessment. For instance, the CSGO system enhances cell segmentation specificity through global optimization[74].

Despite the significant progress in AI technology, there are still multiple bottlenecks in its practical application. First, existing models over-rely on labeled data, which requires substantial manpower to train personnel for labeling high-quality pathological images. Notably, the scarcity of samples for rare liver cancer subtypes may result in model bias. To address this issue, self-supervised learning technologies should be explored in the future with pre-training tasks (e.g., image puzzle reconstruction and rotation prediction) designed to reduce reliance on manual labeling. Secondly, the lack of cross-institutional generalization ability remains a critical issue, as variations in staining protocols across medical centers significantly reduce algorithm accuracy. Improvement strategies include developing dynamic color normalization modules, automatically calibrating color spaces during preprocessing, or introducing domain-adaptive training strategies to enhance robustness. Finally, the lack of interpretability in AI decision-making processes limits clinical trust. Thus, it is necessary to integrate attention heatmaps with pathological prior knowledge to construct a visual reasoning pathway. For example, when predicting microsatellite status, regions in mitotic images with high contribution can be synchronously labeled to help pathologists understand the model's logic. Future breakthroughs will involve constructing a multimodal joint learning framework, integrating genomic and proteomic data to enhance prediction specificity, and establishing real-time feedback mechanisms for dynamic model optimization.

Multi-omics data integration: Traditional liver cancer research primarily relies on a single type of omics data (e.g., genomics or transcriptomics), making it difficult to comprehensively analyze tumor heterogeneity. For example, early prediction models based on gene mutations (e.g., p53 and CTNNB1) lack integration of immune microenvironment dynamics, leading to inaccurate prognosis assessment[75]. Additionally, the isolation of radiomics features from molecular data limits the interpretation of TME spatial characteristics[76]. Prediction of response to immunotherapy is also inadequate because spatial distribution features such as peritumoral TLS are overlooked[17].

In recent years, multi-omics techniques (e.g., spatial transcriptomics and radiomics) have significantly improved liver cancer diagnosis accuracy. For example, integrating MRI radiomics with TLS spatial distribution data enables non-invasive prediction of response to immunotherapy (AUC = 0.91)[17]. Single-cell transcriptomics combined with machine learning reveals MYC pathway activation and MIF signaling-mediated cell interactions in MVI-associated malignant cells, potentially identifying new targets for therapy[23]. Furthermore, by integrating epigenomic and transcriptomic data, denoising network algorithms like DeFusion have identified survival-related HCC subtypes (C-index = 0.78)[77].

Although multi-omics integration has shown great potential in liver cancer research, current AI technologies still have notable limitations. First, data noise is a significant issue: Batch effects from different omics platforms (e.g., transcriptomics and epigenomics) can interfere with key signal extraction, and existing denoising methods (e.g., non-negative matrix factorization) are not sufficiently adaptive to complex noise. Secondly, models have limited ability to capture dynamic changes in the TME. Most studies still use static samples, making it difficult to reflect real-time interactions between immune cells and tumor cells during treatment. Additionally, there are challenges in clinical practicality: MRI-based radiomics models are difficult to implement in primary hospitals, and the prediction accuracy of alternative methods (e.g., ultrasound) needs improvement. To overcome these limitations, future research can be enhanced in three areas: (1) Developing adaptive denoising algorithms combined with graph neural networks to capture deep correlations in multi-omics data; (2) Introducing time-series analysis techniques to monitor microenvironment evolution via liquid biopsy or continuous imaging; and (3) Optimizing lightweight model architectures to enable high-precision prediction even with widely used equipment such as ultrasound. These improvements need to be verified by multi-center clinical trials before they can truly promote technology implementation.

Decision support for liver disease treatment

AI system for donor matching in liver transplantation: At present, the core advantages of AI in liver transplantation donor immune matching are reflected in the improved accuracy and efficiency. By integrating the human leukocyte antigen (HLA) spectrum and antibody characteristics of donors and recipients, AI can quickly predict cross-matching compatibility and significantly reduce rejection risk[78]. For example, by analyzing HLA antibody profiles and donor-specific antibodies, the machine learning model achieved a receiver operating characteristic-AUC value of 0.975 in predicting immune response after transplantation, which was far superior to traditional statistical methods[78]. In addition, the introduction of gene editing technology further optimized the immune compatibility of xenotransplantation. For example, knocking out porcine hyperacute rejection genes (GGTA1, B4GALNT2) and inserting human complement regulatory protein genes (CD46, CD55) decreased the rate of rejection after liver xenotransplantation. These studies confirmed the potential of AI in multi-dimensional immune regulation[79].

AI exemplifies the advantages of dynamic risk assessment and personalized management in the prediction of postoperative complications. The prediction model based on Model for End-Stage Liver Disease score, psoas major thickness/height ratio and nutritional status score can stratify and evaluate the risk of postoperative complications at 90 days after surgery (as low as 3.6% and as high as 75.5%), and its clinical practicability can be verified by decision curve[80]. In addition, AI has outstanding performance in imaging and multi-omics data integration, such as predicting the risk of liver failure through radiomics analysis of tumor characteristics or identifying immunosuppression-related hepatotoxicity combined with metabolomics, which provides a basis for precise intervention in postoperative management[79].

At present, the application of AI technology in the field of liver transplantation faces multiple challenges. Most models rely on retrospective data from a single center and small sample numbers. For example, Giglio et al[80] only verified 234 samples. This lack of power leads to insufficient generalization ability, and the model is difficult to adapt to different populations or medical scenarios. The existing algorithms are primarily based on static parameters at fixed time points, which cannot integrate dynamic indicators (such as fluctuations in drug concentration and changes in immune status) in real time, limiting the timeliness of early risk warning. In addition, the depth of cross-species research is insufficient, and the observation period of xenograft experiments is too short. For example, the pig-human model is only followed up for 10 days, which fails to reveal the molecular mechanism of long-term rejection and limits the optimization space of gene editing strategies. Future improvements can be made in three aspects: (1) Dynamic data network construction: Joint centers to establish standardized databases, incorporating more genomic, metabolic, and continuous monitoring data, and developing adaptively updated prediction models; (2) Interdisciplinary mechanism mining: Combining single-cell sequencing and AI simulation to analyze the spatiotemporal dynamics of donor-recipient immune interactions to guide targeted gene editing (such as optimizing the expression of complement regulatory proteins); and (3) Closed-loop design for clinical validation: Extending the observation period of xenotransplantation to several months and conducting prospective randomized trials to verify model robustness in real scenarios.

Prediction of AI immunotherapy response in liver disease: In the past, the prediction of immune response to liver cancer immunotherapy primarily relied on a single clinical marker, such as alpha-fetoprotein (AFP), or limited molecular features, and there was a lack of systematic evaluation of TME heterogeneity[81]. Traditional molecular classification failed to integrate single-cell resolution data, leading to deviations in immune checkpoint inhibitor sensitivity prediction[82]. In addition, biopsy-based prediction models cannot dynamically reflect molecular changes during treatment, and invasive detection limits clinical application[83].

Current AI techniques have significantly improved prediction accuracy by integrating multi-omics and imaging data. For example, the transcriptome-based DNA damage repair (DDR) score model can predict treatment resistance and overall survival in patients receiving anti-PD-1 therapy, as well as the interaction mechanism between DDR and NK/B cells[84]. Single cell-driven molecular subtyping divides HCC into five subtypes and accurately distinguishes immunotherapy and targeted therapy responses[82]. Furthermore, a tissue-sampling-based imaging omics model using CT features can predict the objective response rate to lenvatinib plus anti-PD-1 inhibitors[85]. Integrated cross-omics models, such as protein-metabolome, have also revealed novel predictive markers such as complement pathways and LysoPCs[83].

Although AI models have shown potential in predicting liver cancer immunotherapy response, there are still multiple bottlenecks. Firstly, there is insufficient integration of multi-source data: There are differences in the standardization of data generated by different technical platforms (such as transcriptomics and radiomics), which limits the generalization ability of cross-modal models. For example, the DDR score model based on transcriptomics focuses on molecular interaction, while the CT imaging model relies on morphological features, and the two have not yet achieved the deep correlation of biological mechanisms. Second, model interpretability needs to be improved: Some AI prediction results lack clear biological support. For example, although the stemness gene-based classifier can predict treatment efficacy, how key markers (such as S100A10) regulate the immune microenvironment still needs to be verified experimentally. In addition, there are significant barriers to clinical translation: Most existing studies are based on retrospective cohorts with limited sample sizes (e.g., 10-25 cases) and lack of multicenter validation, which may overestimate model performance. Future optimization needs three breakthroughs: At the technical level, the development of a unified data standardization framework, the integration of multi-omics and imaging features, and the construction of a more biologically meaningful prediction dimension; At the mechanistic level, combined with organoid or spatial transcriptome technology, the functional role of AI screening markers (such as complement pathway and LysoPCs) in treatment response should be analyzed. At the clinical level, it is necessary to promote prospective, large-scale cohort studies and use XAI technology to extract key decision-making features, assisting doctors in formulating personalized treatment plans.

AI prediction of HCC recurrence after treatment: In the past, the prediction of postoperative recurrence of HCC primarily relied on clinical staging systems (such as BCLC staging) and single biomarkers (such as AFP), but these methods have significant shortcomings. Traditional models are difficult to integrate multimodal data (such as imaging, pathology, and blood indicators), resulting in limited prediction accuracy[86]. For example, the random forest model based on blood neutrophil-to-eosinophil ratio is superior to traditional staging, but it relies on a single blood indicator and cannot reflect tumor heterogeneity[86]. In addition, traditional radiomics relies on manual delineation of the tumor region, which is time-consuming and prone to subjective bias, especially in the assessment of residual lesions after thermal ablation[87]. These limitations lead to a high rate of missed diagnosis of early recurrence, which cannot guide individualized treatment.

Current AI techniques have significantly improved recurrence prediction performance through deep learning and multimodal data fusion. Automatic MRI segmentation based on 3D U-net can accurately extract tumor volume and radiomics features; when combined with clinical indicators, it achieves an AUC of 0.91 for early recurrence prediction[88]. Deep learning can also quantify the TTB, optimize the BCLC subclassification, and refine the recurrence risk stratification of BCLC stage A/B patients[88]. Multicenter studies have further verified the generalization of AI models: The RDC model, which integrates MR imaging omics, 3D CNN, and clinical indicators, shows stable cross-center AUC values of 0.85 to 0.88 for predicting recurrence after thermal ablation[16]. In addition, the CNN-SASM model based on pathological images can not only predict recurrence but also evaluate sorafenib treatment response, facilitating treatment decisions[18].

Current AI prediction models still face three core challenges: The bottleneck of data integration, the limitation of computational power, and the lack of biological interpretation. Most existing studies focus on a single data type, such as independent modeling of MRI radiomics or blood indicators (for example, Wei et al[88] only used MRI to predict recurrence). However, HCC recurrence involves the interaction of multiple factors, such as TME and treatment response, and cross-modal data fusion is not yet mature. The multi-center validation showed that differences in MRI scanning parameters among hospitals lead to fluctuations in model performance, indicating the need to establish standardized data acquisition protocols and synthesize diverse data through GAN to improve generalization ability. The issue of dependency on computing resources is also prominent: 3D deep learning models have high GPU requirements; for example, the TTB model takes 10 hours to complete a single case segmentation, making it difficult to apply in primary hospitals. In this regard, 2.5D deep learning architectures, such as the multi-view CT model by Zhang et al[89], can be used for reference to partially sacrifice the spatial dimension for computational efficiency, and knowledge distillation technology can be introduced to compress the large model into a lightweight version. The most critical limitation is that the model is out of step with clinical practice. For example, although TTB indicators can predict recurrence, the biological mechanism reflected by TTB is not clear, and doctors cannot make intervention plans based on it. In the future, it is necessary to promote the model design of "interpretability-driven": On the one hand, it is valuable to combine pathological images and genomic data (such as the CNN-SASM model of Li et al[18] to correlate sorafenib efficacy). On the other hand, it is necessary to visualize the decision-making basis of the model through the attention mechanism, such as positioning the recurrence-prone subregions to guide targeted radiotherapy. Only by translating AI output into clinically actionable biological insights can we truly achieve individualized intervention.

KEY CHALLENGES OF MULTIMODAL AI TECHNOLOGY

Data standardization and sharing

Multimodal AI technology has significantly improved the accuracy of diagnosis and treatment of digestive system tumors by integrating endoscopic, imaging, pathological, and omics data[21]. However, approximately 80% of medical AI models fail to achieve clinical translation due to data quality issues[20]. The lack of standardization of cross-modal data leads to a decrease in model generalization. For example, the AUC difference of the gastric cancer CT model between different devices can reach 0.15[7]. This "data reef" has become the core bottleneck[11], restricting the application of AI technology.

Intra-modality heterogeneity: Differences in imaging equipment parameters (such as CT slice thickness) lead to feature drift and reduce model stability[7]. The sensitivity of the deep learning model decreased by 20% due to the batch difference of pathological stains[74]. The scale mismatch between radiomics (millimeter-level) and spatial transcriptome (micron-level) hindered microenvironment modeling[17], and the standard divergence of endoscopic video annotation increased training noise by 40%[11]. Data siloing dilemma: Only 18% of the world's Top 100 hospitals are open to cross-institutional data sharing, and regulations such as GDPR cause the cost of data desensitization to surge[19].

Technological innovation: Device-independent domain adaptive framework (such as CycleGAN-MRI) was developed to reduce inter-vendor differences in CT features to within 5%[30]; Real-time staining normalization tools (HistoNorm) enable color alignment of H&E sections. Sharing mechanism: The Digestive System Diseases Federated Learning Consortium (GI-FedNet) has been constructed, covering 300 hospitals in 30 countries, with inter-center AUC fluctuations of the model < 0.0519[19]. The diffusion model was used to generate synthetic data and expand the liver cancer CT dataset by 10-fold[32]. Standardization construction: A white paper on Medical AI Data Labeling has been formulated, and labeling standards for endoscopic lesions (Paris classification) and molecular subtypes (TCGA) are mandatory[11]. A multimodal GastroData Commons was established, and differential privacy (ε = 0.1) was used to balance data utility and security[19].

Bottleneck of technology validation and clinical transformation

Lack of multi-modality labeling standards: There is no unified standard for the joint labeling of endoscopic, pathological, and imaging data, leading to increased noise in model training. For example, the false negative rate of an AI model for LNM of colorectal cancer increased to 8% due to the difference in scanning equipment in cross-institutional validation[90]. Insufficient generalization of the model: Device dependence and biological heterogeneity reduce the performance of cross-center validation. Studies have shown that imaging-based inflammatory bowel disease prediction models generally have a high risk of bias due to the small sample size (median: 200 cases) and lack of external validation[91]. Fragmentation of the validation system: The existing validation overly relies on AUC and other single indicators, ignoring clinical relevance. According to Nature Medicine, the AI pathology tool AIM-MASH can reduce the variance of manual scores, but it is not directly related to treatment outcomes[92].

Difficulties in clinical workflow integration: AI tools increase endoscopic procedure time (+30 seconds/case), and the adoption rate of AI recommendations by doctors is only 58%[11]. The regulatory and payment systems are lagging: Approval standards for multimodal AI have not been unified, with an average approval time of 3.2 years, and there is a lack of medical insurance coverage[90]. Data privacy and liability risks: GDPR compliance requirements have forced 30% of key clinical features to be desensitized, weakening models’ predictive performance[11]. There is a perception gap between doctors and patients: Only 12% of gastroenterologists have received AI training, and 43% of patients are concerned about decision-making transparency[92].

Dynamic verification framework: An international consensus on multimodal labeling has been developed to unify endoscope-pathology-image grading standards[11]; Device-independent algorithms (such as cross-scanner adaptive models) have been developed to reduce feature differences to < 5%[90].

Real-world evidence acceleration: Establishing multinational prospective cohorts (such as the GI-AI Registry) to enforce dynamic updating of model performance data[91]. Regulation-payment collaborative innovation: Establish AI approval "green channels" (such as low-risk polyp testing tools) to shorten the period to 6 months; AI will be included in the DRG "precision medicine surcharge" and paid[11] according to efficacy. Explainable enhancement design: Embedding neural-symbolic AI modules, outputting decision logic chains (such as "malignant probability 72%: Vascular density + surface structural abnormality"), and improving clinical trust[92].

FUTURE DIRECTIONS

In the future, AI technology in gastroenterology will evolve towards full-cycle closed-loop management and deep integration of multi-dimensional data. At the precision diagnosis level, the multimodal AI system will further integrate endoscopic video streaming, radiomics, pathological ultrastructure and liquid biopsy data to construct a dynamic risk assessment model. For example, through real-time endoscopic image analysis combined with ctDNA methylation characteristics, the molecular classification of gastrointestinal tumors and depth of invasion can be synchronized to optimize endoscopic resection decisions. In the treatment stage, the dynamic decision support system based on reinforcement learning will integrate the patient's genomic characteristics, treatment response time series data and intestinal flora dynamics, and adjust the chemotherapy/immunotherapy regimen in real time to reduce the risk of overtreatment (such as avoiding unnecessary lymph node dissection in low-risk T1 gastric cancer patients). In the field of whole-process management, AI-driven digital twin technology can simulate disease progression trajectory, combine with wearable devices to monitor symptom fluctuations, and provide personalized intervention strategies (such as stress warning and diet adjustment) for patients with functional gastrointestinal diseases such as IBS. In addition, the construction of interdisciplinary knowledge maps will promote the risk prediction of gastrointestinal tumors from single lesion analysis to systemic evaluation (such as association modeling of metabolic syndrome and colorectal cancer), ultimately realizing the whole-chain "screening-diagnosis-treatment-follow-up" intelligent management.

AI technology for liver diseases will break through the traditional single-dimensional analysis model and reshape the individualized treatment paradigm through cross-scale data integration and dynamic microenvironment analysis. In the management of liver cirrhosis and liver cancer, spatial multi-omics technologies (such as spatial transcriptomics + radiomics) will reveal the coupling mechanism of cellular heterogeneity and hemodynamic changes in liver fibrosis and build predictive models from molecular abnormalities to organ failure. The AI-driven donor matching system for liver transplantation will incorporate three-dimensional vascular modeling of the donor liver, the characteristics of the immune repertoire of the recipient and the dynamic data of the postoperative metabolome and optimize transplantation prognosis through graph neural network. For the treatment of liver cancer, platforms based on bionic organ chips and AI combined training can simulate tumor-immune microenvironment interactions and quickly screen the optimal timing combinations of target-immune combined therapy. In addition, a lightweight AI model based on liquid biopsy combined with ultrasound elastography will realize home monitoring of liver cancer recurrence and predict the risk of subclinical recurrence through the temporal correlation between ctDNA mutation spectrum and liver stiffness values. In the future, treatment decisions for liver diseases will deeply integrate cross-scale biomarkers, such as single-cell resolution immune features and peritumor image texture, form a precise regulatory network from molecular intervention to organ function maintenance, and promote the transformation of end-stage liver disease management from "experience-driven" to "mechanism-driven".

CONCLUSION

Multimodal AI technology can significantly improve the accuracy of disease identification, molecular typing and prognosis prediction by integrating endoscopic imaging, radiomics, pathological features and multi-omics data in the diagnosis and treatment of gastrointestinal and liver diseases. AI-driven models for predicting LNM in gastric cancer, response to immunotherapy in colorectal cancer, and recurrence of liver cancer have gradually broken through the limitations of relying on a single indicator or subjective experience in traditional diagnosis and treatment models. However, existing technologies still face core challenges such as data heterogeneity, insufficient model interpretability, and low clinical translation efficiency.

At the technical optimization level, cross-device data standardization based on transfer learning, interpretability enhancement design of neural-symbolic AI, and multi-center dynamic verification supported by FL are becoming key paths to solve these bottlenecks. For example, real-time fusion analysis of endoscopic video streaming and ctDNA methylation features can simultaneously realize molecular typing of gastric cancer and determination of invasion depth. In liver disease management, the cross-scale integration of spatial multi-omics and radiomics can analyze the coupling mechanism of cellular heterogeneity and hemodynamics in the process of fibrosis.

The cross-integration of current clinical practice and AI technology has far-reaching potential. From the combined detection of serum multiple markers to image segmentation models based on UNet, AI has not only improved the sensitivity of early screening of gastrointestinal tumors but also optimized individualized treatment strategies (such as avoiding excessive lymph node dissection in low-risk gastric cancer patients) through dynamic decision support systems. At the same time, the dynamic data closed-loop of organ-chip-AI co-simulation platforms, wearable devices and liquid biopsy is promoting the transformation of digestive system disease management to the full-cycle mode of "precise intervention-dynamic monitoring-individualized treatment".

The future development of gastroenterology and hepatology should o deeply integrate basic research, engineering technology innovation and clinical needs. On the one hand, causal inference models need to be used to analyze the relationship between AI features and biological mechanisms (such as immune microenvironment dynamics and metabolic pathway abnormalities). On the other hand, an interdisciplinary collaborative ecology should be constructed, integrating genomic intervention, real-time imaging analysis, and treatment response prediction into a unified framework, ultimately realizing the transformation of intelligent medical care from "single-point breakthrough" to "full-chain reconstruction".