Explainable artificial intelligence for personalized management of inflammatory bowel disease: A minireview of recent advances

Abstract

Personalized management of inflammatory bowel disease (IBD) is crucial due to the heterogeneity in disease presentation, variable therapeutic response, and the unpredictable nature of disease progression. Although artificial intelligence (AI) and machine learning algorithms offer promising solutions by analyzing complex, multidimensional patient data, the “black-box” nature of many AI models limits their clinical adoption. Explainable AI (XAI) addresses this challenge by making data-driven predictions more transparent and clinically actionable. This minireview focuses on recent advancements and clinical relevance of integrating XAI for personalized IBD management. We explore the importance of XAI in prioritizing treatment and highlight how XAI techniques, such as feature-attribution explanations and interpretable model architectures, enhance transparency in AI models. In recent years, XAI models have been applied to diagnose IBD anomalies by prioritizing the predictive features for gastrointestinal bleeding and dietary intake patterns. Furthermore, studies have revealed that XAI application enhances IBD risk stratification and improves the prediction of drug efficacy and patient responses with high accuracy. By transforming opaque AI models into interpretable tools, XAI fosters clinician trust, supports personalized decision-making, and enables the safe deployment of AI systems in sensitive, individualized IBD care pathways.

Key Words: Precision medicine; Ulcerative colitis; Crohn’s disease; Heterogeneous population; Machine learning; Interpretability; Feature attribution; Clinical decision-making

Core Tip: Personalized management of inflammatory bowel disease is essential because of its heterogeneous clinical presentations and variable treatment responses. While artificial intelligence (AI) offers powerful tools for analyzing patient data and guiding treatment, many AI models lack transparency, which limits their clinical adoption. Explainable AI addresses this issue by making AI predictions more interpretable and trustworthy. This minireview highlights recent advancements in the application of explainable AI to inflammatory bowel disease management, including its use in predicting disease progression, selecting therapies, and monitoring treatment response.

INTRODUCTION

Inflammatory bowel disease (IBD) is a chronic, immune-mediated disorder of the gastrointestinal tract, comprising Crohn’s disease (CD), ulcerative colitis (UC), and indeterminate colitis. IBD occurs in genetically predisposed individuals due to complex interactions arising from dysregulated immune response to the gut microbiota and environmental factors such as diet, geography, and smoking. IBD imposes a significant health burden on millions of people globally, with growing case numbers driven by aging populations and improved survival[1,2]. The heterogeneity in IBD’s clinical presentation, including variable disease location, histopathology, severity, and treatment response, makes personalized care essential but challenging[3].

Personalized management of IBD aims to integrate clinical, genetic, serological, and microbial data to better stratify patients, predict outcomes, and optimize treatment. It considers the heterogeneity of IBD clinical presentations and the underlying disease mechanisms in order to provide several opportunities to improve patient care by maximizing treatment efficacy while minimizing adverse events[4]. Despite notable advances in diagnostics (e.g., colonoscopy, ultrasound, magnetic resonance imaging, histopathology)[5] and therapeutics, clinicians still face major challenges in determining which patients will respond to which treatment, when to escalate therapy, and how to predict the long-term outcomes. These uncertainties hinder timely, targeted intervention and may contribute to suboptimal care or overtreatment.

Studies have demonstrated that approximately one-third of patients requiring corticosteroids do well without needing advanced therapy, indicating a heterogeneous disease course[4]. Ethnic and geographic variability in IBD clinical presentation further influences care strategy. For example, Asian populations show different genetic risk profiles and disease phenotypes compared to Western populations, warranting context-specific diagnostic pathways and regionally adapted treatment protocols[4]. Prognostic models that incorporate clinical variables [e.g., albumin levels, prior surgery, tumor necrosis factor (TNF) exposure] are under development but require broader validation[6,7]. Therefore, future care in IBD requires identifying limitations to current treatment and defining treatment targets that encompass clinical symptoms, biomarkers, endoscopy, cross-sectional imaging, and histopathology.

Artificial intelligence (AI) and machine learning (ML) are increasingly utilized to transform IBD management by enabling the identification of predictive biomarkers and therapeutic response patterns, thereby informing personalized management paradigms in IBD care. These computational approaches have been developed to identify subtle patterns in patients’ datasets, and generate predictive models that assist in clinical decision-making[8]. Notably, AI models have been developed to predict treatment response to biologics, assess disease severity using medical imaging and histopathology, and estimate the risk of relapse[9,10]. However, the widespread clinical adoption of AI in IBD remains limited due to the opacity of many AI-driven models, commonly referred to as the “black-box” AI, whose internal processes are unknown to its users, and which lack interpretability and transparency. Traditional “black-box” algorithms often fail to provide insight into the rationale behind predictions, making it difficult for clinicians to interpret or trust decision-making processes, especially in high-risk patients. Additional challenges in implementing personalized medicine for IBD include developing new technologies for individualized patient management and data analysis, validating existing biomarkers, and overcoming regulatory hurdles[11].

This is where explainable AI (XAI) becomes critical. Unlike traditional “black-box” AI systems, XAI enhances model interpretability by offering human-understandable explanations for its predictions by allowing clinicians to identify which features, such as inflammatory biomarkers, imaging scores, or genetic variants, are most influential in a model’s prediction. By making complex model outputs transparent, XAI equips clinicians with actionable insights into how and why decisions are made, using techniques such as feature attribution, decision trees, attention mechanisms, and inherently interpretable model architectures[12,13]. In IBD management, XAI has been applied to prioritize plasma and fecal metabolites and analyze their dietary associations[14]. Fecal metabolites were found to be more robust predictors of IBD compared to plasma metabolites, with stronger diet-metabolite associations. XAI models could, for example, elucidate why a patient is predicted to respond poorly to anti-TNF therapy or why early surgical intervention is recommended. This interpretability facilitates shared decision-making between clinicians and patients, supporting more precise and personalized care strategies. With the increasing global burden of age-related mortality in IBD and the growing demand for cost-effective, personalized treatment strategies, the integration of XAI into clinical workflows offers significant clinical advantages, by fostering clinical trust, regulatory acceptance, and ethical deployment. XAI facilitates the translation of innovative algorithms into practical clinical applications.

The objective of this minireview is to provide an overview of XAI techniques relevant to IBD management and highlight recent advances in XAI application in IBD management, with a focus on their capacity to improve interpretability and optimize patient outcomes. In contrast to previous reviews that broadly cover AI applications in gastroenterology, our minireview specifically centers on XAI-based approaches, their relevance to IBD care, and the current challenges to their real-world implementation.

XAI TECHNIQUES

XAI encompasses a set of methods and approaches designed to provide transparent, interpretable reasoning behind AI-generated decisions and predictions from ML algorithms. It allows clinicians to understand why a particular treatment is recommended or why a patient is classified as high risk. Before examining how XAI enhances IBD treatment, we outline key interpretability frameworks and classification approaches that underpin explainable modeling in clinical AI systems. Various classification criteria have been proposed in recent literature[15,16], and Figure 1 summarizes these criteria and their respective categories. Below, we elaborate on the categories and characteristics of XAI techniques.

Figure 1 Summary of explainable artificial intelligence methods categorized in this study. AI: Artificial intelligence.

Explanation scope

The scope of model interpretability can be categorized into local and global explanations, each serving distinct yet complementary purposes.

Local explanations: Local explanations focus on the rationale behind individual model predictions, treating the model as a black box and identifying the features that influenced a specific decision. Consider a scenario in which a doctor uses a classifier to guide treatment in a patient with complex clinical features. To build trust and ensure accountability, the doctor must understand why the model made that recommendation. This is where locally explainable methods, such as local interpretable model-agnostic explanation (LIME), SHapley Additive exPlanations (SHAP), and integrated gradients are critical to providing case-level interpretability. These approaches generate explanations centered on a single input instance by analyzing how individual features contribute to the prediction, thus enabling end users to evaluate and validate the model’s reasoning[17,18].

Global explanation: In contrast, global explainability methods aim to provide insights into the overall behavior of an AI model across a broad set of input data points. These approaches help uncover how the model processes patterns, assigns importance to features, and makes predictions[19]. Global methods often involve simplifying complex, non-linear models into more interpretable forms, such as linear approximations or rule-based representations. For instance, decision trees and other tree-based algorithms are inherently interpretable at a global level because their hierarchical structure allows each decision path to be traced back to specific input features. Typically, global explanation techniques operate on an array of input instances to address the question: “How does the model generally make decisions?” It often relies on the model’s internal parameters and performance, enabling users to understand the interactions among variables and the general decision-making mechanism.

Importantly, under certain conditions, local explanations can be aggregated to construct meaningful global interpretations of a model’s behavior[20]. Hybrid approaches that combine local and global methods are increasingly being used to provide more holistic and concept-based explanations. For example, integrated gradients can provide fine-grained local feature attributions, whereas testing with concept activation vectors enables global, concept-level insights[19]. Similarly, a rule-based model-agnostic method generalizes global explanations from local ones using a scoring system based on rule relevance[20]. These approaches aim to provide comprehensive, human-understandable insights into the decision-making processes of ML models.

Methodology types

XAI techniques can also be classified based on their compatibility with different model types. The key methodological difference lies in how explanations are computed. Model-specific methods require a forward and backward pass through the model to compute gradients or internal activations. Model-agnostic methods typically require only a forward pass, using repeated input perturbations and response evaluations to infer explanatory patterns.

Model-specific explanations: Model-specific methods are tailored to specific model architectures. They rely on the internal structure of the model, such as weights, gradients, or layer activations, to generate explanations. For example, backpropagation-based techniques such as saliency maps and class activation maps require access to gradients and can only be applied to models like neural networks. Although they offer detailed insights, their use is limited to specific models[21].

Model-agnostic explanations: In contrast, model-agnostic methods treat the model as a black box and do not depend on its internal workings. Techniques such as LIME and SHAP perturb input features and analyze output changes to infer the feature importance. These methods are more flexible and applicable across a wide range of models, although they may be less precise and computationally intensive[21,22].

Interpretation types

XAI methods are further categorized into intrinsic and post hoc approaches based on when and how interpretability is introduced.

Intrinsic (model-based) explainability: This refers to models that are inherently interpretable due to their transparent structure. Examples include linear regression, decision trees, rule lists, and generalized additive models[23]. These models embed explainability into their architecture, allowing predictions and rationales to be derived directly from the model’s internal rules or coefficients. However, intrinsic methods are often model-specific and may sacrifice predictive performance in complex tasks[24].

Post hoc explainability: This involves the application of interpretation techniques after the model is trained. These methods treat models as black or white boxes, offering explanations without altering the original architecture. Techniques such as SHAP, LIME, saliency maps, and class activation mapping (CAM) fall into this category. Post hoc methods are typically model-agnostic, providing flexibility and allowing the integration of explainability into high-performing but opaque models such as deep neural networks[25,26]. While intrinsic methods offer transparency by design, post hoc techniques provide broad applicability and are crucial for interpreting complex, pre-trained models without compromising accuracy.

WHY IS XAI IMPORTANT FOR PRIORITIZING TREATMENT IN IBD

With foundational XAI concepts in place, the following section discusses their relevance and benefits in IBD management. The diagram in Figure 2 illustrates three levels of AI interpretability in diagnosing IBD from endoscopic images. At the top, a black-box AI model simply identified the image as “IBD” without context. This leaves the clinician confused, questioning the reliability and clinical validity of the output, a critical challenge that influences clinical decisions such as initiating or de-escalating biologics.

Figure 2 Visual comparison of conventional artificial intelligence (with a black box model) and two interpretation types of explainable artificial intelligence models (described in the second section under “interpretation types”) and the implication for user feedback of endoscopic images from inflammatory bowel diseases. The black box model (arrow, above the broken line) provides limited information for classification results such as distinguishing between inflammatory bowel diseases and non-inflammatory bowel diseases endoscopic images. While explainable artificial intelligence methods are represented by the middle and bottom branched arrows represent post hoc and intrinsic method explainable artificial intelligence methods. IBD: Inflammatory bowel diseases; AI: Artificial intelligence; XAI: Explainable artificial intelligence.

In contrast, the middle layer adds explainability by highlighting regions of the image that contributed to the diagnosis [e.g., via saliency maps or gradient-weighted CAM (Grad-CAM)]. This allows the physician to understand the model’s reasoning, thereby increasing confidence in applying the result to treatment planning. The bottom layer represents an even more transparent model, which not only points to important areas but also explains why those features indicate IBD (e.g., pattern recognition similar to known clinical indicators). This level of interpretability supports shared decision-making, aligns with clinical reasoning, and helps guide nuanced treatment strategies, such as recommending early biologic intervention in severe endoscopic patterns or tapering therapy in stable UC. Therefore, integrating XAI frameworks into clinical pipelines for IBD management would serve essential functions that collectively enhance patient safety, regulatory alignment, and physician adoption.

Clinical accountability and decision justification

Treatment decisions in IBD, ranging from immunosuppressive therapy to surgical interventions, carry substantial risk. The risk profile varies according to drug class, patient age, and gender, necessitating personalized benefit-risk assessments[27]. Given this complexity, shared decision-making is essential, requiring patients to understand the potential benefits, adverse effects, and the implications of non-treatment[28]. For example, models that forecast the response to anti-TNF therapy can be paired with feature-attribution explanations (e.g., SHAP values) that reveal which clinical or molecular factors drove the prediction. This transparency promotes clinician accountability and justifies treatment choices in ethically and legally sensitive contexts. Furthermore, when misclassifications occur, due to data quality issues or atypical cases, XAI supports root-cause analysis and refinement, ensuring the system evolves safely over time. By adopting a personalized approach and optimizing drug safety profiles, clinicians can enhance treatment outcomes while minimizing risks in IBD[27].

Building trust among clinicians and patients

Trust is foundational in medical management. AI-driven decision support must be sufficiently transparent to be interpretable by domain experts, particularly in clinical decision support systems[29]. XAI allows gastroenterologists and radiologists to understand the rationale behind AI predictions (e.g., why a certain patient would require early biological intervention), thereby fostering confidence and facilitating shared decision-making between clinicians and patients[29,30].

Regulatory compliance and ethical mandates

International regulations, including the General Data Protection Regulation in Europe, have increasingly demanded that automated systems provide meaningful explanations for their decisions[31]. XAI supports these regulatory requirements by making the inner workings of the AI models more transparent and auditable. The General Data Protection Regulation’s “right to explanation” has significant implications for the design and deployment of automated data processing systems, although the exact nature of this right is debated. Data protection authorities have enhanced powers to enforce and interpret this right, potentially leading to the adoption of algorithmic auditing and “data protection by design” methodologies as new industry standards[32]. These developments support regulatory requirements for meaningful explanations of AI decisions.

Enhancing model robustness and clinical performance

Explainability facilitates model optimization. By revealing which input features (e.g., specific imaging markers, C-reactive protein levels, or patient history) disproportionately influence outcomes, data scientists and clinicians can collaboratively fine-tune models for better generalization. These models can predict significant outcomes such as hospitalizations and treatment response, potentially reducing healthcare costs[33]. This iterative refinement is vital for IBD, where heterogeneity in disease presentation requires adaptable, context-sensitive algorithms.

Detecting bias and ensuring equitable care

AI in healthcare presents both opportunities and risks for equitable care delivery. Although AI can improve clinical decision-making, it may perpetuate or exacerbate biases and health disparities if not carefully developed and implemented[34,35]. Algorithmic biases can arise from various sources, including data acquisition, genetic variation, and labeling variability, leading to unequal outcomes across patient subpopulations[35]. To address these challenges, researchers propose interdisciplinary approaches to advance equitable XAI, leveraging expertise from social epidemiology and health equity to critically assess model explanations and identify potential biases[36]. Emerging technologies such as federated learning, disentanglement, and model explainability offer promising avenues for mitigating bias in AI systems[36].

XAI METHODS AND APPLICATIONS IN IBD TREATMENT

This section provides an overview of prominent XAI methods and their specific contributions to personalized treatment of IBD, including CD and UC. Table 1 outlines the most commonly used XAI techniques in IBD management, studies employing these methods and their significant contributions.

Table 1 Overview of studies selected for the review of explainable artificial intelligence techniques in inflammatory bowel disease care.

Ref.	XAI technique	IBD type	Modality	Classifier	Contribution	Evaluation metrics (highest)
Onwuka et al[14]	SHAP	CD and UC	Multiomics dataset	LGBM	This study applies SHAP-based XAI to identify key fecal metabolites that robustly predict IBD and their diet associations. It demonstrates that XAI not only enhances metabolite prioritization across datasets but also clarifies how diet-driven microbial metabolites distinctly influence IBD pathology	AUC = 0.93 (distinguishing between IBD and non-IBD cases)
Patel et al[26]	Grad-CAM, saliency maps, integrated gradients, and LIME	UC	Endoscopic images	ResNet50 and MobileNetV2	This study evaluates four XAI techniques predictions in IBD-related endoscopic image classification. The analysis reveals that Grad-CAM most consistently highlights relevant regions, supporting its use in visualizing model focus and validating TeleXGI’s multi-XAI strategy for clinical interpretability	ACC = 98.8% (health risk prediction of UC endoscopic images remotely)
Chierici et al[37]	Saliency and gradient-based techniques	CD and UC	Endoscopic images	DenseNet121	This study applied saliency and guided backpropagation to a ResNet50 model for IBD endoscopic image classification. The attribution maps revealed clinically relevant features, with guided backpropagation offering clearer, less noisy insights	MCC = 0.94 (classification of healthy controls and IBD patients)
Sutton et al[38]	Grad-CAM	UC	Endoscopic images	DenseNet121	This study uses explainable deep learning on HyperKvasir endoscopic images to accurately distinguished UC from non-UC conditions and stratified disease severity based on Mayo scores. The DenseNet121 model achieved the highest performance, with Grad-CAM enhancing interpretability through visual explanations	AUC = 0.90 (classification of UC and non-UC images)
Tsai and Lee[39]	Grad-CAM	UC	Endoscopic images	DenseNet201, InceptionV3, and VGG19	This study leverages Grad-CAM to interpret deep learning model predictions for UC classification, comparing individual CNNs and ensemble models. The triplet ensemble produced the most focused and clinically relevant heatmaps	ACC = 91%
Ma et al[40]	Grad-CAM	CD	Ultrasound images and clinical data	ResNet50	This study developed a deep learning model combining intestinal ultrasound images and clinical data to predict mucosal healing in CD after one year of treatment. Using Grad-CAM for interpretability, the model highlighted key features such as the bowel wall and mesentery, achieving a high PPV	AUC = 0.73 (classification of mucosal healing and non-mucosal healing)
Maurício and Domingues[41]	Grad-CAM, LIME, SHAP, occlusion sensitivity	CD and UC	Endoscopic images	CNN + LSTM	This study develops CNN and ViT models to classify CD and UC from endoscopic images, creating lighter versions via knowledge distillation for clinical use. The ViT-S/16 model achieved the best performance, accurately identifying IBD features while ignoring irrelevant elements. Multiple XAI interpretability analysis confirmed its reliability, with minimal misclassifications after distillation using temperature-based methods	ACC = 95% (distinguishing between active and non-active inflammation)
Weng et al[42]	SHAP and LIME	CD	Clinical data	Extreme gradient boosting	This study integrates SHAP and LIME to interpret an extreme gradient boosting-based model for differentiating intestinal tuberculosis from CD. These XAI methods successfully identified and visualized the key clinical features influencing the machine learning model’s predictions for differentiating intestinal tuberculosis and CD	MCC = 96.9%
Zhen et al[43]	LIME	CD and UC	Clinical data	SVC	This study applied LIME to interpret an SVM model predicting quality-of-life impairment in IBD patients, identifying key modifiable risk factors such as anxiety, abdominal pain, and glucocorticoid use	AUC = 80% (evaluating IBD-related quality-of-life impairments)
Deng et al[44]	Trainable attention mechanisms	CD	Pathological images	RFC and GNN	This study introduces a cross-scale attention mechanism for multi-instance learning that captures inter-scale interactions in whole slide images for CD diagnosis. Trained on approximately 250000 hematoxylin and eosin-stained patches. Cross-scale attention visualizations localize lesion patterns at different magnifications, enhancing both diagnostic accuracy and model interpretability	AUC = 0.89 (distinguishing between healthy CD patient)
de Maissin et al[45]	Trainable attention mechanisms	CD	Endoscopic images	ResNet 34	This study introduces a recurrent attention neural network trained on a multi-expert annotated CD capsule endoscopy dataset. It demonstrates that higher annotation quality significantly boosts diagnostic accuracy (up to 93.7% precision). The network mimics human visual focus, enabling interpretable lesion localization and outperforming standard CNNs as annotations improve	ACC = 94.6% (detection of pathological and non-pathological images)
Wu et al[46]	Trainable attention mechanisms	UC	Endoscopic images	FLATer	This study presents FLATer, an explainable transformer-based model combining CNN and ViT for GIT disease classification. An ablation study reveals the crucial role of the residual block and spatial attention in boosting performance and interpretability, with saliency maps confirming enhanced localization of pathological region	ACC = 99.7% (multi-class classification to categorize images into specific diseases)
Sucipto et al[47]	Trainable attention mechanisms	CD	Pathological images	RFC and GNN	This study introduces a cross-scale attention mechanism for multi-instance learning that captures inter-scale interactions in whole slide images for CD diagnosis. Trained on approximately 250000 hematoxylin and eosin-stained patches, it achieved an AUC of 0.8924. Cross-scale attention visualizations localize lesion patterns at different magnifications, enhancing both diagnostic accuracy and model interpretability	ACC = 87% and 85% (prediction of histologic remission)
Ahamed et al[48]	SHAP, heatmap, Grad-CAM, and saliency maps	UC	Endoscopic images	EELM	This study introduces a lightweight, parallel-depth CNN optimized for gastrointestinal image classification using the GastroVision dataset, including IBD cases. Integrated with multiple XAI techniques (Grad-CAM, SHAP, saliency maps), the model enhances interpretability and diagnostic transparency. Using EELM for final classification, the system achieved high accuracy, robust generalizability	AUC = 0.987 (distinguishing between normal and benign ulcer)
Elmagzoub et al[49]	Trainable attention mechanisms	UC	Endoscopic images	ResNet101	This study presents a grid search-optimized ResNet101 model with an integrated attention mechanism for classifying gastrointestinal diseases, including UC, from endoscopic images	ACC = 93.5%

IBD: Inflammatory bowel diseases; XAI: Explainable artificial intelligence; SHAP: SHapley Additive exPlanations; Grad-CAM: Gradient-weighted class activation mapping; LIME: Local interpretable model-agnostic explanation; UC: Ulcerative colitis; CD: Crohn’s disease; EELM: Ensemble extreme learning machine; RFC: Random forest classifier; GNN: Graph neural network; SVC: Support vector classifier; CNN: Convolutional neural network; LSTM: Long short-term memory; LGBM: Light gradient boosting machine; ViT: Vision transformer; EBV: Epstein-Barr virus; SVM: Support vector machine; GIT: Gastrointestinal tract; AUC: Area under the curve; ACC: Accuracy; MCC: Matthews correlation coefficient.

Saliency and gradient-based techniques

Saliency maps and guided backpropagation have been applied to deep learning models such as ResNet50 for classifying endoscopic images in IBD. These methods highlight relevant image features that influence the predictions. Guided backpropagation, in particular, provided clearer, less noisy attribution maps, assisting clinicians in verifying AI outputs against visible pathological features[37].

Grad-CAM

Grad-CAM has been widely adopted for IBD-related image interpretation. Studies using DenseNet121, ResNet variants, and ensemble convolutional neural networks (CNNs) have demonstrated the effectiveness of Grad-CAM in localizing clinically relevant areas in endoscopic and ultrasound images. It enabled visual validation of model attention, with several works showing heatmap alignment with expert annotations and highlighting critical regions such as mucosal surfaces and bowel walls[38-41].

Occlusion sensitivity

Occlusion-based techniques evaluate the effect of masking specific regions of an input image on a model’s prediction, thereby identifying the visual areas that are most critical to their decision-making process. In the context of IBD classification using vision transformer occlusion sensitivity, combined with SHAP and Grad-CAM, was used to compare different transformer distillation approaches. The analysis revealed that the data-efficient image transformers-based distilled model occasionally relied on irrelevant image elements, suggesting a lack of precise disease focus. Conversely, the temperature-distilled model demonstrated greater interpretability, consistently attending to clinically meaningful regions while ignoring visual noise[41].

SHAP

SHAP is an interpretability method designed to attribute a model’s prediction to its input features. SHAP quantifies the marginal contribution of each feature toward a particular output, offering both local and global explanations of model behavior. One of SHAP’s major advantages is its model-agnostic nature, that is, it can be applied across a wide range of ML algorithms while providing consistent, theoretically sound explanations. SHAP has proven valuable in both image-based and non-imaging modalities of IBD research[14,40,41]. In endoscopic image classification and multiomics analyses, SHAP provides insight into which features (e.g., specific image regions or fecal metabolites) most influence predictions. In one study, SHAP clarified the role of diet-driven microbial metabolites in IBD pathology, aiding biomarker discovery[14]. Another study used SHAP to differentiate CD from intestinal tuberculosis using clinical data[42].

LIME

LIME is an XAI technique that approximates the behavior of complex ML models by creating simple, interpretable models around individual predictions. LIME perturbs the input data and observes how these changes affect the model’s output. By evaluating prediction changes across a range of perturbed samples, LIME constructs a linear surrogate model that mirrors the behavior of a black-box model in a small, localized region around the input. This approach is particularly valuable when clinicians or researchers need to understand the rationale behind a specific decision rather than general model trends.

LIME has been successfully applied in IBD to interpret clinical prediction models. Notably, in a multicenter study predicting quality-of-life impairment, LIME identified modifiable risk factors such as anxiety, glucocorticoid use, and abdominal pain that predicted quality-of-life impairments among patients with IBD[43]. It has also been combined with SHAP to distinguish CD from intestinal tuberculosis by highlighting influential clinical features in an extreme gradient boosting model[42].

Trainable attention mechanisms

Trainable attention mechanisms allow a neural network to focus on specific parts of the input sequence by assigning weight to the input sequence. They are increasingly utilized in deep learning models to enhance interpretability by focusing on the most relevant parts of the input data during prediction. Similar to occlusion and saliency-based techniques that examine how masking or modifying input regions affect predictions, attention mechanisms internally learn which regions to prioritize, offering a built-in layer of interpretability. In CD diagnosis, trainable attention has shown significant promise in endoscopic and histopathological image analysis[44-47]. For example, attention-guided models have been trained on capsule endoscopy datasets annotated by multiple experts to diagnose CD[45]. These networks outperformed traditional CNNs as annotation quality improved, achieving high diagnostic precision. Similarly, cross-scale attention mechanisms applied to whole slide pathological images have successfully localized lesion patterns at different magnification levels achieving an area under the curve (AUC) of 0.8924 and aiding in the diagnosis of CD[44]. Another approach, FLATer (a hybrid transformer-CNN model optimized with attention), demonstrated that spatial attention significantly enhances both the classification accuracy and pathological region localization in UC[46].

Multi-XAI frameworks (saliency, Grad-CAM, LIME, SHAP, occlusion sensitivity)

Some studies evaluated multiple XAI techniques simultaneously such as Grad-CAM, saliency maps, integrated gradients, LIME, and SHAP, to enhance model interpretability from diverse perspectives. These approaches were especially useful in evaluating classification models for IBD (Table 1)[14,26,37-49].

DISCUSSION ON CLINICAL USE CASES OF XAI IN IBD MANAGEMENT

Numerous studies have applied XAI techniques to develop interpretable models for the classification process and for identifying key features in IBD. These approaches have demonstrated high accuracy in distinguishing IBD from non IBD cases[26,48]. This section explored studies identifying the use of XAI and ML techniques in clinical settings for diagnosis, risk stratification, treatment response prediction and clinical trial design, aiming to predict outcomes of interventions.

Diagnosis of IBD anomalies

XAI techniques are increasingly being applied in the diagnosis and management of IBD. XAI has also been applied to prioritize variables for predicting gastrointestinal bleeding and dietary intake patterns. Furthermore, ML models, particularly those using SHAP, have been employed to identify discriminative metabolites in plasma and feces for IBD diagnosis[14]. These models have shown high accuracy, with fecal metabolites providing more robust predictors (AUC = 0.93) than plasma metabolites (AUC = 0.74). In IBD endoscopy, AI techniques such as support vector machines and CNNs have improved diagnosis reliability and image processing[50]. AI-enhanced endoscopy has been applied in various aspects of IBD care, including diagnosis, assessment of mucosal activity, and prediction of recurrence and complications[51]. AI models have demonstrated potential in predicting treatment response and supporting shared decision-making between clinicians and patients[52].

Risk stratification

XAI offers potential for improved risk stratification and personalized care[53]. Class-contrastive and feature-attribution techniques, including a modified kernel SHAP algorithm, have been used to identify potential target genes and modules for patient stratification in CD[54]. AI models, particularly random forest and least absolute shrinkage and selection operator, have demonstrated high accuracy in predicting adverse outcomes such as hospitalization, surgery, and long-term steroid use in IBD patients[55]. These models can be applied for risk stratification and implementation of preemptive measures in clinical settings. While risk stratification is integral to IBD management according to care pathways and guidelines, its explicit use and influence on patient management in community-based practices requires further examination[56].

Drug response prediction

Recent studies have demonstrated the value of XAI in predicting treatment response and identifying patient-specific biomarkers in IBD. The integration of multi-modal data, including genomic, transcriptomic, and demographic information, improved prediction of drug efficacy and patient responses with high accuracy. Multi-omics studies using RNA-sequencing and ML models have linked specific genetic variants to differential drug efficacy[57]. For example, in patients treated with the p38 mitogen-activated protein kinase inhibitor BIRB796 (doramapimod), the presence of the alternate allele at rs2240467 in ADAM22 was associated with reduced transcript length and lower predicted TNFα levels, suggesting enhanced drug responsiveness. Similarly, a variant in PRDM1 (rs811925) was associated with both lower gene expression and a higher predicted TNFα level, indicating reduced treatment efficacy. XAI techniques, such as SHAP values, further clarify how these polymorphisms contribute to model predictions, offering clinicians mechanistic insights into patient-specific drug responsiveness[57]. In another study, ML models incorporating early treatment data, such as fecal calprotectin levels and drug concentrations, have successfully predicted long-term response to biologic agents like vedolizumab. Patients with high baseline fecal calprotectin were more likely to fail vedolizumab therapy, while combining fecal calprotectin and drug levels after six weeks yielded an area under the receiver operating characteristic curve of 0.73 (95% confidence interval: 0.65-0.82) for predicting remission at one year[14]. These approaches aim to identify important features associated with treatment efficacy, potentially linking genetic polymorphisms or metabolite concentration to variations in drug responses[1,57]. Additionally, computational models and digital twins have been developed to simulate patient-specific responses and optimize treatment strategies[52].

Clinical trial design

AI is improving clinical trial design in IBD by enhancing diagnostic precision, improving recruitment strategies, and enabling adaptive trial frameworks by leveraging electronic health records. AI can more efficiently identify eligible patients while enhancing the cohort diversity. Automated analysis of endoscopic and histologic images improves diagnostic accuracy and standardization across trial sites, reducing inter-observer variability. Additionally, AI-powered predictive modeling informs adaptive trial designs by forecasting patient responses and disease trajectories, helping tailor interventions in real time. Wearable sensors and mobile applications further facilitate continuous disease monitoring, allowing timely treatment adjustments and more dynamic endpoint assessments[56,57].

Recent clinical trials have explored AI’s predictive potential in real-world settings. For instance, Telesco et al[58] conducted a phase 2a study in UC patients using mucosal gene expression data to predict response to Golimumab. While their model achieved AUCs of 0.688 at week six and 0.671 at week thirty, its lower-than-expected performance highlighted challenges such as variability between training and validation cohorts. Similarly, Noor et al[59] evaluated a prognostic assay in a randomized CD trial aimed at distinguishing between step-up and top-down therapy strategies. However, the assay failed to accurately predict disease progression, underlining the need for more robust and well-validated predictive tools[60].

CHALLENGES AND FUTURE DIRECTIONS

A critical challenge highlighted in this reviewed study on XAI in IBD management is the difficulty of effectively integrating local and global explanations in a clinically meaningful manner. In IBD care, especially when personalizing treatment strategies, clinicians require case-specific interpretability to make informed decisions regarding individual patients[59]. Local explanation methods such as LIME, SHAP, and integrated gradients are essential for this purpose[18]. They help clinicians understand why a model makes a particular prediction in a given case, such as identifying anxiety or abdominal pain as predictors of reduced quality of life. This form of case-level transparency supports trust and accountability in AI-assisted decision-making. The challenge arises because local and global explanations often yield different perspectives, and synthesizing them into a unified, clinically actionable insight remains technically and conceptually complex. Hybrid methods, such as combining integrated gradients[17] with concept activation vectors or using rule-based aggregation of local SHAP scores, attempt to bridge this gap, however, they remain methodologically underdeveloped in IBD management, particularly in real-time clinical workflows. Furthermore, while broadly applicable, model-agnostic techniques tend to be computationally expensive and may oversimplify relationships by ignoring model-specific internal dynamics[20]. Conversely, model-specific methods, like saliency maps or CAM, are more precise but restricted to certain architectures (e.g., deep neural networks), limiting their generalizability across diverse clinical tools.

Another issue is the lack of accessible and comprehensive information resources that clearly communicate XAI concepts to end-users. Clinicians and healthcare stakeholders often lack the tools or guidance required to understand how different explanation methods work, how to interpret their outputs, and when each method is most appropriate. Challenges related to data quality, privacy, and ethical concerns need to be addressed for widespread clinical implementation. Moreover, current XAI systems often fall short in addressing the cognitive and contextual needs of end-users. There is a notable absence of robust requirements elicitation frameworks tailored specifically for healthcare professionals, which would ensure that explanation mechanisms align with how clinicians think, reason, and make decisions. Without this alignment, even technically sound explanations may fail to support meaningful clinical judgments.

In addition, few XAI systems are designed from a user-centric perspective. Algorithmic performance is often prioritized over usability, leading to interfaces or outputs that are difficult for non-technical users to interpret or apply. The limited deployment of explanation systems that are both clinically intuitive and responsive to end-user priorities continues to hinder real-world implementation.

Lastly, despite the promising advancements of XAI in adult IBD care, its application in pediatric populations presents several unique challenges. Children and adolescents with IBD often present atypically, manifesting symptoms like poor growth, anemia, or extraintestinal manifestations instead of classic gastrointestinal complaints[61]. These subtleties complicate early diagnosis and risk stratification, areas where AI and ML could provide critical support. However, the complexity and variability in pediatric disease expression mean that models trained predominantly on adult data may fail to capture the nuanced clinical presentations seen in younger patients. XAI tools could bridge this gap by offering transparent, clinician-interpretable outputs. However, the pediatric context demands heightened caution. Misinterpretation of XAI output, especially when models fail to generalize to younger patients, may lead to diagnostic delays or inappropriate treatment adjustments.

The future of IBD care will be increasingly shaped by intelligent, explainable technologies that extend support beyond clinical settings and empower both patients and clinicians. One promising direction is the development of AI-powered tools trained on colonoscopy images and biomarker data (e.g., calprotectin levels, sweat biosensors) to automatically grade disease severity and provide transparent reasoning behind their assessments[20,62]. Unlike opaque “black-box” models, XAI systems can communicate the reasoning behind their outputs, helping gastroenterologists, clinicians, and patients alike understand why a flare is predicted or a treatment is recommended. For example, activity data from wearables (like step trackers or heart monitors) can be combined with biomarkers (e.g., sweat-based markers) to track subtle shifts in health status. XAI can then correlate these patterns with flare risk, enabling early warnings and personalized alerts.

At-home ultrasound systems, guided live by remote experts via tablets or iPads, could further democratize access to care for patients with chronic, debilitating conditions who struggle to access in-person care. Coupled with XAI, these systems could provide interpretable, annotated feedback to patients and clinicians, bridging gaps in digital literacy and ensuring that clinical insights are both accessible and actionable.

Additionally, the integration of gut microbiome analysis, especially in combination with conventional IBD therapies, has demonstrated promising outcomes[60]. By analyzing microbiota shifts alongside metabolic and inflammatory profiles, XAI can help identify which patient subgroups are most likely to benefit from specific therapies. This is especially crucial in light of variable clinical response rates (e.g., 30%-40% for anti-TNF therapies like adalimumab)[63]. Furthermore, early intervention strategies supported by XAI, as seen in studies from SONIC trial, can aid in predicting and reducing time to first hospitalization, surgery, or complication[64,65]. By learning from historical patient trajectories and explaining risk predictions clearly, XAI enhances clinical decision-making while fostering patient trust. In summary, XAI has the capacity to reshape IBD care, from early detection and personalized therapy selection to at-home monitoring and long-term disease management, provided it remains transparent, inclusive, and grounded in real patient needs.

CONCLUSION

XAI holds transformative potential in the personalized management of IBD by making complex AI-driven predictions transparent and clinically interpretable. From improving diagnostic accuracy and risk stratification to predicting drug response and optimizing clinical trials, XAI enhances trust and usability in high-stakes decision-making. Despite current challenges in integration, interpretability, and real-world implementation, particularly in pediatric IBD care, XAI offers a promising path toward more precise, equitable, and patient-centered IBD care. Future efforts must focus on user-centric design, multi-modal integration, and context-aware deployment to fully realize the potential of XAI in clinical practice.