Development and clinical application of an ultrasound-based deep learning model for preoperative staging of colorectal cancer

Abstract

BACKGROUND

Considering that colorectal cancer (CRC) has high global incidence and mortality, accurate tumor staging before surgery is essential for individualized treatment formulation. Limited by insufficient capacities of traditional imaging modalities for assessing tumor invasion depth and lymph node metastasis, the exploration of more objective and accurate auxiliary diagnostic tools is critical.

AIM

To develop and validate an ultrasound (US) image-based deep learning (DL) model for automated, high-precision preoperative tumor node metastasis (TNM) staging prediction in CRC patients, this research intended to address subjectivity and poor repeatability issues in traditional US diagnoses and presents an objective clinical decision-making model.

METHODS

The artificial intelligence diagnostic model was developed and validated. 680 CRC patients retrospectively enrolled from The Third People’s Hospital of Henan Province (January 2019 to June 2022) constituted the development cohort, with their US images analyzed. A temporally independent test set, consisting of another 170 CRC cases (January 2023 to December 2024), was used for final evaluation. Postoperative pathological TNM staging served as the gold standard. A dual-path context-aware fusion network was designed to separately extract local tumor features and global contextual information from surrounding tissues, integrating them through a cross-attention mechanism to simulate clinical diagnostic thinking. Model performance for T (T1-T4) and N (N0-N2) staging was evaluated on the independent test set and compared with three sonographers of different seniority. Accuracy, area under the curve (AUC), and decision curve analysis (DCA) served as the primary outcome measures.

RESULTS

On the independent test set, the DL model demonstrated excellent diagnostic performance, with an overall AUC of 0.907 [95% confidence intervals (CI): 0.862-0.954] for the T-stage prediction and 0.912 (95%CI: 0.867-0.956) for the N-stage prediction. Accuracy was significantly higher than that of the physician group for both T (89.8% vs 81.9%) and N (94.9% vs 83.9%) staging, as were AUC values (all P < 0.05). DCA indicated that the model provided clear net clinical benefits across a wide range of threshold probabilities for both T and N staging.

CONCLUSION

In this study, an US-based dual-path DL model was successfully developed and validated, demonstrating superior diagnostic performance to sonographers in preoperative T and N staging of CRC on a temporally independent test set. The model shows promise as an objective and reliable auxiliary diagnostic tool to improve staging accuracy and consistency, although multicenter prospective validation is warranted.

Key Words: Colorectal cancer; Ultrasonic image; Deep learning; Preoperative staging; Tumor node metastasis

Core Tip: This study proposes an ultrasound-based deep learning model for preoperative tumor node metastasis staging of colorectal cancer (CRC). The model demonstrates strong diagnostic performance for T and N staging and provides clear clinical net benefits, providing an objective artificial intelligence-based tool for accurate preoperative staging of CRC.

INTRODUCTION

Colorectal cancer (CRC) ranks among the top malignancies worldwide in terms of morbidity and mortality. Accurate preoperative staging is fundamental for individualized treatment planning, prognosis evaluation, and improving patients’ quality of life[1-3]. Computed tomography (CT), magnetic resonance imaging (MRI), and endoscopic ultrasound (EUS) are the major diagnostic approaches for preoperative staging at present. CT and MRI, despite their effectiveness in evaluating distant metastases and regional lymph nodes, are limited by an inadequate ability in microscopic bowel wall infiltration evaluation[4-6]. EUS, given its high-resolution visualization of bowel wall layers, is considered the gold standard for T staging; yet, it is invasive and relies heavily on operator experience, causing interpretation variability[7-10]. Ultrasonography, including transabdominal (TAUS) or transrectal (TRUS) techniques, is radiation-free and allows for real-time dynamic imaging, with low cost and wide accessibility. With the ability to visualize bowel wall structure, tumor morphology, internal echogenicity, and tumor-peritumoral relationship clearly, it holds great promise in applications in tumor infiltration depth (T stage) and lymph node metastasis (LNM) (N stage) assessments[11-14]. However, traditional ultrasound (US) diagnoses exhibit drawbacks like operator-dependent interpretation and variability, speckle noises, artifacts, and anatomical complexities, hindering quantitative feature extraction.

The constantly advancing deep learning (DL) models, convolutional neural networks (CNNs) in particular, have gained increasingly attention in medical image analysis due to its strong performance in illness identification, segmentation, classification, and outcome prediction[15-19]. DL can address subjectivity by learning complex feature patterns from massive datasets, thus providing objective, repeatable quantitative analyses. When applied to US imaging, it shows non-inferior or even superior diagnostic performance compared to experienced physicians in differentiating the nature (benign/malignant) of thyroid nodules and breast lesions[20-23]. However, systematically applying DL to US image-based automated preoperative CRC staging, especially end-to-end models for simultaneous T and N staging, remains at the experimental stage. Optimizing model generalizability, interpretability, and multimodal data fusion is required.

In light of this, we aim to develop and validate an US image-based DL model for CRC to enable automated, high-precision preoperative tumor node metastasis (TNM) staging. Specifically, a large, high-quality US dataset with a strictly split temporally independent test set was constructed. An innovative framework, namely the dual-path context-aware fusion network integrating local and global features, was also designed to mimic clinical diagnosis practices. Furthermore, through rigorous prospective man–machine experiments, we systematically compared model performance with sonographers of different seniority. Also, the model’s clinical utility was evaluated to support its translation into a reliable decision-making tool. The model proposed is expected to be a brand-new and objective auxiliary diagnostic tool for accurate preoperative CRC evaluation, with profound clinical and translational implications.

MATERIALS AND METHODS

Research design

This is an artificial intelligence (AI)-based diagnostic model development and validation study, strictly following the Technical Guidance Principle for Clinical Evaluation of AI Medical Devices principles regarding algorithm performance evaluation. The research design consisted of: (1) Model building and internal validation stage (retrospective cohort study); and (2) Temporally independent external validation and clinical efficacy comparison stage (retrospective cohort study). See Figure 1 for the overall study design.

Figure 1 Overall study design. The study consisted of four phases: (1) Data collection from development cohort (n = 680) and independent test cohort (n = 170); (2) Image preprocessing including tumor annotation and quality assessment; (3) Construction of the dual-path context-aware fusion network for T and N staging; and (4) Model evaluation using cross-validation and independent test set. FPN: Feature pyramid networks; AUC: Area under the curve; AI: Artificial intelligence.

Research participants and datasets-data sources and case selection criteria

By reviewing the Ultrasonic Picture Archiving and Communication System and the associated Electronic Medical Record System of The Third People’s Hospital of Henan Province, we collected the information needed. Data were selected according to the following criteria for high-quality, low-bias dataset construction.

Model development cohort: Enrollment timeframe: January 1, 2019 to June 30, 2022. Eligibility: (1) ≥ 18 age groups; (2) A primary colorectal adenocarcinoma diagnosis by colonoscopy biopsy and postoperative pathology; (3) Radical surgery at our center; (4) Standardized TAUS or TRUS at our institution two weeks preceding the operation, with intact dynamic image sequences preserved; and (5) Accessible and intact postoperative pathological reports using the American Joint Committee on Cancer (AJCC) 8^th edition-based TNM staging (gold standard). Ineligibility: (1) Prior relevant treatment (e.g., neoadjuvant chemotherapy, radiotherapy, targeted therapy); (2) Inadequate image quality (≥ 50% ill-defined tumor contour due to intestinal gas or artifacts, or missing key anatomical layers); (3) Gastrointestinal stromal/neuroendocrine tumor, or lymphoma; (4) Inflammatory bowel disease (active); or (5) Missing key clinical/pathological information.

Temporally independent test cohort: This study further established a temporally separated test set to independently and objectively evaluate model generalizability and timeliness. Enrollment timeframe: January 1, 2023 to December 30, 2024. Using the same selection criteria as the development cohort, all eligible patients during this period were consecutively included. This cohort was specifically designed for the final performance evaluation and human-machine comparison, without involvement in model development.

Dataset split

We employed stratified patient-level random sampling within the development cohort to ensure assessment unbiasedness. Following pathological T- and N-staging-based stratification, the participants were allocated to the following sets (7:1:2 ratio). Training: For initial model parameter learning and updates. Tuning: For hyperparameter tuning, model structure selection, and early stopping criterion during training. Internal validation: For conducting phased performance evaluations during development. The temporally independent test cohort, serving as the final external test set, was used for model performance assessment and with man-machine performance comparison.

Gold standard, image preprocessing, and annotation

Gold standard: Pathological TNM staging determined by postoperative histopathology (AJCC 8^th edition) served as the diagnostic reference standard. Two senior pathologists, blinded to US findings and modeling results, conducted independent specimen review; disagreements were resolved through joint review and discussion.

Ultrasonic image preprocessing: For each patient, three representative images, optimally depicting the tumor while also involving the serosal surface and surrounding tissue relationships, were selected by a sonographer (> medium-grade professional title) from dynamic US sequences. All images were converted to grayscale and resized to 224 × 224 pixels, with pixel intensities normalized to the [0, 1] interval.

Image annotation and quality control: Tumor segmentation and annotation: Using ITK-SNAP software and a blind technique, two senior physicians (> 10 years’ experience in TAUS) independently and manually performed pixel-level delineation of the tumor region. The Dice similarity coefficient between annotations was required to exceed a mean of 0.75. Discrepancies were resolved by a third senior expert. Image quality scoring: Following usability evaluation using a 5-point Likert scale (1 = undiagnosable; 5 = excellent), images scored ≥ 3 on average were selected.

DL model construction

Model architecture design: This study proposed a dual-path, context-aware fusion network designed to simulate an US physician’s diagnostic process by integrating tiny local features with macroscopic anatomical relationships.

Local path: The input was the cropped image of the minimum tumor-area bounding rectangle. With the EfficientNet-B0 network as the encoder, intratumoral echo texture, microcalcification, boundary spiculations, and other fine features, were extracted.

Global context path: The original panoramic image, with the lesion, surrounding adipose tissue, fascia, and suspected lymph nodes involved, was the input. ConvNeXt-Tiny, a deeper network, encoded tumor-surrounding tissue spatial relationships, serosal surface continuity, and other macroscopic information.

Feature fusion module: We performed feature fusion on these two paths at the penultimate network level. Employing cross-attention mechanisms, local-global feature mutual weights were calculated to realize adaptive calibration and information complementarity among the features. Multi-scale semantic information was then fused by splicing and sending the calibrated features to a feature pyramid network.

Prediction head: The fused features, after passing through a global average pooling layer and two independent fully connected layers, output T- (T1-T4) and N (N0, N1-2)-stage probability distributions. To prevent overfitting, a Dropout layer (P = 0.5) was applied before each fully connected layer.

Model training and tuning: Loss function: An asymmetrically weighted multi-class focal loss function was used. Categories with greater clinical significance and smaller sample sizes (T4 and N+ staging) were assigned higher loss weights to guide the model toward these critical yet challenging samples.

Hyperparameter tuning: Model hyperparameters were optimized using grid search on the validation set. The learning rate ranged from 1 × 10^-5 to 1 × 10^-3, sampled logarithmically with a step size of 0.5. Candidate batch sizes were (8, 16, 32). The AdamW optimizer was used, with weight decay (regularization coefficient) set to 1 × 10^-4. A final learning rate of 5 × 10^-4 and a batch size of 16 yielded the minimum validation loss and the highest weighted Kappa (WK) coefficient.

Training strategy: Both encoders were initialized with weights pre-trained on the ImageNet-1K dataset. The AdamW optimizer was used with an initial learning rate of 5 × 10^-4 and a cosine annealing scheduler. The training batch size was 16, and the maximum number of iterations was 150. WK score monitoring was performed on the tuning set; training was terminated early if no improvement was observed for 15 consecutive cycles.

Statistical analysis

Model performance evaluation indicator: Model performance was evaluated using the temporally independent dataset. Accuracy, sensitivity and specificity for T- and N-stage diagnoses were calculated. Receiver operating characteristic (ROC) curves were plotted, and areas under the curve (AUCs) with corresponding 95% confidence intervals (CIs) were computed. The consistency strength was assessed using the WK coefficient between model-predicted and pathology-determined stages.

Comparative analysis of man-machine diagnosis: Three sonographers of different seniority (junior: < 5 years; intermediate: 5-10 years; senior: > 10 years) formed the physician group. Blinded to pathological diagnoses and model predictions, they independently performed T/N staging on all images in the temporally independent test set. Final diagnoses were determined by majority consensus. Differences in AUC between the DL model and physician group were assessed using the DeLong test. For key clinical decision points, the McNemar test was used to compare paired differences in sensitivity and specificity. Kappa values were calculated to assess agreement between model predictions and each physician’s diagnosis.

Decision curve analysis

Decision curve analysis (DCA) was conducted to quantify the clinical net benefit of using the model to assist clinical decision-making across different threshold probabilities, compared with default “treat-all” or “treat-none” strategies, thereby illustrating its potential clinical utility. Statistical data analyses were executed by R (v 4.2.0), adopting a P < 0.05 significance threshold. Statistical description used the mean ± SD or median (interquartile range) for continuous data and n (%) for categorical variables.

RESULTS

Baseline cohort data

Of 850 pathologically confirmed CRC patients undergoing radical surgery formed the study cohort, with 680 cases (January 2019 to June 2022) comprised the model development cohort and 170 (January 2023 to December 2024) constituted the temporally independent test cohort. Baseline characteristics, including age, sex, body mass index, ileus comorbidity, tumor site, maximum tumor diameter, preoperative carcinoembryonic antigen (CEA), T/N staging, pathological differentiation, and US image quality score, differed insignificantly across the cohorts (P > 0.05). It suggests high cohort comparability that enables objective evaluation of model generalization and clinical timeliness (Table 1).

Table 1 Baseline characteristic comparison (development vs temporally independent test cohort), mean ± SD/n (%).

Characteristics	Development cohort (n = 680)	Independent test cohort (n = 170)	Statistic	P value
Age (years)	62.56 ± 10.70	63.24 ± 11.44	0.918	0.359
Sex (male/female)	375/305	95/75	0.030	0.863
BMI (kg/m2)	23.5 ± 3.2	23.8 ± 3.4	t = 1.06	0.291
Ileus			0.803	0.370
Yes	68 (10.0)	21 (12.4)
No	612 (90.0)	149 (87.6)
Tumor location
Colon cancer	425 (62.5)	105 (61.8)	χ2 test	0.861
Rectal cancer	255 (37.5)	65 (38.2)
Maximum tumor diameter (cm)	4.2 ± 1.8	4.3 ± 1.7	t-test	0.512
Preoperative CEA (≥ 5 ng/mL)	306 (45.0)	77 (45.3)	0.005	0.945
T-staging
T1	85 (12.5)	22 (12.9)	0.051	0.997
T2	170 (25.0)	43 (25.3)
T3	306 (45.0)	75 (44.1)
T4	119 (17.5)	30 (17.6)
N-staging
N0	374 (55.0)	93 (54.7)	0.020	0.995
N1	204 (30.0)	51 (30.0)
N2	102 (15.0)	26 (15.3)
Pathological differentiation			0.203	0.652
High/moderate differentiation	476 (70.0)	122 (71.8)
Poor differentiation/mucinous adenocarcinoma	204 (30.0)	48 (28.2)
Image quality score
3 points (acceptable)	204 (30.0)	51 (30.0)	0.000	1.000
4 points (good)	340 (50.0)	85 (50.0)
5 points (excellent)	136 (20.0)	34 (20.0)

BMI: Body mass index; CEA: Carcinoembryonic antigen.

Model diagnostic performance

On the independent test set, the model achieved an overall predictive AUC of 0.907 for T staging and 0.912 for N staging (Table 2). To evaluate prediction consistency with the pathological gold standard, we calculated WK coefficients as follows: WK value = 0.862 (95%CI: 0.821-0.903) for T staging and 0.874 (95%CI: 0.835-0.913) for N staging during validation; WK value = 0.841 (95%CI: 0.792-0.890) for T stages and 0.858 (95%CI: 0.811-0.905) for N stage on the independent test set. The data support good model-gold standard agreement.

Table 2 Diagnostic performance of the prediction model for the development and independent test cohorts.

		AUC (95%CI)	Sensitivity	Specificity	Accuracy	Weighted Kappa (95%CI)
T staging
T1 vs T2-4	Development cohort	0.868 (0.747-0.988)	0.769	0.966	0.941
	Independent test cohort	0.898 (0.809-0.959)	0.864	0.932	0.925
T2 vs others	Development cohort	0.934 (0.867-1.000)	0.880	0.987	0.961
	Independent test cohort	0.899 (0.832-0.952)	0.860	0.937	0.917
T3 vs others	Development cohort	0.958 (0.918-0.999)	0.935	0.982	0.961
	Independent test cohort	0.896 (0.855-0.946)	0.813	0.979	0.910
T4 vs others	Development cohort	0.927 (0.849-1.000)	0.889	0.964	0.951
	Independent test cohort	0.949 (0.888-0.982)	0.933	0.964	0.959
Total	Development cohort	0.888 (0.802-0.974)
	Development cohort	0.888 (0.802-0.974)				0.862 (0.821-0.903)
	Independent test cohort	0.907 (0.862-0.954)				0.841 (0.792-0.890)
N staging
N0 vs others	Development cohort	0.868 (0.801-0.935)	0.911	0.826	0.873
	Independent test cohort	0.917 (0.807-1.000)	1.000	0.833	0.923
N1 vs others	Development cohort	0.847 (0.767-0.927)	0.806	0.887	0.863
	Independent test cohort	0.915 (0.850-0.980)	0.830	1.000	0.946
N2 vs others	Development cohort	0.955 (0.888-1.000)	0.933	0.977	0.971
	Independent test cohort	0.900 (0.816-0.984)	0.800	1.000	0.966
Total	Development cohort	0.874 (0.828-0.920)
	Development cohort	0.874 (0.828-0.920)				0.874 (0.835-0.913)
	Independent test cohort	0.912 (0.867-0.956)				0.858 (0.811-0.905)

AUC: Area under the curve; CI: Confidence interval.

Model prediction performance for various T stages

In the test set, optimal model performance was observed when differentiating T4 tumors, which usually indicate most severe infiltration (AUC = 0.949, with 0.933 sensitivity and 0.964 specificity). For T2 and T3 discrimination, the AUC was 0.899 and 0.896, respectively.

Model prediction performance for various N stages

The model was effective in LNM (N1 and N2) identification, with test-set AUCs of 0.915 and 0.900, respectively. Notably, a 1.000 specificity was achieved for both N1 and N2. It indicates that cases judged by the model as “no such metastasis” are almost always correct, which is crucial for avoiding overtreatment. The prediction for N0 stage (no LNM) also performed well (AUC = 0.917, sensitivity = 1.000).

Ablation experimental results

To clarify how each component in the proposed dual-path context-aware fusion network contributed, we conducted ablation experiments on the independent test set. Following successive removal of the dual-path fusion module, cross-attention mechanism, and feature pyramid network while keeping other training settings unchanged, four model variants were built.

The AUC of the complete model for T- and N-staging reached 0.907 and 0.912, respectively, outperforming the variant models (Table 3). Following dual-path fusion module removal, a marked N-staging AUC (0.912-0.879) was noted, indicating that the removed module is crucial in capturing complementary features when assessing LNM. Similarly, a T-staging AUC reduction (0.907-0.881) was observed after excluding the cross-attention mechanism, underscoring the mechanism’s importance in modeling long-distance dependencies. Likewise, after removing the feature pyramid network, we found weakened overall classification accuracy of the model, particularly in identifying T3 and T4 tumors, possibly attributed to insufficient multi-scale feature representation. The above demonstrates each component’s positive contribution to the model’s diagnostic performance.

Table 3 Ablation experimental results on the independent test set.

Model configuration	T-stage AUC	N-stage AUC	T-staging accuracy	N-staging accuracy	Weighted Kappa coefficient
Single-path backbone network	0.852	0.861	0.835	0.871	0.784
Removal of dual-path fusion	0.873	0.879	0.859	0.891	0.812
Removal of cross-attention mechanism	0.881	0.886	0.864	0.902	0.828
Removal of feature pyramid network	0.892	0.898	0.876	0.918	0.846
Complete model (the method proposed herein)	0.907	0.912	0.898	0.949	0.883

AUC: Area under the curve.

Confusion matrix analysis

According to T-stage predictions on the independent test set (Table 4), the model achieved 86.4% accuracy for T1 tumors (19/22), misclassifying five T2 and four T3 cases as T1. The accuracy was 86.0% (37/43) for T2 tumors, with seven T3 cases under staged as T2. T3 tumors showed the highest accuracy at 81.3% (61/75), with two T1 and three T4 cases misclassified as T3. For T4 tumors, accuracy was 93.3% (28/30), with one T1 and one T2 case misclassified as T4.

Table 4 Confusion matrices (T staging).

Gold standard	T1 (n = 22)	T2 (n = 43)	T3 (n = 75)	T4 (n = 30)	Total
Prediction model
T1	19	5	4	1	29
T2	0	37	7	1	45
T3	2	0	61	0	63
T4	1	1	3	28	33
Total	22	43	75	30	170
Accuracy (%)	86.4	86.0	81.3	93.3

Table 5 presents the model’s N-stage predictions. Classification accuracy for N0 reached 100% (93/93), with no misclassification. For N1, accuracy was 82.4% (42/51), with nine cases underestimated as N0. For N2, accuracy was 76.9% (20/26), with six underestimated as N0.

Table 5 Confusion matrices (N staging).

Gold standard	N0 (n = 93)	N1 (n = 51)	N2 (n = 26)	Total
Prediction model
N0	93	9	6	108
N1	0	42	0	42
N2	0	0	20	20
Total	93	51	26	170
Accuracy (%)	100.0	82.4	76.9

Comparison of diagnostic performance under static image conditions

As shown in Table 6, for T-stage diagnosis, the model demonstrated statistically superior accuracy (89.8%) than the physician group (81.9%; P = 0.043). ROC analysis showed a significantly higher AUC for the model (AUC = 0.907, 95%CI: 0.862-0.954) than for the physician group (AUC = 0.817, 95%CI: 0.757-0.873; P = 0.016). For N-stage diagnosis, model accuracy was 94.9%, higher than that of the physician group (83.9%; P = 0.038). The prediction model also achieved a significantly higher AUC (0.912, 95%CI: 0.867-0.956) than the physician group (AUC = 0.765, 95%CI: 0.680-0.857; P = 0.001). Overall, the predictive model demonstrated significantly greater accuracy and discriminative ability in assessing preoperative T and N staging than the physician group (Figure 2).

Figure 2 Receiver operating characteristic curves of the independent test set. A: T-staging receiver operating characteristic curve; B: N-staging receiver operating characteristic curve. AUC: Area under the curve.

Table 6 Diagnostic performance comparison (prediction model vs physician group) on the independent test set.

Comparison item	Prediction model	Physician group	P value
T-staging
Accuracy	0.898	0.819	0.043
AUC (95%CI)	0.907 (0.862-0.954)	0.817 (0.757-0.873)	0.016
N-staging
Accuracy	0.949	0.839	0.038
AUC (95%CI)	0.912 (0.867-0.956)	0.765 (0.680-0.857)	0.001

AUC: Area under the curve; CI: Confidence interval.

It should be noted that the sonographers based their diagnoses solely on static images, without simulating dynamic US scanning (e.g., probe angle adjustment, applying pressure, and multi-angle observation). Therefore, these results primarily reflect an AI-physician comparison regarding static-image information extraction ability.

DCA

The results, visualized in Figure 3A, showed that within the 40%-80% possibility threshold range, the prediction model achieved higher net benefits than “treat-all” and “treat-none” strategies in diagnosing T stages, demonstrating practical utility. When the possibility threshold was 10%-80%, the net benefit achieved in diagnosing N stages remained higher than the default strategies, with the maximum net benefit (0.13) achieved at a 15% threshold (Figure 3B). This indicates that for every 100 patients evaluated, this model can bring about a net clinical benefit equivalent to the additional correct identification and appropriate intervention of 13 high-risk patients.

Figure 3 Decision curve analysis graphs. A: T-staging decision curve analysis graph; B: N-staging decision curve analysis graph.

Analysis of typical case image features

To enhance model interpretability and validate misclassification analysis results, we reviewed representative cases from the independent test set (Figure 4): (1) Correct T1 tumor classification: A submucosa-confined tumor with intact and continuous serosa, accurately classified as T1. It suggests the model’s ability to accurately identify tumor invasion depth; (2) Correct T2 tumor classification: A muscularis propria-invading tumor with serosa integrity and no interruption, correctly classified as T2. It indicates that the model can accurately assess serosa layer integrity; (3) Misclassified case (T3 by pathology vs T2 by the model): Ill-defined tumor’s deep margins due to acoustic shadowing, hindering serosal involvement assessment and thus leading to under staging; and (4) Misclassified case (N2 by pathology vs N0 by the model): Metastatic lymph nodes with regular morphology, clear boundaries, and small short-axis diameters that resembled reactive lymphoid hyperplasia, thus causing stage underestimation. The above image feature analysis corroborates the misclassification distribution of the confusion matrix, further supporting the causal hypothesis proposed in the misclassification analysis.

Figure 4 Image feature analysis of typical cases. A: A correctly classified T1 tumor; B: A correctly classified T2 tumor; C: A misclassified case (T3 by pathology vs T2 by the model); D: A misclassified case (N2 by pathology vs N0 by the model).

DISCUSSION

In this study, an US-based dual-path DL model was developed and validated for preoperative TNM staging predictions in CRC. In the temporally independent test cohort, the model’s excellent diagnostic performance was supported by its higher accuracy and AUC compared to physicians with varying seniority. DCA further confirmed the model’s ability to provide greater net benefits than “treat-all” or “treat-none” strategies across a wide threshold range. Thus, the proposed AI model may be an objective and reliable tool to enhance preoperative US staging accuracy for CRC.

Importantly, the DL model achieved high accuracy in tumor invasion depth (T staging) and LNM (N staging) assessments, outperforming human experts. Accurate preoperative staging, the cornerstone of CRC treatment decisions, is critical to patient prognosis as it directly determines the feasibility of neoadjuvant therapy and the surgery scope[24-26]. TAUS and TRUS are extensively applied to clinical staging, given their convenience, radiationless, and real-time capability; nonetheless, heavy dependency on physician experience introduces significant inter-observer variability[27,28]. Our model, with its highly consistent diagnostic performance with the gold standard pathological staging, provides a standardized and reproducible assessment method. It may help achieve uniformity in staged assessment across various medical institutions, especially providing diagnostic assistance to primary care physicians and in resource-poor and constrained settings. Consequently, more patients may access precise staging-based standardized treatment, which aligns with the precision medicine concept.

Recent years have seen the increasing application and rapid development of AI technology in the image analysis of CRC. CT- or MRI-based DL models have been indicated to predict T/N staging and peritoneal metastasis with an AUC range of 0.88-0.93[29,30]. However, these modalities are costly or radiative. US-based AI staging, on the contrary, is radiation-free, cost-effective, and reproducible. Nonetheless, US-based AI staging remains in its infancy with multiple challenges to be addressed, like inherent speckle noises and imaging artifacts demanding greater model robustness[31-34]. Notably, little research has been conducted specifically on T staging, while most studies center on automated partitioning or tumor benign-malignant classification[35-38]. The following is the study’s key contributions: (1) By employing a dual-path fusion architecture to explicitly model local-global feature interactions, the proposed model shows superior performance over traditional single-path CNNs in complex US contexts[9]; and (2) A more comprehensive preoperative evaluation is made possibly by simultaneously and accurately predicting multi-category T (T1-T4) and N (N0-N2) stages.

The model’s performance advantages are likely attributed to several factors. First, a high-quality annotated dataset is fundamental. Through strict case selection criteria, double-blind annotation, and arbitration procedures, training label reliability can be ensured. Second, the innovative model architecture is central. The dual-path network captures both microscopic intratumoral characteristics (e.g., uneven echoes, irregular boundaries) and macroscopic tumor-tissue relationships (e.g., serosal surface, fat gap), closely aligning with the cognitive process of US physicians when interpreting images. The cross-attention mechanism enables the model to weigh these two kinds of information adaptively, so as to judge the depth of invasion (T stage) and the possibility of LNM (N stage) more accurately. This architecture design that simulates the clinical reasoning process may be the key to surpassing the traditional end-to-end classification model. Furthermore, the use of a temporally independent test set strengthens the robustness of the findings. Because this cohort was collected after model development and remained completely time-separated, it closely simulates real-world deployment and demonstrates excellent temporal generalization rather than overfitting to historical data.

Model misclassifications, as indicated by the confusion matrix, centered on T2/T3 confusion and N-stage underestimation. Through retrospective analysis of the ultrasonic images of the misclassified cases, we identified the following main reasons: T staging: (1) Inadequate Image quality (intestinal gas interference, acoustic shadowing, etc.) resulted in poorly defined serosa and underestimated T3 as T2; (2) The particularity of the anatomical location makes it difficult to clearly distinguish the boundaries between tumors seated in the anterior rectal wall or the sigmoid colon and the adjacent organs, which can cause an overestimation of the tumor stage; and (3) Atypical pathological features in some T3 tumors, presenting as connective tissue proliferation without typical serosal rupture sign, made differentiation from T2 tumors challenging. N staging: Misclassifications were all underestimations, showing an asymmetry characterized by high specificity (100%) but low sensitivity. This might be due to the fact that during training, a higher penalty weight was assigned to false positives in order to avoid over-treatment, but at the expense of some sensitivity. Clinically, this could lead to under-staging. The main cause of misclassifications was poor US imaging quality, attributed to the atypical morphology of the metastatic lymph nodes or deep-seated tumors. Dynamic thresholding or multimodal imaging can be considered for optimization in the future.

Recent years have seen the mushrooming of research on US image-based DL models for CRC. Yi et al[39] reported 94.66% T-staging accuracy on 1078 images using the Xception-MS network. Another CNN model demonstrated effectiveness in identifying T2 and T3 stages but showed limited performance for T1 and T4 tumors[40]. In a recent two-center study, the efficacy of multiple models in tumor type differentiation was verified, though without delving into staging predictions[41]. The present study demonstrates unique advantages in its dual-path context-aware fusion architecture, a relatively sufficient sample size (n = 850), a robust validation strategy (temporally independent test set), and a wide task scope (T and N staging).

Several limitations are present. First, as data were sourced from a single medical institution, consistencies in US devices, operator skills, and acquisition parameters may have introduced specific biases that potentially limit model generalizability across centers and platforms. External validation in multicenter, prospective studies is therefore necessary to assess robustness and support clinical translation. Second, although with a temporally independent validation cohort, the retrospective design makes it necessary to confirm the model’s role in informing decision-making and enhancing patient outcomes in real clinical workflow through prospective clinical trials. Third, the model, US image-based without integrating clinical indices (e.g., CEA level) or other imaging modalities (e.g., CT), may show weakened performance. Integrating multi-dimensional information is expected to build a more powerful preoperative comprehensive evaluation system for CRC. Future efforts should also be devoted to improving the model’s interpretability. During application, techniques, like gradient-weighted class activation mapping, can be used to visualize the image regions that the model’s decision focuses on, thus increasing clinicians’ trust and understanding depth regarding the AI “black box”. In this study, qualitative analysis of representative cases partially addressed this limitation by illustrating differences in image characteristics between correctly classified and misclassified cases. Future studies should incorporate technologies such as gradient-weighted class activation mapping to visualize the model’s key areas of concern to improve its clinical interpretability. And given that the sonographers made diagnoses based solely on static images in the man-machine comparison, without the core advantage of dynamic US scanning, the experimental results can not fully represent real-world clinical performance differences. Prospective studies incorporating real-time, AI-assisted dynamic analysis are needed to more accurately assess clinical utility.

CONCLUSION

An US-based dual-path DL model for preoperative CRC staging was developed and validated. The model exhibited marked superiority over the physician group in a temporally independent test set for both T and N staging, indicating promising clinical applicability. It offers a potential AI solution for objective and precise preoperative staging and may support more reasonable, individualized treatment planning.