Development and clinical application of an ultrasound-based deep learning model for preoperative staging of colorectal cancer

Table 1 Baseline characteristic comparison (development vs temporally independent test cohort), mean ± SD/n (%)

Characteristics	Development cohort (n = 680)	Independent test cohort (n = 170)	Statistic	P value
Age (years)	62.56 ± 10.70	63.24 ± 11.44	0.918	0.359
Sex (male/female)	375/305	95/75	0.030	0.863
BMI (kg/m2)	23.5 ± 3.2	23.8 ± 3.4	t = 1.06	0.291
Ileus			0.803	0.370
Yes	68 (10.0)	21 (12.4)
No	612 (90.0)	149 (87.6)
Tumor location
Colon cancer	425 (62.5)	105 (61.8)	χ2 test	0.861
Rectal cancer	255 (37.5)	65 (38.2)
Maximum tumor diameter (cm)	4.2 ± 1.8	4.3 ± 1.7	t-test	0.512
Preoperative CEA (≥ 5 ng/mL)	306 (45.0)	77 (45.3)	0.005	0.945
T-staging
T1	85 (12.5)	22 (12.9)	0.051	0.997
T2	170 (25.0)	43 (25.3)
T3	306 (45.0)	75 (44.1)
T4	119 (17.5)	30 (17.6)
N-staging
N0	374 (55.0)	93 (54.7)	0.020	0.995
N1	204 (30.0)	51 (30.0)
N2	102 (15.0)	26 (15.3)
Pathological differentiation			0.203	0.652
High/moderate differentiation	476 (70.0)	122 (71.8)
Poor differentiation/mucinous adenocarcinoma	204 (30.0)	48 (28.2)
Image quality score
3 points (acceptable)	204 (30.0)	51 (30.0)	0.000	1.000
4 points (good)	340 (50.0)	85 (50.0)
5 points (excellent)	136 (20.0)	34 (20.0)

BMI: Body mass index; CEA: Carcinoembryonic antigen.

Table 2 Diagnostic performance of the prediction model for the development and independent test cohorts

		AUC (95%CI)	Sensitivity	Specificity	Accuracy	Weighted Kappa (95%CI)
T staging
T1 vs T2-4	Development cohort	0.868 (0.747-0.988)	0.769	0.966	0.941
	Independent test cohort	0.898 (0.809-0.959)	0.864	0.932	0.925
T2 vs others	Development cohort	0.934 (0.867-1.000)	0.880	0.987	0.961
	Independent test cohort	0.899 (0.832-0.952)	0.860	0.937	0.917
T3 vs others	Development cohort	0.958 (0.918-0.999)	0.935	0.982	0.961
	Independent test cohort	0.896 (0.855-0.946)	0.813	0.979	0.910
T4 vs others	Development cohort	0.927 (0.849-1.000)	0.889	0.964	0.951
	Independent test cohort	0.949 (0.888-0.982)	0.933	0.964	0.959
Total	Development cohort	0.888 (0.802-0.974)
	Development cohort	0.888 (0.802-0.974)				0.862 (0.821-0.903)
	Independent test cohort	0.907 (0.862-0.954)				0.841 (0.792-0.890)
N staging
N0 vs others	Development cohort	0.868 (0.801-0.935)	0.911	0.826	0.873
	Independent test cohort	0.917 (0.807-1.000)	1.000	0.833	0.923
N1 vs others	Development cohort	0.847 (0.767-0.927)	0.806	0.887	0.863
	Independent test cohort	0.915 (0.850-0.980)	0.830	1.000	0.946
N2 vs others	Development cohort	0.955 (0.888-1.000)	0.933	0.977	0.971
	Independent test cohort	0.900 (0.816-0.984)	0.800	1.000	0.966
Total	Development cohort	0.874 (0.828-0.920)
	Development cohort	0.874 (0.828-0.920)				0.874 (0.835-0.913)
	Independent test cohort	0.912 (0.867-0.956)				0.858 (0.811-0.905)

AUC: Area under the curve; CI: Confidence interval.

Table 3 Ablation experimental results on the independent test set

Model configuration	T-stage AUC	N-stage AUC	T-staging accuracy	N-staging accuracy	Weighted Kappa coefficient
Single-path backbone network	0.852	0.861	0.835	0.871	0.784
Removal of dual-path fusion	0.873	0.879	0.859	0.891	0.812
Removal of cross-attention mechanism	0.881	0.886	0.864	0.902	0.828
Removal of feature pyramid network	0.892	0.898	0.876	0.918	0.846
Complete model (the method proposed herein)	0.907	0.912	0.898	0.949	0.883

AUC: Area under the curve.

Table 4 Confusion matrices (T staging)

Gold standard	T1 (n = 22)	T2 (n = 43)	T3 (n = 75)	T4 (n = 30)	Total
Prediction model
T1	19	5	4	1	29
T2	0	37	7	1	45
T3	2	0	61	0	63
T4	1	1	3	28	33
Total	22	43	75	30	170
Accuracy (%)	86.4	86.0	81.3	93.3

Table 5 Confusion matrices (N staging)

Gold standard	N0 (n = 93)	N1 (n = 51)	N2 (n = 26)	Total
Prediction model
N0	93	9	6	108
N1	0	42	0	42
N2	0	0	20	20
Total	93	51	26	170
Accuracy (%)	100.0	82.4	76.9

Table 6 Diagnostic performance comparison (prediction model vs physician group) on the independent test set

Comparison item	Prediction model	Physician group	P value
T-staging
Accuracy	0.898	0.819	0.043
AUC (95%CI)	0.907 (0.862-0.954)	0.817 (0.757-0.873)	0.016
N-staging
Accuracy	0.949	0.839	0.038
AUC (95%CI)	0.912 (0.867-0.956)	0.765 (0.680-0.857)	0.001

AUC: Area under the curve; CI: Confidence interval.