Development and prospective validation of a machine learning model to predict mortality in cirrhosis with esophageal variceal bleeding

Table 1 Baseline characteristics, n (%)/mean ± SD

Characteristic	Retrospective (n = 94)	Prospective (n = 24)
Age	56.76 ± 9.76	58.5 ± 9.1
Sex male	73 (77.7)	19 (79.2)
Race
White	7 (16.7)	14 (58.3)
Black	2 (4.8)	7 (29.2)
Other	33 (78.6)	3 (12.5)
Etiology
Alcohol	58 (61.7)	14 (58.3)
Alcohol + HCV	13 (13.8)	3 (12.5)
HCV	17 (18.1)	6 (25.0)
Hepatorenal syndrome yes	18 (19.1)	2 (8.3)
Medication use
Omeprazole yes	28 (29.8)	6 (25.0)
Spironolactone yes	15 (16.0)	7 (29.2)
Furosemide yes	15 (16.0)	6 (25.0)
Propranolol yes	23 (24.5)	10 (41.7)
Clinical parameters
Dialysis yes	2 (2.1)	0 (0.0)
Portal vein thrombosis yes	3 (3.2)	2 (8.3)
Ascites yes	57 (60.6)	7 (33.3)
Hepatocellular carcinoma yes	9 (9.6)	3 (12.5)
Laboratory values
Albumin	2.66 ± 0.59	2.7 ± 1.0
Total bilirubin	3.30 ± 4.00	6.7 ± 11.1
Direct bilirubin	2.24 ± 3.14	5.1 ± 9.1
INR	1.62 ± 0.66	1.6 ± 0.4
Creatinine	1.31 ± 1.13	1.2 ± 0.7
Platelets	102892.47 ± 62639.32	117734.2 ± 119141.2
Other parameters
Varices yes	76 (98.7)	23 (95.8)
Active bleeding yes	12 (15.8)	5 (20.8)
1-year death yes	48 (51.1)	8 (33.3)

Baseline characteristics of patients with cirrhosis with acute esophageal variceal bleeding in the development (2010-2016) and prospective validation (2017-2018) cohorts. Laboratory values are presented in conventional units. Percentages were calculated excluding missing values. HCV: Hepatitis C virus; INR: International normalized ratio.

Table 2 Prediction using machine learning

Model	Sensitivity (recall)	Specificity	Accuracy
Logistic regression	0.67	0.71	0.69
Random forest	0.80	0.86	0.83
SVM	0.73	0.64	0.69
Naive bayes	0.93	0.21	0.59
Decision tree	0.53	0.79	0.66
MLP	0.67	0.86	0.76

Comparison of model performance metrics across different machine learning algorithms in the development cohort. Sensitivity represents the true-positive rate, specificity the true-negative rate, and accuracy the overall correct classification rate. MLP: Multilayer perceptron; SVM: Support vector machine.

Table 3 Performance metrics for the random forest model in internal and prospective validation, mean ± SD

Metric	Internal validation	Prospective validation
AUC	0.715 ± 0.106	0.927 ± 0.053
Accuracy	0.688 ± 0.089	0.829 ± 0.078
Recall	0.752 ± 0.127	0.867 ± 0.126
F1 score	0.657 ± 0.109	0.760 ± 0.120
Brier score	0.218 ± 0.038	0.175 ± 0.016

Internal validation metrics were derived from bootstrap resampling (1000 iterations). Prospective validation was performed on an independent cohort (n = 24). AUC: Area under the curve.

Table 4 Performance comparison of the random forest model vs traditional clinical scoring systems for predicting 1-year mortality in patients with cirrhosis with acute esophageal variceal bleeding (retrospective cohort, n = 97)

Model	AUC (95%CI)	Sensitivity	Specificity	Accuracy	Brier score
Random forest	0.915 (0.856-0.961)	0.800	0.860	0.830	0.124
MELD-Na	0.742 (0.651-0.823)	0.688	0.717	0.702	0.186
MELD	0.726 (0.634-0.809)	0.667	0.696	0.681	0.194
Child-Pugh	0.685 (0.591-0.771)	0.625	0.674	0.649	0.217
Glasgow-Blatchford	0.598 (0.502-0.690)	0.542	0.609	0.574	0.248

A Brier score ranges from 0 to 1, with lower values indicating better calibration (0 = perfect calibration). Random forest model used 100 trees, bootstrap aggregation, and isotonic calibration. P < 0.001 compared to random forest model (DeLong test). All metrics except area under the receiver operating characteristic curve were calculated at the optimal threshold determined by maximizing the Youden index (sensitivity + specificity - 1). Confidence intervals (CIs) for area under the receiver operating characteristic curve were calculated using 1000 bootstrap iterations. AUC: Area under the curve; MELD: Model for end-stage liver disease; MELD-Na: MELD-sodium.