Machine learning-based decision tool for selecting patients with idiopathic acute pancreatitis for endosonography to exclude a biliary aetiology

Table 1 Variables distribution at the Ludwig-Maximilians-Universität University Hospital Munich (identification cohort)

Variable	Microlithiasis (n = 47)	Other (n = 171)	Total (n = 218)	P value
Age (yr)				0.122
mean ± SD	59.1 ± 18.8	54.6 ± 17.1	55.6 ± 17.6
Range	30-92	24-90	24-92
Sex				0.474
Female	16/47 (34%)	68/171 (39.8%)	84/218 (38.5%)
Male	31/47 (66%)	103/171 (60.2%)	134/218 (61.5%)
Albumin				0.706
N-Miss	32/47 (68%)	90/171 (52.6%)	122/218 (55.9%)
LLN	2/47 (4.2%)	14/171 (8.1%)	16/218 (7.3%)
WL	13/47 (27.6%)	67/171 (39.1%)	80/218 (36.6%)
Alkaline phosphatase				0.667
N-Miss	1/47 (2.1%)	5/171 (2.9%)	6/218 (2.7%)
LLN	0/47 (0%)	1/171 (0.5%)	1/218 (0.4%)
ULN	22/47 (46.8%)	69/171 (40.3%)	91/218 (41.7%)
WL	24/47 (51.0%)	96/171 (56.1%)	120/218 (55.0%)
Total bilirubin				0.110
N-Miss	0/47 (0%)	2/171 (1.1%)	2/218 (0.9%)
ULN	23/47 (48.9%)	61/171 (35.6%)	84/218 (38.5%)
WL	24/47 (51.0%)	108/171 (63.1%)	132/218 (60.5%)
Calcium				0.033
N-Miss	20/47 (42.5%)	60/171 (35.0%)	80/218 (36.6%)
LLN	6/47 (12.7%)	7/171 (4.0%)	13/218 (5.9%)
ULN	0/47 (0%)	2/171 (1.1.%)	2/218 (0.9%)
WL	21/47 (44.6%)	102/171 (59.6%)	123/218 (56.4%)
Creatine kinase				0.073
N-Miss	29/47 (61.7%)	107/171 (62.5%)	136/218 (62.3%)
ULN	0/47 (0%)	10/171 (5.8%)	10/218 (4.5%)
WL	18/47 (38.2%)	54/171 (31.5%)	72/218 (33%)
CRP				0.391
ULN	37/47 (78.7%)	124/171 (72.5%)	161/218 (73.9%)
WL	10/47 (21.3%)	47/171 (27.5%)	57/218 (26.1%)
Total protein				0.743
N-Miss	30/47 (63.8%)	104/171 (60.8%)	134/218 (61.4%)
LLN	0/47 (0%)	1/171 (0.5%)	1/218 (0.4%)
ULN	3/47 (6.3%)	16/171 (9.3%)	19/218 (8.7%)
WL	14/47 (29.7%)	50/171 (29.2%)	64/218 (29.3%)
Erythrocytes				0.880
N-Miss	1/47 (2.1%)	0/171 (0%)	1/218 (0.4%)
LLN	14/47 (29.7%)	53/171 (30.9%)	67/218 (30.7%)
ULN	3/47 (6.3%)	8/171 (4.6%)	11/218 (5.0%)
WL	29/47 (61.7%)	110/171 (64.3%)	139/218 (63.7%)
Gamma-GT				0.108
N-Miss	0/47 (0%)	1/171 (0.5%)	1/218 (0.4%)
ULN	37/47 (78.7%)	113/171 (66%)	150/218 (68.8%)
WL	10/47 (21.3%)	57/171 (33.3%)	67/218 (30.7%)
AST/GOT				0.444
N-Miss	17/47 (36.1%)	51/171 (29.8%)	68/218 (31.1%)
ULN	21/47 (44.6%)	75/171 (43.8%)	96/218 (44.0%)
WL	9/47 (19.1%)	45/171 (26.3%)	54/218 (24.7%)
ALT/GPT				0.016
ULN	34/47 (72.3%)	90/171 (52.6%)	124/218 (56.9%)
WL	13/47 (27.7%)	81/171 (47.4%)	94/218 (43.1%)
Urea				0.429
N-Miss	31/47 (65.9%)	80/171 (46.7%)	111/218 (50.9%)
LLN	0/47 (0%)	7/171 (4%)	7/218 (3.2%)
ULN	3/47 (6.3%)	11/171 (6.4%)	14/218 (6.4%)
WL	13/47 (27.6%)	73/171 (42.6%)	86/218 (39.4%)
Hematocrit				0.304
N-Miss	1/47 (2.1%)	0/171 (0%)	1/218 (0.4%)
LLN	0/47 (0%)	2/171 (1.1%)	2/218 (0.9%)
ULN	41/47 (87.2%)	160/171 (93.5%)	201/218 (92.2%)
WL	5/47 (10.6%)	9/171 (5.2%)	14/218 (6.4%)
Haemoglobin				0.574
N-Miss	1/47 (2.1%)	0/171 (0%)	1/218 (0.4%)
LLN	12/47 (25.5%)	45/171 (26.3%)	57/218 (26.1%)
ULN	0/47 (0%)	4/171 (2.3%)	4/218 (1.8%)
WL	34/47 (72.3%)	122/171 (71.3%)	156/218 (71.5%)
INR				0.443
N-Miss	3/47 (6.3%)	9/171 (1.1%)	12/218 (5.5%)
ULN	8/47 (17.0%)	22/171 (12.8%)	30/218 (13.7%)
WL	36/47 (76.5%)	140/171 (81.8%)	176/218 (80.7%)
Potassium				0.270
N-Miss	9/47 (19.1%)	2/171 (1.1%)	11/218 (5%)
LLN	1/47 (2.1%)	7/171 (4%)	8/218 (3.6%)
ULN	0/47 (0%)	10/171 (5.8%)	10/218 (4.5%)
WL	37/47 (78.7%)	152/171 (88.8%)	189/218 (86.6%)
Serum creatinine				0.738
N-Miss	6/47 (12.7%)	0/171 (0%)	6/218 (2.7%)
LLN	1/47 (2.1%)	5/171 (2.9%)	6/218 (2.7%)
ULN	5/47 (10.6%)	29/171 (16.9%)	34/218 (15.5%)
WL	35/47 (74.4%)	137/171 (80.1%)	172/218 (78.8%)
LDH				0.020
N-Miss	7/47 (14.8%)	19/171 (11.1%)	26/218 (11.9%)
ULN	30/47 (63.8%)	83/171 (48.5%)	112/218 (51.8%)
WL	10/47 (21.2%)	69/171 (40.3%)	79/218 (36.2%)
Leukocytes				0.347
N-Miss	1/47 (2.1%)	0/171 (0%)	1/218 (0.4%)
LLN	1/47 (2.1%)	3/171 (1.7%)	4/218 (1.8%)
ULN	16/47 (34%)	80/171 (46.7%)	96/218 (44%)
WL	29/47 (61.7%)	88/171 (51.5%)	117/218 (53.6%)
Lipase				0.653
N-Miss	0/47 (0%)	1/171 (0.5%)	1/218 (0.4%)
LLN	0/47 (0%)	3/171 (1.7%)	3/218 (1.3%)
ULN	44/47 (93.6%)	157/171 (91.8%)	201/218 (92.2%)
WL	3/47 (6.4%)	10/171 (5.8%)	13/218 (5.9%)
MCH				0.498
N-Miss	1/47 (2.1%)	0/171 (0%)	1/218 (0.4%)
LLN	3/47 (6.3%)	20/171 (11.6%)	23/218 (10.5%)
ULN	4/47 (8.5%)	10/171 (5.8%)	14/218 (6.4%)
WL	39/47 (82.9%)	141/171 (82.4%)	180/218 (82.5%)
MCHC				0.108
N-Miss	1/47 (2.1%)	0/171 (0%)	1/218 (0.4%)
LLN	0/47 (0%)	13/171 (7.6%)	13/218 (5.9%)
ULN	1/47 (2.1%)	8/171 (4.6%)	9/218 (4.1%)
WL	45/47 (95.7%)	150/171 (87.7%)	195/218 (89.4%)
Triglycerides				0.004
N-Miss	27/47 (57.4%)	110/171 (64.3%)	137/218 (62.8%)
ULN	1/47 (2.1%)	24/171 (14%)	25/218 (11.4%)
WL	19/47 (40.4%)	37/171 (21.6%)	56/218 (25.6%)
RDW				0.329
N-Miss	5/47 (10.6%)	36/171 (21%)	41/218 (18.8%)
LLN	1/47 (2.1%)	11/171 (6.3%)	12/218 (5.5%)
ULN	5/47 (10.6%)	21/171 (12.2%)	26/218 (11.9%)
WL	36/47 (76.5%)	103/171 (60.2%)	139/218 (63.7%)
MCV				0.893
N-Miss	1/47 (2.1%)	0/171 (0%)	1/218 (0.4%)
LLN	4/47 (8.5%)	13/171 (7.6%)	17/218 (7.7%)
ULN	4/47 (8.5%)	12/171 (7.0%)	16/218 (7.3%)
WL	38/47 (80.8%)	146/171 (85.3%)	184/218 (84.4%)
Sodium				0.020
N-Miss	8/47 (17.0%)	1/171 (0.5%)	9/218 (4.1%)
LLN	1/47 (2.1%)	29/171 (16.9%)	30/218 (13.7%)
WL	38/47 (80.8%)	141/171 (82.4%)	179/218 (82.1%)
Quick’s value				0.479
N-Miss	3/47 (6.3%)	7/171 (4%)	10/218 (4.5%)
LLN	8/47 (17%)	23/171 (13.4%)	31/218 (14.2%)
ULN	20/47 (42.5%)	65/171 (38%)	85/218 (38.9%)
WL	16/47 (34%)	76/171 (44.4%)	92/218 (42.2%)
Thrombocytes				0.434
N-Miss	1/47 (2.1%)	0/171 (0%)	1/218 (0.4%)
LLN	8/47 (17%)	22/171 (12.8%)	30/218 (13.7%)
ULN	4/47 (8.5%)	26/171 (15.2%)	30/218 (13.7%)
WL	34/47 (72.3%)	123/171 (71.9%)	157/218 (72%)
TSH				0.567
N-Miss	27/47 (57.4%)	118/171 (69%)	145/218 (66.5%)
LLN	2/47 (4.2%)	4/171 (2.3%)	6/218 (2.7%)
ULN	4/47 (8.5%)	6/171 (3.5%)	10/218 (4.5%)
WL	14/47 (29.7%)	43/171 (25.1%)	57/218 (26.1%)
Bilirubin-urine				0.027
N-Miss	26/47 (55.3%)	102/171 (59.6%)	128/218 (58.7%)
Normal	8/47 (17%)	45/171 (26.3%)	53/218 (24.3%)
Abnormal	13/47 (27.6%)	24/171 (14.0%)	37/218 (16.9%)
Total protein-urine				0.231
N-Miss	26/47 (55.3%)	102/171 (59.6%)	128/218 (58.7%)
Normal	10/47 (21.3%)	43/171 (25.1%)	53/218 (24.3%)
Abnormal	11/47 (23.4%)	26/171 (15.2%)	37/218 (16.9%)
Ketones-urine				0.020
N-Miss	26/47 (55.3%)	106/171 (61.9%)	132/218 (60.5%)
Normal	21/47 (44.6%)	51/171 (29.8%)	72/218 (33%)
Abnormal	0/47 (0%)	14/171 (8.1%)	14/218 (6.4%)
Leukocytes-urine				0.162
N-Miss	26/47 (55.3%)	102/171 (59.6%)	128/218 (58.7%)
Normal	7/47 (14.8%)	35/171 (20.4%)	42/218 (19.2%)
Abnormal	14/47 (29.7%)	34/171 (19.8%)	48/218 (22%)
Specific gravity-urine				0.918
N-Miss	29/47 (61.7%)	113/171 (66%)	142/218 (65.1%)
mean ± SD	1018.33 ± 5.941	1018.103 ± 8.777	1018.158 ± 8.159
Range	1005.000-1025.000	1005.000-1030.000	1005.000-1030.000

All variables that were used for the final predictive model are listed. For laboratory or urine values, the first available value during the inpatient stay was used. For the investigated groups of microlithiasis-induced acute pancreatitis (n = 47) vs the group of pancreatitis induced by other aetiologies (n = 171), the variables were categorised as whether collected or not (N-Miss), in the case of laboratory values whether below the lower limit value, within the limit values, or above the upper limit value. For urine values, in addition to the rate of missing variables (N-Miss), it was categorised whether normal or abnormal. For the P value calculation using χ² test, variables with missing data shares of > 25% were not excluded. LLN: Lower limit value; WL: Within the limit value; ULN: Upper limit value; CRP: C-reactive protein; Gamma-GT: Gamma-glutamyl transpeptidase; AST: Aspartate aminotransferase; GOT: Glutamic oxalacetic transaminases; ALT: Alanine transaminase; GPT: Glutamic pyruvic transaminase; INR: International normalized ratio; LDH: Lactate dehydrogenase; MCH: Mean corpuscular hemoglobin; MCHC: Mean corpuscular hemoglobin concentration; RDW: Red blood cell distribution width; MCV: Mean corpuscular volume; TSH: Thyrotropin.

Table 2 Performance matrix (identification cohort vs validation cohort)

Accuracy	Sensitivity	Specificity	PPV	NPV
ID: 0.8361; 95%CI: 0.7191-0.9185	ID: 0.9792	ID: 0.3077	ID: 0.8393	ID: 0.800
VD: 0.7607; 95%CI: 0.673-0.8347	VD: 0.9630	VD: 0.3056	VD: 0.7573	VD: 0.7857

The performance values for the auto-machine-learning-based predictive model are listed. The identification cohort of the Ludwig-Maximilians-Universität in Munich University Hospital (ID) vs the data of the predictive models from the validation cohort with patient data from the University Hospital Göttingen and the Klinikum Rechts der Isar (VD). CI: Confidence interval; PPV: Positive predictive value; NPV: Negative predictive value.

Table 3 Variable distribution in the validation cohort (UMG + Technical University Munich)

Variable	Microlithiasis (n = 36)	Other (n = 81)	Total (n = 117)	P value
Age (yr)				0.162
mean ± SD	60.1 ± 18.4	55.3 ± 16.8	56.8 ± 17.4
Range	23-93	21-87	21-93
Sex				0.315
Female	12/36 (33.3%)	35/81 (43.2%)	47/117 (40.2%)
Male	24/36 (66.7%)	46/81 (56.8%)	70/117 (59.8%)
Alkaline phosphatase				0.032
N-Miss	1/36 (2.7%)	12/81 (14.8%)	13/117 (11.1%)
ULN	23/36 (63.8%)	30/81 (37%)	53/117 (45.2%)
WL	12/36 (33.2%)	39/81 (48.1%)	51/117 (43.5%)
Total bilirubin				0.003
ULN	21/36 (58.3%)	24/81 (29.6%)	45/117 (38.5%)
WL	15/36 (41.7%)	57/81 (70.4%)	72/117 (61.5%)
Creatine kinase				0.498
N-Miss	8/36 (22.2%)	32/81 (39.5%)	40/117 (34.1%)
ULN	5/36 (13.8%)	6/81 (7.4%)	11/117 (9.4%)
WL	23/36 (63.8%)	43/81 (53%)	66/117 (56.4%)
CRP				0.199
ULN	32/36 (88.9%)	64/81 (79%)	96/117 (88.8%)
WL	4/36 (11.1%)	17/81 (21%)	21/117 (17.9%)
Total protein				0.405
N-Miss	31/36 (86.1%)	73/81 (90.1%)	104/117 (88.8%)
WL	5/36 (13.8%)	8/81 (9.8%)	13/117 (11.1%)
Erythrocytes				0.650
N-Miss	0/36 (0%)	1/81 (1.2%)	1/117 (0.8%)
LLN	10/36 (27.7%)	25/81 (30.8%)	35/117 (29.9%)
ULN	4/36 (11.1%)	5/81 (6.1%)	9/117 (7.6%)
WL	22/36 (61.1%)	50/81 (61.7%)	72/117 (61.5%)
Gamma-GT				0.082
N-Miss	0/36 (0%)	2/81 (2.4%)	2/117 (1.7%)
ULN	32/36 (88.8%)	59/81 (72.8%)	91/117 (77.7%)
WL	4/36 (11.1%)	20/81 (24.6%)	24/117 (20.5%)
AST/GOT				0.079
N-Miss	0/36 (0%)	4/81 (4.9%)	4/117 (3.4%)
ULN	28/36 (77.8%)	47/81 (58%)	75/117 (64.1%)
WL	8/36 (22.2%)	30/81 (37%)	38/117 (32.4%)
ALT/GPT				0.052
N-Miss	0/36 (0%)	2/81 (2.4%)	2/117 (1.7%)
ULN	28/36 (77.8%)	43/81 (53%)	71/117 (60.6%)
WL	8/36 (22.2%)	30/81 (37%)	38/117 (32.4%)
Urea
N-Miss	7/36 (19.4%)	20/81 (24.6%)	27/117 (23%)
LLN	13/36 (36.1%)	41/81 (50.6%)	54/117 (46.1%)
ULN	0/36 (0%)	1/81 (1.2%)	1/117 (0.8%)
WL	16/36 (44.4%)	19/81 (23.4%)	35/117 (29.9%)
Hematocrit				< 0.001
N-Miss	0/36 (0%)	1/81 (1.2%)	1/117 (0.8%)
ULN	36/36 (100%)	80/81 (98.7%)	116/117 (99.1%)
Haemoglobin				0.725
N-Miss	0/36 (0%)	1/81 (1.2%)	1/117 (0.8%)
LLN	9/36 (25%)	18/81 (22.2%)	27/117 (23%)
ULN	3/36 (8.3%)	4/81 (4.9%)	7/117 (5.9%)
WL	32/36 (88.9%)	64/81 (79.0%)	96/117 (82%)
INR				0.440
ULN	2/36 (5.5%)	8/81 (9.8%)	10/117 (8.5%)
WL	34/36 (94.4%)	73/81 (90.1%)	107/117 (91.4%)
Potassium				0.985
LLN	2/36 (5.5%)	4/81 (4.9%)	6/117 (5.1%)
ULN	1/36 (2.7%)	2/81 (2.4%)	3/117 (2.5%)
WL	33/36 (91.6%)	75/81 (92.5%)	108/117 (92.3%)
Serum creatinine				0.909
LLN	2/36 (5.5%)	6/81 (7.4%)	8/117 (6.8%)
ULN	7/36 (19.4%)	14/81 (17.2%)	21/117 (17.9%)
WL	27/36 (75%)	61/81 (75.3%)	88/117 (75.2%)
LDH				0.018
N-Miss	6/36 (16.6%)	28/81 (34.5%)	34/117 (29%)
ULN	26/36 (72.2%)	33/81 (40.7%)	59/117 (50.4%)
WL	4/36 (11.1%)	20/81 (24.6%)	24/117 (20.5%)
Leukocytes				0.143
N-Miss	0/36 (0%)	1/81 (1.2%)	1/117 (0.8%)
LLN	1/36 (2.7%)	0/81 (0%)	1/117 (0.8%)
ULN	16/36 (44.4%)	47/81 (58%)	63/117 (53.8%)
WL	19/36 (52.7%)	33/81 (40.7%)	52/117 (44.4%)
Lipase				0.237
N-Miss	0/36 (0%)	2/81 (2.4%)	2/117 (1.7%)
ULN	32/36 (88.9%)	75/81 (92.5%)	107/117 (91.4%)
WL	4/36 (11.1%)	4/81 (4.9%)	8/117 (6.8%)
MCV				0.315
N-Miss	0/36 (0%)	1/81 (1.2%)	1/117 (0.8%)
LLN	3/36 (8.3%)	7/81 (8.6%)	10/117 (8.5%)
ULN	1/36 (2.7%)	9/81 (11.1%)	10/117 (8.5%)
WL	32/36 (88.9%)	64/81 (79%)	96/117 (82%)
Triglycerides				0.582
N-Miss	26/36 (72.2%)	43/81 (53%)	69/117 (58.9%)
ULN	3/36 (8.3%)	15/81 (18.5%)	18/117 (15.3%)
WL	7/36 (19.4%)	23/81 (28.3%)	30/117 (25.6%)
RDW
N-Miss	36/36 (100%)	81/81 (100%)	117/117 (100%)
False	0/36 (0%)	0/81 (0%)	0/117 (0%)
True	0/36 (0%)	0/81 (0%)	0/117 (0%)
Sodium				0.154
LLN	2/36 (5.5%)	12/81 (14.8%)	14/117 (11.9%)
WL	34/36 (94.4%)	69/81 (85.1%)	103/117 (88%)
Quick’s value				0.130
LLN	2/36 (5.5%)	6/81 (7.4%)	8/117 (6.8%)
ULN	13/36 (36.1%)	44/81 (54.3%)	57/117 (48.7%)
WL	21/36 (58.3%)	31/81 (38.2%)	52/117 (44.4%)
Thrombocytes				0.627
N-Miss	22/36 (61.1%)	51/81 (62.9%)	73/117 (62.3%)
LLN	2/36 (5.5%)	3/81 (3.7%)	5/117 (4.2%)
ULN	2/36 (5.5%)	2/81 (2.4%)	4/117 (3.4%)
WL	10/36 (27.7%)	25/81 (30.8%)	35/117 (29.9%)
TSH				0.773
N-Miss	6/36 (16.6%)	13/81 (16%)	19/117 (16.2%)
LLN	0/36 (0%)	1/81 (1.2%)	1/117 (0.8%)
ULN	1/36 (2.7%)	3/81 (3.7%)	4/117 (3.4%)
WL	29/36 (80.5%)	64/81 (79%)	93/117 (79.4%)

All variables that were used for the final predictive model are listed. For laboratory or urine values, the first available value during the inpatient stay was used. For the investigated groups of microlithiasis-induced acute pancreatitis (n = 36) vs the group of pancreatitis induced by other aetiologies (n = 81), the variables were categorised as whether collected or not (N-Miss), in the case of laboratory values whether below the lower limit value, within the limit values, or above the upper limit value. For the P value calculation using χ² test, variables with missing data shares of > 25% were not excluded. LLN: Lower limit value; WL: Within the limit value; ULN: Upper limit value; CRP: C-reactive protein; Gamma-GT: Gamma-glutamyl transpeptidase; AST: Aspartate aminotransferase; GOT: Glutamic oxalacetic transaminases; ALT: Alanine transaminase; GPT: Glutamic pyruvic transaminase; INR: International normalized ratio; LDH: Lactate dehydrogenase; MCH: Mean corpuscular hemoglobin; MCHC: Mean corpuscular hemoglobin concentration; RDW: Red blood cell distribution width; MCV: Mean corpuscular volume; TSH: Thyrotropin.