Multivariable prognostic models for post-hepatectomy liver failure: An updated systematic review

Table 1 Characteristics of studies included for analysis, n (%)

Ref.	Year	Country	IF	Indication(s) for resection	Sample size (training set)	Study design/sites	Outcome(s)	Event	Predictors in final model (number and list)	EPV	Competitor(s)
Model development only
Fang et al[21]	2021	China	5.738	HCC (BCLC criteria)	378	RC/S	PHLF-ISGLS	163 (43.1)	6: Cirrhosis; PT; tumor size; ICG-R 15%; blood loss; APRI	27.2	CP score, ALBI score, MELD score
Meng et al[22]	2023	China	2.808	HCC (major liver resection)	971	RC/S	PHLF-ISGLS	183 (18.8)	7: Age; BMI; preoperative ascites; intraoperative blood loss; TBIL; spleen volume-to-platelet ratio; prealbumin	26.1	ALBI score, MELD score
Peng et al[23]	2019	China	3.752	HCC	164	RC/S	PHLF-ISGLS	23 (14.6)	3: PSR; HIO; major hepatectomy (resection of 3 Couinaud’s segments or more)	7.7	NR
Shi et al[24]	2021	China	3.189	HCC (open hepatectomy)	767	RC/M	"50-50" criteria	102 (13.3)	6 (postoperative): Cirrhosis; PH; ALBI score; APRI score; major hepatectomy; intraoperative blood transfusion	17	ALBIAPRICP score, MELD scores
Choi et al[25]	2020	Korea	3.752	HCC	1174	RC/S	INR ≥ 1.5 or TBIL ≥ 2.9 mg/dL on or after POD 5	100 (8.5)	7: Sex; age; major resection; PLT; Alb; PT; ICG R15%	14.3	Previous scoring systems, no specified
Xu et al[26]	2021	China	3.421	HCC	258	RC/S	PHLF-ISGLS (grade A-B)	92 (35.66)	4: Bilirubin; PCT; Alb; PH	23	PALBI score, ALBI score, MELD score
Mai et al[27]	2020	China	4.348	HCC (hemihepatectomy)	265	RC/S	PHLF-ISGLS (grade B-C)	66 (24.9)	5: PLT count; PT; TBIL; AST; standardized FLR (sFLR)	13.2	CP grade, MELD score, ALBI score, FIB-4 APRI
Yugawa et al[28]	2022	Japan	3.253	HCC	451	RC/M	PHLF-ISGLS (grade B-C)	30 (6.7)	4: APRI; MELD score; operating time (minute); intraoperative blood loss (mL)	7.5	Each of these final individual predictors
Zhu et al[29]	2020	China	5.374	HCC (major liver resection with HBV)	101	RC/S	Encephalopathy with hyperbilirubinemia, TBIL > 4.1 mg/dL without an obstruction or bile leak, INR > 2.5, and ascites (drainage > 500 mL/day)	15 (14.9)	2: ICG-R15%Radiomics scores = -4.712031 - 1.529694 × 10-4 × minimum intensity + 5.788767 × uniformity + 7.658610 × energy - 3.207572 × 10-9 × cluster prominence (GLCM) - 1.566187 × 10-6 × minimum intensity (GLCM)	7.5	Clinical prediction model radiomics signature
Lee et al[30]	2020	Korea	2.885	PHCC (major liver resection with bile duct resection)	348	RC/S	PHLF-ISGLS (grade B-C)	40 (11.4)	5: Sex; Alb; preoperative cholangitis; FLRV/body weight; portal vein resection	8	NR
Cho et al[31]	2022	Korea	4.321	HCC	160	RC/S	PHLF-ISGLS (grade B-C)	24 (15)	5: MRE-LS; low serum albumin; major hepatic resection; higher ALBI score; higher serum AFP	4.8	Single biomarker(s) MRE-LSALBI score, ICG R15%, FIB-4 APRI
Li et al[32]	2021	China	3.297	HCC, ICC, hepatic hemangioma, intrahepatic cholelithiasis, metastatic hepatoma, other diseases	1080	RC/S	PHLF-ISGLS	622 (57.6)	9: Age; gender; PLT; Cr; GGT; fibrinogen; thrombin time; HBe antigen; number of resected liver segments	69.1	NR
Zhang et al[33]	2022	China	4.964	HCC	1081	RC/S	PHLF-ISGLS	113 (10.5)	7: BMI; ICG-R15%; EHBF; INR; tumor size; operation method; HIO time	16.1	NR
Wang et al[34]	2022	China	4.964	HCC	416	RC/M	"50-50" criteria	96 (23.08)	5: PH; extent of resection; ALT; TBIL; PLT	19.2	FIB-4 score, APRI score, ALBI score, MELD score
Prodeau et al[35]	2019	France	30.083	HCC	343	RC/M	PHLF-ISGLS (grade B-C)	132 (38.5)	4 (postoperative): Non converted laparoscopic liver resection; RTLV; PLT count; blood loss	33	NR
Xiang et al[36]	2021	China	3.253	HCC (huge)	131	RC/S	PHLF-ISGLS	41 (31.3)	3: Rad score; MELD score; extent of resection	13.7	CP score, MELD score, ALBI score
Zhong et al[37]	2021	China	4.501	HCC	383	RC/S	PHLF-ISGLS (grade B-C)	59 (15.4)	6: Cirrhosis; major hepatectomy; ascites; intraoperative blood loss (mL); PALBI score; FIB-4 score	9.8	CP score, MELD score, ALBI score, APRI score, PALBI score, FIB-4 score
Chin et al[38]	2020	Singapore	3.282	HCCCRLM	472	RC/S	"50-50" criteria	22 (4.7)	3: ALBI index; ln (POD1 bilirubin/pre-op bilirubin); PT	7.3	NR
Morino et al[39]	2023	Japan	3.282	No limited to liver tumor	597	RC/S	PHLF-ISGLS (grade B-C)	42 (7.03)	3: Rem-ALPlat index; number of PMs; blood loss	14	Rem-ALPlat alone
Wang et al[40]	2022	China	5.783	HCC	612	RC/M	PHLF-ISGLS	137 (22.4)	5: PLT count; age; Cr; INR; AFP	27.4	ALBI FIB-4 APRI MELD CTP
Dhir et al[41]	2021	United States	3.253	HCC (partial lobectomy, right lobectomy, left lobectomy, and trisegmentectomy)	7376	RC/M	PHLF-ISGLS (grade B-C)	226 (3.1)	20: Age; BMI; sex; diabetes status; ascites; bleeding disorder; dyspnea; steroid; biliary stent; ASA score; neoadjuvant therapy; HBV or HCV; concurrent partial resections; biliary reconstruction; procedure type; preoperative sodium; preoperative Cr; preoperative Alb; preoperative bilirubin; preoperative INR	11.3	NR
Model development and external validation
Lei et al[42]	2022	China	5.738	HCC (major hepatectomy)	688	RC/M	PHLF-ISGLS (grade B-C)	93 (13.5)	5: Age; sex; TBIL; PT; CSPH	18.6	MELDALBI score
Xu et al[43]	2022	China	3.253	HCC (huge)	343	RC/M	PHLF-ISGLS (grade B-C)	52 (15.2)	5: MALBI grade; CP class; intraoperative blood loss; cirrhosis; INR	10.4	ALBI scores, CP score
Peng et al[44]	2022	China	7.034	HCC	121	RC/M	PHLF-ISGLS	48 (39.67)	3: CT-derived ECV; serum Alb; serum TBIL	16	ALBI score
Ye et al[45]	2020	China	4.638	HCC (positive for HBV)	900	RC/M	PHLF-ISGLS (grade B-C)	121 (13.5)	6: TBIL; PLT count; prealbumin; AST; PT; sFLR (%)	20.2	CP grade MELDALBIPALBIAPRI
Hobeika et al[46]	2022	France	11.782	HCC	305	RC/M	PHLF-ISGLS (grade B-C)	19 (6.2)	4: MELD score; FIB-4 score (HCV 0/1); liver surface nodularity score; future liver remnant ratio volume (%)	4.8	Based on pathological data and HVPG measurement (two invasive models): IB and IB + MELD score, model
Li et al[47]	2022	China	3.388	HCC	199	RC, PC/M	PHLF-ISGLS	46 (23.12)	6: Tumor number; PM; blood loss preoperative PLT; ascites; anticoagulants	7.7	Dasari et al[48] and Citterio et al[49]
Chen et al[50]	2021	China	5.738	HCC (hemihepatectomy)	111	RC/M	PHLF-ISGLS	56 (50.45)	3: Radiomics score (from 24 radiomics features); PLT count; tumor size	18.7	Clinical model Radiomics model
Shen et al[51]	2019	China	1.817	HCC	325	RC/M	PHLF-ISGLS	27 (8.3)	4: Serum TBIL; serum Cr; intraoperative hemorrhage; CSPH	6.8	MELD score, ALBI score
Ding et al[52]	2023	China	5.738	HCC	271	RC/M	PHLF-ISGLS (grade B-C)	156 (37.1) state: Total sample	4: ASA score; SMI; Child-Pugh score; MELD score	Cannot be calculated	NR
Xu et al[53]	2021	China	5.065	No limited to liver diseases	344	RC/M	PHLF-ISGLS	91 (26.5)	5: TBIL; INR; PLT count; extent of resection; blood loss	18.2	MELD score, ALBI score, PALBI score
Wang et al[54]	2021	China	13.787	HCC	1036	RC/M	PHLF-ISGLS (grade B-C)	105 (10.1)	6: TBIL; Alb; GGT; PT; CSPH; planned extent of resection	17.5	Child-Pugh score, MELD score, ALBI score, EASL recommended algorithm
External validation of pre-existing model
Guo et al[55]	2021	China	4.478	HCC (major hepatectomy)	745	RC/M	"50-50" criteria	103 (13.8)	1: Each of the following scores (ALBI; MELD; APRI; FIB4; PALBI; King’s score)	103	ALBI, MELD, APRI, FIB4, PALBI, King’s score
Noji et al[56]	2022	Japan	8.265	PHCC (major hepatectomy with extrahepatic bile duct resection)	254	RC/S	PHLF-ISGLS (grade B-C)	71 (27.95)	4: FLRV; jaundice at presentation; immediate preoperative bilirubin > 50 mmol/L (> 2.9 mg/dL); preoperative cholangitis	17.8	NR

IF: Impact factor; EPV: Events-per-predictor variable; HCC: Hepatocellular carcinoma; BCLC criteria: Barcelona Clinical Liver Cancer criteria; HBe: Hepatitis B e; HBV: Hepatitis B virus; HCV: Hepatitis C virus; PHCC: Perihilar cholangiocarcinoma; ICC: Intra-hepatic cholangiocarcinoma; CRLM: Colorectal cancer liver metastases; RC: Retrospective cohort; PC: Prospective cohort; S: Single-centre; M: Multi-centre; PHLF: Post-hepatectomy liver failure; ISGLS: International Study Group for Liver Surgery; INR: International normalized ratio; TBil: Total bilirubin; POD: Postoperative day; PT: Prothrombin time; ICG-R15%: Indocyanine green retention rate; APRI: Aspartate transaminase to platelet ratio index; BMI: Body mass index; PSR: Platelet to spleen stiffness ratio; HIO: Hepatic inflow occlusion; ALBI: Albumin-bilirubin grade; Alb: Albumin; PCT: Platelet crit; PH: Portal hypertension; PLT: Platelet; AST: Aspartate aminotransferase; sFLR: Standardized future liver remnant; MELD: Model for end stage liver disease; GLCM: Gray-level co-occurrence matrix; FLRV: Future liver remnant volume; sFLRV: Standard future liver remnant volume; MRE-LS: Magnetic resonance elastography-assessed liver stiffness; GGT: Gamma-glutamyl transferase; EHBF: Effective hepatic blood flow; ALT: Alanine transaminase; RTLV: Remnant to total liver volume; PALBI: Platelet-albumin-bilirubin; FIB-4: Fibrosis-4 score; Rem-ALPlat: 0.0334 × Rem + 0.0113 × (90 × Alb + Plt); PM: Pringle maneuver; Cr: Creatinine; AFP: Alpha-fetoprotein; ASA: American Society of Anesthesiologists; CSPH: Clinically significant portal hypertension; mALBI: Modifed albumin-bilirubin; ECV: Extracellular volume; SMI: Skeletal muscle index; CP: Child-Pugh; CTP: Child-Turcotte-Pugh; HVPG: Hepatic venous pressure gradient; IB: Invasive biomarkers; EASL: European Association for the Study of the Liver.

Table 2 Description of model development, performance, and evaluation within included studies

Ref.	Model development						Model performance			Model evaluation				Clinical effectiveness
	Purpose	Blind edpredictor assessment	Predictor selection for modelling	Predictor selection during modelling	Missing data	Statistical modelling method	Model calibration performed	Model discrimination assessment	AUC (95%CI) (training model)	Internalvalidity assessment	AUC (95%CI)	Externalvalidity assessment	AUC (95%CI)
Model development only
Fang et al[21]	Binary	NR	Univariate association	NR	NR	MLR	Yes, calibration plots	Yes, AUC and CI	0.845 (0.806-0.884)	Yes, split sample	0.854 (0.782-0.926)	NR	NR	No study
Meng et al[22]	Binary	NR	Univariate association	Stepwise	Yes	MLR	Yes, calibration plots	Yes, AUC only	0.697	Yes, bootstraps	0.668	NR	NR	No study
Peng et al[23]	Binary	NR	Univariate association	NR	NR	MLR	NR	Yes, AUC and CI	0.867 (0.790-0.943)	NR	NR	NR	NR	No study
Shi et al[24]	Binary	NR	Univariate association	Forward	Yes	MLR	NR	Yes, AUC and CI	0.844 (0.801-0.887)	NR	NR	NR	NR	Developed online calculators
Choi et al[25]	Binary	NR	A bootstrap resampling approach	Backward	Yes	MLR	Yes, calibration curve and HL test	Yes, AUC and CI	0.737 (0.687-0.787)	Yes, split sample	0.672 (0.577-0.767)	NR	NR	No study
Xu et al[26]	Binary	NR	NR	NR	Yes	Improved based on original model	NR	Yes, AUC and CI	0.772 (0.716-0.822)	NR	NR	NR	NR	No study
Mai et al[27]	Binary	NR	Univariate association	NR	Yes	ANN	Yes, calibration plots and HL test	Yes, AUC and CI	0.88 (0.836-0.925)	Yes, split sample	0.876 (0.801-0.950)	NR	NR	No study
Yugawa et al[28]	Binary	NR	Univariate association	NR	Yes	MLR	NR	Yes, AUC only	0.88	NR	NR	NR	NR	No study
Zhu et al[29]	Binary	NR	Univariate association and LASSO regression	Forward	NR	MLR	Yes, calibration curve and HL test	Yes, AUC and CI	0.894 (0.823-0.964)	NR	NR	NR	NR	No study
Lee et al[30]	Binary	NR	Univariate association	Backward	Yes	MLR	Yes, HL test	Yes, AUC and CIs	0.853 (0.802-0.904)	Yes, CV and bootstraps	0.852 (0.795-0.910)	NR	NR	No study
Cho et al[31]	Binary	NR	Univariate association with the Kaplan-Meier plots	Backward	Yes	Cox regression	NR	Yes, AUC with DeLong test	0.877 (0.805-0.948)	Yes, CV	0.8	NR	NR	No study
Li et al[32]	Binary	NR	Univariate association	Backward	Yes	MLR	Yes, calibration plots and HL test	Yes, AUC and CI	0.726 (0.696-0.760)	Yes, bootstraps	0.717 (0.663-0.770)	NR	NR	No study
Zhang et al[33]	Binary	NR	Univariate association	Forest algorithm	Yes	MLR	Yes, calibration plots	Yes, AUC and CI	0.773 (0.729-0.818)	NR	NR	NR	NR	No study
Wang et al[34]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration curve	Yes, AUC and CI	0.857 (0.789-0.925)	Yes, split sample	0.753 (0.696-0.809)	NR	NR	No study
Prodeau et al[35]	Binary	NR	Bivariate ordinal logistic regression model	Backward	Yes	MLR	Yes, Lipsitz and Pulkstenis-Robinson tests	Yes, AUC only	0.77	Yes, bootstraps	0.85	NR	NR	No study
Xiang et al[36]	Binary	NR	Univariate association and LASSO regression	NR	Yes	MLR	Yes, calibration plots and HL test	Yes, AUC and CI	0.842 (0.761-0.922)	Yes, split sample	0.863 (0.750-0.975)	NR	NR	No study
Zhong et al[37]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration plots	Yes, AUC and CI	0.832 (0.777-0.886)	Yes, split sample	0.803 (0.723-0.883)	NR	NR	No study
Chin et al[38]	Binary	NR	Univariate association	LASSO method	Yes	Penalized logistic regression	Yes, HL test	Yes, AUC only	0.823	NR	NR	NR	NR	No study
Morino et al[39]	Binary	NR	Univariate association	Stepwise	NR	MLR	NR	NR	0.877	NR	NR	NR	NR	No study
Wang et al[40]	Binary	NR	SHAP analysis	NR	Yes	ML	NR	Yes, AUC only	0.944 (0.924-0.964)	Yes, split sample	0.870 (0.791-0.950)	No external validity, just a split sample like Internal validity	NR	No study
Dhir et al[41]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration plots and HL test	Yes, AUC only	0.78	Yes, split sample	0.78	NR	NR	No study
Model development and external validation
Lei et al[42]	Binary	NR	The squares of the Spearman correlation coefficients	LASSO method	Yes	MLR	Yes, calibration curve	Yes, AUC with DeLong test	0.73 (0.69-0.76)	Yes, CV (10-fold)	0.73 (0.69-0.76)	Different patient population (other two hospitals)	0.72 (0.65-0.78)	No study
Xu et al[43]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration curve	Yes, AUC only	0.863 (0.812-0.914)	Yes, split sample	0.823 (0.737-0.909)	Different patient population	0.74 (0.624-0.856)	No study
Peng et al[44]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration curve and HL test	Yes, AUC with Delong test	0.828 (0.756-0.901)	NR	NR	Different patient population	0.821 (0.727-0.914)	No study
Ye et al[45]	Binary	NR	Univariate association	Forest algorithm	Yes	MLR	Yes, calibration plots	Yes, AUC and CI	0.868 (0.811-0.926)	Yes, split sample	0.868 (0.811-0.926)	Different patient population	0.82 (0.756-0.861)	Yes
Hobeika et al[46]	Binary	NR	Binomial logistic regressions	Forward/backward	Yes	MLR	Yes, calibration plots and HL test	Yes, AUC with DeLong test	0.77 (0.667, 0.872)	NR	NR	Different patient population	0.888 (0.809-0.968)	No study
Li et al[47]	Binary	NR	Univariate association	Forward	Yes	MLR	Yes, calibration curve	Yes, AUC only	0.911 (0.865-0.958)	NR	NR	Different time period	0.714 (0.697-0.902)	No study
Chen et al[50]	Binary	NR	Univariate association; Pearson’s correlation coefficients	NR	Yes	MLR	NR	Yes, AUC only	0.956 (0.955-0.962)	NR	NR	Different patient population	0.844 (0.833-0.886)	No study
Shen et al[51]	Binary	NR	Univariate association	NR	NR	MLR	Yes, calibration curve	Yes, AUC and CI	0.818 (0.735-0.901)	NR	NR	Different patient population	0.906 (0.833-0.979)	No study
Ding et al[52]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration curve and HL test	Yes, AUC only	0.91	Yes, split sample	0.82	Different patient population	0.89	No study
Xu et al[53]	Binary	NR	LASSO regression with 10-fold cross-validation	LASSO method	NR	MLR	Yes, calibration curve	Yes, AUC with Delong test	0.838 (0.790-0.885)	Yes, split sample	0.788 (0.693-0.884)	Different patient population	0.750 (0.632-0.868)	No study
Wang et al[54]	Binary Mortality + OS	NR	Univariate association	Backward	NR	MLR	Yes, calibration plots	Yes, AUC only	0.883 (0.852-0.915)	Yes, split sample	0.851	Different patient population	0.856	No study
External validation of pre-existing model
Guo et al[55]	Binary	NR	NR	NR	Yes	NR	Yes, loess-smoothed plots	Yes, AUC with DeLong test	NR	NR	NR	NR	0.64 (0.58-0.69); 0.58 (0.52-0.64); 0.59 (0.53-0.64); 0.57 (0.51-0.63); 0.57 (0.51-0.63); 0.61 (0.55-0.67)	No study
Noji et al[56]	Binary	NR	NR	NR	Yes	NR	NR	Yes, AUC only	NR	NR	NR	NR	0.62	No study

NR: Not reported; MLR: Multivariable logistic regression; ANN: Artificial neural network; ML: Machine learning; LASSO method: Least absolute contraction selection operator logistic regression; HL: Hosmer-Leme show test of goodness-of-fit; AUC: Area under the receiver operating characteristic curve; CV: Cross validation.

Table 3 Quality assessment of included studies using the prediction model risk of bias assessment tool

Ref.	Year (publication)	Risk of bias				Applicability			Overall assessment
		Participants	Predictors	Outcome	Analysis	Participants	Predictors	Outcome	Risk of bias	Applicability
Peng et al[23]	2019	L	H	L	H	L	L	L	H	L
Prodeau et al[35]	2019	L	L	L	H	L	L	L	H	L
Shen et al[51]	2019	L	L	L	H	L	L	L	H	L
Choi et al[25]	2020	L	L	L	H	L	L	L	H	L
Mai et al[27]	2020	L	L	L	H	L	L	L	H	L
Zhu et al[29]	2020	L	L	L	H	L	L	L	H	L
Lee et al[30]	2020	L	L	L	H	L	L	L	H	L
Chin et al[38]	2020	L	L	L	H	L	L	L	H	L
Ye et al[45]	2020	L	L	L	H	L	L	L	H	L
Fang et al[21]	2021	L	L	L	H	L	L	L	H	L
Shi et al[24]	2021	L	L	L	H	L	L	L	H	L
Xu et al[26]	2021	L	L	L	H	L	L	L	H	L
Cho et al[31]	2021	L	L	L	H	L	L	L	H	L
Li et al[32]	2021	L	L	L	H	L	L	L	H	L
Xiang et al[36]	2021	L	L	L	H	L	L	L	H	L
Zhong et al[37]	2021	L	L	L	H	L	L	L	H	L
Dhir et al[41]	2021	L	L	L	H	L	L	L	H	L
Chen et al[50]	2021	L	L	L	H	L	L	L	H	L
Xu et al[53]	2021	L	L	L	H	L	L	L	H	L
Wang et al[54]	2021	L	L	L	H	L	L	L	H	L
Guo et al[53]	2021	L	L	L	H	L	L	L	H	L
Yugawa et al[28]	2022	L	L	L	H	L	L	L	H	L
Zhang et al[33]	2022	L	L	L	H	L	L	L	H	L
Wang et al[34]	2022	L	L	L	H	L	L	L	H	L
Noji et al[56]	2022	L	L	L	H	L	L	L	H	L
Wang et al[40]	2022	L	L	L	H	L	L	L	H	L
Lei et al[42]	2022	L	L	L	H	L	L	L	H	L
Xu et al[43]	2022	L	L	L	H	L	L	L	H	L
Peng et al[44]	2022	L	L	L	H	L	L	L	H	L
Hobeika et al[46]	2022	L	L	L	H	L	L	L	H	L
Li et al[47]	2022	L	L	L	H	L	L	L	H	L
Meng et al[22]	2023	L	L	L	H	L	L	L	H	L
Morino et al[39]	2023	L	L	L	H	L	L	L	H	L
Ding et al[52]	2023	L	L	L	H	L	L	L	H	L

L: Low concern; H: High concern.