Preoperative risk modelling for oesophagectomy: A systematic review

Table 1 Development studies of preoperative multivariate models

Ref.	Period + number	Sample region	Operation	Characteristics	Source of data	Models tested	Outcomes tested
Schröder et al[29], 2006	1997-2002 (126)	Germany	TT	N: 46/126; H: 68 AC/54 SCC	Single centre	Cologne score	Morbidity
Steyerberg et al[30], 2006	1980-2002 (3592)	Unites States/Netherlands	TT/TH	N: 878/3592; H: 2118 AC/1307 SCC	SEER database + two centres	Rotterdam score	Mortality
Ra et al[32], 2008	1997-2003 (1172)	United States	TT/TH	N: N/A; H: N/A	SEER database	Philadelphia score	Mortality
Lagarde et al[31], 2008	1993-2005 (663)	Netherlands	TT/TH	N: 114/663; H: 476 AC/187 SCC	Single centre	Amsterdam score	Morbidity
Wright et al[33], 2009	2002-2007 (2315)	United States	TT/TH/MIE	N: 1016/2315; H: N/A	STSGTS database (164 centres)	Original STS model	Major morbidity + mortality
Ferguson et al[34], 2011	1980-2009 (516)	United States	TT/TH/hybrid/MIE	N: 167/516; H: 261 AC/137 SCC	Single centre	Ferguson score	Respiratory complications
Takeuchi et al[35], 2014	2011-2011 (5354)	Japan	Not stated	N: 1268/5354; H: N/A	National database	Tackeuchi model	Mortality
Filip et al[36], 2015	2008-2012 (167)	Italy	TT/TH/MIE	N: 131/167; H: 105 AC/62 SCC	Single centre	PNI-multivariate: PNI; CCI; ACCI; POSSUM; Amsterdam score	Morbidity + major morbidity (PNI-multivariate only)
Raymond et al[37], 2016	2011-2014 (4321)	United States	TT/TH/MIE	N: 2930/4321; H: N/A	STSGTS database (164 centres)	Revised STS model	Major morbidity + mortality
Reeh et al[38], 2016	1994-2007 (498)	Germany	TT/TH	N: 0/498; H: 253 SCC/245 AC	Single centre	PER score	Morbidity + mortality
Saito et al[39], 2019	2007-2015 (90)	Japan	MIE	N: 29/90; H: 87 SCC/3 AC	Single centre	PPCS model	Major morbidity
Ohkura et al[40], 2020	2011-2012 (10826)	Japan	Not stated	N: 2717/10826;H: N/A	National database (4105 centres)	JNCD model	Anastomotic leak
Wan et al[41], 2022	2006-2017 (10602)	United States	Not reported	N: N/A; H: N/A	National (NSQIP)	RAI-revised (CC): RAI-A; RAI-revised; 5 Factor MFI	Morbidity + mortality

TT: Transthoracic; TH: Transhiatal; MIE: Minimally invasive esophagectomy; N: Neoadjuvant chemotherapy; H: Histopathology subtype; SCC: Squamous cell carcinoma; AC: Adenocarcinoma; ACCI: Age-adjusted charlson comorbidity index; CCI: Charlson comorbidty index; POSSUM: Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity; RAI-A: Administrative risk analysis index; MFI: Modified frailty index; STS: Society of Thoracic Surgeons Oesophagectomy Composite Score; PNI: Prognostic nutritional index; PPCS: Predicting postoperative complications score; JNCD: Japanese National Clinical Database; PER: Perioperative esophagectomy risk score.

Table 2 Validation studies of preoperative models

Ref.	Period + number	Sample region	Operation	Characteristics	Source of data	Models tested	Outcomes tested
Zingg et al[42], 2009	1990-2007 (346)	Switzerland/Australia	TT	N: 140/346; H: 259 AC/71 SCC	Two centres	Rotterdam score; Philadelphia score	Mortality
Grotenhuis et al[43], 2010	1991-2008 (777)	Netherlands	TT/TH	N: 221/777; H: N/A	Single centre	Amsterdam score	Morbidity
Bosch et al[44], 2011	1991-2007 (278)	Netherlands	TT	N: 10/278; H: 235 AC/43 SCC	Single centre	ACCI; CCI; O-POSSUM; P-POSSUM	Mortality
Ferguson et al[45], 2011	1980-2009 (514)	United States	TT/TH/hybrid/MIE	N: 167/514; H: 261 AC/137 SCC	Single centre	Amsterdam score	Morbidity + major morbidity
Filip et al[46], 2014	2004-2013 (43)	Romania	TT/TH	N: 22/43; H: 33 SCC/9 AC	Single centre	ACCI; CCI; POSSUM; O-POSSUM; P-POSSUM	Mortality
Yamana et al[47], 2015	2005-2013 (251)	Japan	TT/MIE	N: 150/251; H: N/A	Single centre	GNRI; PNI; E-PASS; POSSUM	Respiratory complications
Lindner et al[48], 2016	2005-2009 (94)	Germany	TT	N: 54/94; H: 94 AC/0 SCC	Single centre	Cologne score	Morbidity
Reinersman et al[49], 2016	2009-2012 (136)	United States	TT/TH/hybrid/MIE	N: 110/136; H: 118 AC/18 SCC	Single centre	Ferguson score	Respiratory complications
Xing et al[50], 2016	2008-2010 (217)	China	TT/TH	N: 0/217; H: 162 SCC/50 AC	Single centre	Ferguson score	Respiratory complications
Takeuchi et al[51], 2018	2000-2016 (438)	Japan	TT	N: 208/438; H: 398 SCC/27 AC	Single centre	Takeuchi model	Mortality
D’Journo et al[52], 2017	2004-2013 (1039)	France	TT/TH	N: 420/1039; H: N/A	National database	Rotterdam score	Mortality
Gray et al[53], 2020	2016-2018 (240)	United States	TT/TH/MIE	N: N/A; H: N/A	Single centre	NSQIP SRC	Morbidity
Peng et al[54], 2020	2012-2019 (218)	United States	MIE	N: 189/218; H: N/A	Single centre	NSQIP SRC	Morbidity + mortality
Ravindran et al[55], 2020	2013-2017 (100)	United States	TT	N: 87/100; H: 75 AC/21 SCC	Single centre	NSQIP SRC	Morbidity + mortality

TT: Transthoracic; TH: Transhiatal; MIE: Minimally invasive esophagectomy; N: Neoadjuvant chemotherapy; H: Histopathology subtype; SCC: Squamous cell carcinoma; AC: Adenocarcinoma; N/A: Not available; NSQIP SRC: National Surgical Quality Improvement Program Surgical Risk Calculator; ACCI: Age-adjusted charlson comorbidity index; CCI: Charlson comorbidty index; POSSUM: Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity; O-POSSUM: Operative - Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity; P-POSSUM: Portsmouth - Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity; GNRI: Geriatric nutritional risk index; PNI: Prognostic nutritional index; E-PASS: Estimation of physiologic and surgical stress.

Table 3 Clinical credibility of preoperative models

Ref.	Model	Oesophageal specific	No thresholds	Timely data	Reliable data	Easy to generate	Understandable	Useful range	Total
Onodera et al[56], 1984	PNI	No	Yes	Yes	Yes	Yes	Yes	Yes	6
Charlson et al[57], 1987	CCI	No	No	Yes	Yes	Yes	Yes	Yes	5
Charlson et al[58], 1994	ACCI	No	No	Yes	Yes	Yes	Yes	Yes	5
Bouillanne et al[59], 2005	GNRI	No	Yes	Yes	Yes	Yes	Yes	Partly	5.5
Schröder et al[29], 2006	Cologne	Yes	No	Yes	Yes	Partly	Yes	No	4.5
Steyerberg et al[30], 2006	Rotterdam	Yes	No	Yes	Yes	Yes	Yes	Yes	6
Ra et al[32], 2008	Philadelphia	Yes	No	Yes	Yes	Yes	Yes	Yes	6
Lagarde et al[31], 2008	Amsterdam	Yes	Partly	Yes	Yes	Partly	Yes	Yes	6
Wright et al[33], 2009	Original STS	Yes	No	Yes	Yes	Yes	Yes	Yes	6
Ferguson et al[34], 2011	Ferguson	Yes	No	Yes	Yes	Partly	Yes	Yes	5.5
Bilimoria et al[60], 2013	NSQIP SRC	No	Partly	Yes	Yes	Yes	Yes	Yes	5.5
Takeuchi et al[35], 2014	Takeuchi	Yes	No	Yes	Yes	Yes	Partly	Yes	5.5
Filip et al[36], 2015	PNI multivariate	Yes	No	Yes	Yes	Yes	Partly	Yes	5.5
Raymond et al[37], 2016	Revised STS	Yes	No	Yes	Yes	Yes	Yes	Yes	6
Reeh et al[38], 2016	PER	Yes	No	Yes	Yes	Partly	Yes	No	4.5
Hall et al[61], 2017	RAI-A	No	No	Yes	Partly	Yes	Yes	Yes	4.5
Subramaniam et al[62], 2018	5 Factor MFI	No	Yes	Yes	Yes	Yes	Yes	Partly	5.5
Saito et al[39], 2019	PPCS	Yes	No	Yes	Yes	Yes	Yes	Partly	5.5
Ohkura et al[40], 2020	JNCD	Yes	No	Yes	Yes	Yes	Partly	Yes	5.5
Arya et al[63], 2020	RAI-revised	No	No	Yes	Partly	Yes	Yes	Yes	4.5
Wan et al[41], 2022	RAI-revised (CC)	No	No	Yes	Partly	Yes	Yes	Yes	4.5

CCI: Charlson comorbidity index; ACCI: Age-adjusted charlson comorbidity index; GNRI: Geriatric nutritional risk index; NSQIP SRC: National Surgical Quality Improvement Program Surgical Risk Calculator; RAI-A: Administrative risk analysis index; MFI: Modified frailty index; STS: Society of Thoracic Surgeons Oesophagectomy Composite Score; PNI: Prognostic nutritional index; PPCS: Predicting postoperative complications score; JNCD: Japanese National Clinical Database; PER: Perioperative esophagectomy risk score.

Table 4 Methodological quality (overall performance) for preoperative models

Model	Study participation (out of 8)	Measurements (out of 4)	Analysis (out of 8)	Total (out of 20)
PNI	N/A	N/A	N/A	N/A
CCI	6	4	5	14
ACCI	6	4	4.5	14.5
GNRI	6.5	3	4.5	14
Cologne	7	3	4	14
Rotterdam	7.5	4	6	17.5
Philadelphia	7.5	4	5	16.5
Amsterdam	8	3.5	7	18.5
Original STS	8	4	5.5	17.5
Ferguson	7.5	4	5	16.5
NSQIP SRC	7.5	3.5	6	16.5
Takeuchi	8	3	7	18
PNI multivariate	8	4	7	19
Revised STS	8	4	4.5	16.5
PER	7	4	4	15
RAI-A	7	4	6.5	17.5
5 Factor MFI	6.5	3	6.5	16
PPCS	7	4	5.5	16.5
JNCD	8	4	6.5	18.5
RAI-revised	7	4	8	19
RAI-revised (CC)	8	4	6.5	18.5

CCI: Charlson comorbidity index; ACCI: Age-adjusted charlson comorbidity index; GNRI: Geriatric nutritional risk index; NSQIP SRC: National Surgical Quality Improvement Program Surgical Risk Calculator; RAI-A: Administrative risk analysis index; MFI: Modified frailty index; STS: Society of Thoracic Surgeons Oesophagectomy Composite Score; PNI: Prognostic nutritional index; PPCS: Predicting postoperative complications score; JNCD: Japanese National Clinical Database; PER: Perioperative esophagectomy risk score.

Table 5 Summary of the performance for all preoperative models in predicting perioperative mortality

Ref.	Predictive model	Discrimination	Calibration	Outcome
Bosch et al[44], 2011	CCI (2)	AUC = 0.567	HL P value (0.659)	Mortality
Filip et al[46], 2014		AUC = 0.736	Not reported	Mortality
Bosch et al[44], 2011	ACCI (2)	AUC = 0.684	HL P value (0.270)	Mortality
Filip et al[46], 2014		AUC = 0.744	Not reported	Mortality
Steyerberg et al[30], 2006	Rotterdam score (3)	AUC = 0.70	“Excellent”	Mortality
Zingg et al[42], 2009		P value = 0.003	HL P value (0.266)	Mortality
D’Journo et al[52], 2017		AUC = 0.64	Fair (overpredicts)	Mortality
Ra et al[32], 2008	Philadelphia score (2)	“Effective”	“Good”	Mortality
Zingg et al[42], 2009		P value = 0.001	HL P value (0.735)	Mortality
Wright et al[33], 2009	Original STS model	AUC = 0.621	Not reported	Major morbidity or mortality
Peng et al[54], 2020	NSQIP SRC (2)	AUC = 0.627	O:E = 1.13	Mortality
Ravindran et al[55], 2020		AUC = 0.880	Not reported	Mortality
Takeuchi et al[35], 2014	Takeuchi model (2)	AUC = 0.766	Not reported	Mortality
Takeuchi et al[51], 2018		AUC = 0.697	Not reported	Mortality
Raymond et al[37], 2016	Revised STS model	AUC = 0.71	Not reported	Mortality
Reeh et al[38], 2016	PER score	P = 0.001	Not reported	Mortality
Wan et al[41], 2022	RAI-A	AUC = 0.58	Not reported	Mortality
	5 Factor MFI	AUC = 0.58	Not reported	Mortality
	RAI-revised	AUC = 0.62	Not reported	Mortality
	RAI-revised (CC)	AUC = 0.60	Not reported	Mortality

AUC: Area under the receiver operating characteristic curve; HL: Hosmer-Lemeshow; O:E: Observed:expected ratio; CCI: Charlson comorbidity index; ACCI: Age-adjusted charlson comorbidity index; NSQIP SRC: National Surgical Quality Improvement Program Surgical Risk Calculator; RAI-A: Administrative risk analysis index; STS: Society of Thoracic Surgeons Oesophagectomy Composite Score; PNI: Prognostic nutritional index; PER: Perioperative esophagectomy risk score.

Table 6 Summary of the performance for all preoperative models in predicting perioperative major morbidity

Ref.	Predictive model	Discrimination	Calibration	Outcome
Ferguson et al[45], 2011	Amsterdam score	AUC = 0.653	Not stated	Major morbidity
Wright et al[33], 2009	Original STS model	AUC = 0.621	Not reported	Major morbidity or mortality
Filip et al[36], 2015	PNI multivariate	AUC = 0.80	HL P value (0.67)	Major morbidity
Raymond et al[37], 2016	Revised STS model	AUC = 0.63	Not reported	Major morbidity
Saito et al[39], 2019	PPCS model	AUC = 0.798	Not reported	Major morbidity

AUC: Area under the receiver operating characteristic curve; HL: Hosmer-Lemeshow; O:E: Observed:expected ratio; PPCS: Predicting postoperative complications score; STS: Society of Thoracic Surgeons Oesophagectomy Composite Score; PNI: Prognostic nutritional index.

Table 7 Summary of the performance for all preoperative models in predicting perioperative morbidity

Ref.	Predictive model	Discrimination	Calibration	Outcome
Filip et al[36], 2015	PNI	AUC = 0.65	HL P value (0.85)	Morbidity
Filip et al[36], 2015	CCI	AUC = 0.59	Pearson P value (0.48)	Morbidity
Filip et al[36], 2015	ACCI	AUC = 0.61	Pearson P value (0.17)	Morbidity
Schröder et al[29], 2006	Cologne score (2)	P value ≤ 0.001	Not reported	Morbidity
Lindner et al[48], 2016		P value = 0.010	Not reported	Morbidity
Lagarde et al[31], 2008	Amsterdam score (4)	AUC = 0.65	HL P value (0.366)	Morbidity
Grotenhuis et al[43], 2010		AUC = 0.64	HL P value (0.84)	Morbidity
Ferguson et al[45], 2011		AUC = 0.639	Not stated	Morbidity
Filip et al[36], 2015		AUC = 0.60	HL P value (0.55)	Morbidity
Gray et al[53], 2020	NSQIP SRC (3)	AUC = 0.553	“Insufficient”	Morbidity
Peng et al[54], 2020		AUC = 0.600	O:E = 1.89	Morbidity
Ravindran et al[55], 2020		AUC = 0.628	Not reported	Morbidity
Reeh et al[38], 2016	PER score	P ≤ 0.001	Not reported	Morbidity
Wan et al[41], 2022	RAI-A	AUC = 0.54	Not reported	Morbidity
Wan et al[41], 2022	5 Factor MFI	AUC = 0.57	Not reported	Morbidity
Wan et al[41], 2022	RAI-revised	AUC = 0.54	Not reported	Morbidity
Wan et al[41], 2022	RAI-revised (CC)	AUC = 0.51	Not reported	Morbidity

AUC: Area under the receiver operating characteristic curve; HL: Hosmer-Lemeshow; O:E: Observed:expected ratio; CCI: Charlson comorbidity index; ACCI: Age-adjusted charlson comorbidity index; NSQIP SRC: National Surgical Quality Improvement Program Surgical Risk Calculator; RAI-A: Administrative risk analysis index; PNI: Prognostic nutritional index; PER: Perioperative esophagectomy risk score.

Table 8 Summary of the performance for all preoperative models in predicting respiratory complications, return to theatre, readmission and anastomotic leak

Ref.	Predictive model	Discrimination	Calibration	Outcome
Yamana et al[47], 2015	PNI	AUC = 0.609	Not reported	Respiratory complications
Yamana et al[47], 2015	GNRI	AUC = 0.651	Not reported	Respiratory complications
Ferguson et al[34], 2011	Ferguson score (3)	AUC = 0.708	HL P value (0.16)	Respiratory complications
Reinersman et al[49], 2016		AUC = 0.726	HL P value (0.2394)	Respiratory complications
Xing et al[50], 2016		AUC = 0.539	Not reported	Respiratory complications
Gray et al[53], 2020	NSQIP SRC (3)	AUC = 0.533	Insufficient	Return to theatre
		AUC = 0.625	Insufficient	Readmission
Peng et al[54], 2020		AUC = 0.558	O:E = 0.48	Return to theatre
		AUC = 0.558	O:E = 1.11	Readmission
Ravindran et al[55], 2020		AUC = 0.584	Not reported	Return to theatre
		AUC = 0.767	Not reported	Readmission
Ohkura et al[40], 2020	JNCD model	AUC = 0.531	Not reported	Anastomotic leak

AUC: Area under the receiver operating characteristic curve; HL: Hosmer-Lemeshow; O:E: Observed:expected ratio; PNI: Prognostic nutritional index; GNRI: Geriatric nutritional risk index; NSQIP SRC: National Surgical Quality Improvement Program Surgical Risk Calculator; JNCD: Japanese National Clinical Database.

Table 9 Summary of the preoperative models across the five categories

Ref.	Model	Clinical credibility (out of 7)	Methodological quality (out of 20)	Model performance (overall utility)	External validation	Clinical effectiveness
Onodera et al[56], 1984	PNI	6	N/A	No	Yes	No
Charlson et al[57], 1987	CCI	5	14	No	Yes	No
Charlson et al[58], 1994	ACCI	5	14.5	No	Yes	No
Bouillanne et al[59], 2005	GNRI	5.5	14	No	Yes	No
Schröder et al[29], 2006	Cologne	4.5	14	No	Yes	No
Steyerberg et al[30], 2006	Rotterdam	6	17.5	No	Yes	No
Ra et al[32], 2008	Philadelphia	6	16.5	No	Yes	No
Lagarde et al[31], 2008	Amsterdam	6	18.5	No	Yes	No
Wright et al[33], 2009	Original STS	6	17.5	No	No	No
Ferguson et al[34], 2011	Ferguson	5.5	16.5	No	Yes	No
Bilimoria et al[60], 2013	NSQIP SRC	5.5	16.5	Mortality	Yes	No
Takeuchi et al[35], 2014	Takeuchi	5.5	18	Mortality	Yes	No
Filip et al[36], 2015	PNI multivariate	5.5	19	Major morbidity	No	No
Raymond et al[37], 2016	Revised STS	6	16.5	Mortality	No	No
Reeh et al[38], 2016	PER	4.5	15	No	No	No
Hall et al[61], 2017	RAI-A	4.5	17.5	No	Yes	No
Subramaniam et al[62], 2018	5 Factor MFI	5.5	16	No	Yes	No
Saito et al[39], 2019	PPCS	5.5	16.5	Major morbidity	No	No
Ohkura et al[40], 2020	JNCD	5.5	18.5	No	No	No
Arya et al[63], 2020	RAI-revised	4.5	19	No	Yes	No
Wan et al[41], 2022	RAI-revised (CC)	4.5	18.5	No	Yes	No

PNI: Prognostic nutritional index; GNRI: Geriatric nutritional risk index; NSQIP SRC: National Surgical Quality Improvement Program Surgical Risk Calculator; JNCD: Japanese National Clinical Database; RAI: Risk analysis index; PPCS: Predicting postoperative complications score; MFI: Modified frailty index; RAI-A: Administrative risk analysis index; PER: Perioperative esophagectomy risk score; STS: Society of Thoracic Surgeons Oesophagetomy Composite Score.