Systematic Reviews
Copyright ©The Author(s) 2023.
World J Gastrointest Surg. Mar 27, 2023; 15(3): 450-470
Published online Mar 27, 2023. doi: 10.4240/wjgs.v15.i3.450
Table 1 Development studies of preoperative multivariate models
Ref.
Period + number
Sample region
Operation
Characteristics
Source of data
Models tested
Outcomes tested
Schröder et al[29], 20061997-2002 (126)GermanyTTN: 46/126; H: 68 AC/54 SCCSingle centreCologne scoreMorbidity
Steyerberg et al[30], 20061980-2002 (3592)Unites States/NetherlandsTT/THN: 878/3592; H: 2118 AC/1307 SCCSEER database + two centresRotterdam scoreMortality
Ra et al[32], 20081997-2003 (1172)United StatesTT/THN: N/A; H: N/ASEER databasePhiladelphia scoreMortality
Lagarde et al[31], 20081993-2005 (663)NetherlandsTT/THN: 114/663; H: 476 AC/187 SCCSingle centreAmsterdam scoreMorbidity
Wright et al[33], 20092002-2007 (2315)United StatesTT/TH/MIEN: 1016/2315; H: N/ASTSGTS database (164 centres)Original STS modelMajor morbidity + mortality
Ferguson et al[34], 20111980-2009 (516)United StatesTT/TH/hybrid/MIEN: 167/516; H: 261 AC/137 SCCSingle centreFerguson scoreRespiratory complications
Takeuchi et al[35], 20142011-2011 (5354)JapanNot statedN: 1268/5354; H: N/ANational databaseTackeuchi modelMortality
Filip et al[36], 20152008-2012 (167)ItalyTT/TH/MIEN: 131/167; H: 105 AC/62 SCCSingle centrePNI-multivariate: PNI; CCI; ACCI; POSSUM; Amsterdam scoreMorbidity + major morbidity (PNI-multivariate only)
Raymond et al[37], 20162011-2014 (4321)United StatesTT/TH/MIEN: 2930/4321; H: N/ASTSGTS database (164 centres)Revised STS modelMajor morbidity + mortality
Reeh et al[38], 20161994-2007 (498)GermanyTT/THN: 0/498; H: 253 SCC/245 ACSingle centrePER scoreMorbidity + mortality
Saito et al[39], 20192007-2015 (90)JapanMIEN: 29/90; H: 87 SCC/3 ACSingle centrePPCS modelMajor morbidity
Ohkura et al[40], 20202011-2012 (10826)JapanNot statedN: 2717/10826;H: N/ANational database (4105 centres)JNCD modelAnastomotic leak
Wan et al[41], 20222006-2017 (10602)United StatesNot reportedN: N/A; H: N/ANational (NSQIP)RAI-revised (CC): RAI-A; RAI-revised; 5 Factor MFIMorbidity + mortality
Table 2 Validation studies of preoperative models
Ref.
Period + number
Sample region
Operation
Characteristics
Source of data
Models tested
Outcomes tested
Zingg et al[42], 20091990-2007 (346)Switzerland/AustraliaTTN: 140/346; H: 259 AC/71 SCCTwo centresRotterdam score; Philadelphia scoreMortality
Grotenhuis et al[43], 20101991-2008 (777)NetherlandsTT/THN: 221/777; H: N/ASingle centreAmsterdam scoreMorbidity
Bosch et al[44], 20111991-2007 (278)NetherlandsTTN: 10/278; H: 235 AC/43 SCCSingle centreACCI; CCI; O-POSSUM; P-POSSUMMortality
Ferguson et al[45], 20111980-2009 (514)United StatesTT/TH/hybrid/MIEN: 167/514; H: 261 AC/137 SCCSingle centreAmsterdam scoreMorbidity + major morbidity
Filip et al[46], 20142004-2013 (43)RomaniaTT/THN: 22/43; H: 33 SCC/9 ACSingle centreACCI; CCI; POSSUM; O-POSSUM; P-POSSUMMortality
Yamana et al[47], 20152005-2013 (251)JapanTT/MIEN: 150/251; H: N/ASingle centreGNRI; PNI; E-PASS; POSSUMRespiratory complications
Lindner et al[48], 20162005-2009 (94)GermanyTTN: 54/94; H: 94 AC/0 SCCSingle centreCologne scoreMorbidity
Reinersman et al[49], 20162009-2012 (136)United StatesTT/TH/hybrid/MIEN: 110/136; H: 118 AC/18 SCCSingle centreFerguson scoreRespiratory complications
Xing et al[50], 20162008-2010 (217)ChinaTT/THN: 0/217; H: 162 SCC/50 ACSingle centreFerguson scoreRespiratory complications
Takeuchi et al[51], 20182000-2016 (438)JapanTTN: 208/438; H: 398 SCC/27 ACSingle centreTakeuchi modelMortality
D’Journo et al[52], 20172004-2013 (1039)FranceTT/THN: 420/1039; H: N/ANational databaseRotterdam scoreMortality
Gray et al[53], 20202016-2018 (240)United StatesTT/TH/MIEN: N/A; H: N/ASingle centreNSQIP SRCMorbidity
Peng et al[54], 20202012-2019 (218)United StatesMIEN: 189/218; H: N/ASingle centreNSQIP SRCMorbidity + mortality
Ravindran et al[55], 20202013-2017 (100)United StatesTTN: 87/100; H: 75 AC/21 SCCSingle centreNSQIP SRCMorbidity + mortality
Table 3 Clinical credibility of preoperative models
Ref.
Model
Oesophageal specific
No thresholds
Timely data
Reliable data
Easy to generate
Understandable
Useful range
Total
Onodera et al[56], 1984PNINoYesYesYesYesYesYes6
Charlson et al[57], 1987CCINoNoYesYesYesYesYes5
Charlson et al[58], 1994ACCINoNoYesYesYesYesYes5
Bouillanne et al[59], 2005GNRINoYesYesYesYesYesPartly5.5
Schröder et al[29], 2006CologneYesNoYesYesPartlyYesNo4.5
Steyerberg et al[30], 2006RotterdamYesNoYesYesYesYesYes6
Ra et al[32], 2008PhiladelphiaYesNoYesYesYesYesYes6
Lagarde et al[31], 2008AmsterdamYesPartlyYesYesPartlyYesYes6
Wright et al[33], 2009Original STSYesNoYesYesYesYesYes6
Ferguson et al[34], 2011FergusonYesNoYesYesPartlyYesYes5.5
Bilimoria et al[60], 2013NSQIP SRCNoPartlyYesYesYesYesYes5.5
Takeuchi et al[35], 2014TakeuchiYesNoYesYesYesPartlyYes5.5
Filip et al[36], 2015PNI multivariateYesNoYesYesYesPartlyYes5.5
Raymond et al[37], 2016Revised STSYesNoYesYesYesYesYes6
Reeh et al[38], 2016PERYesNoYesYesPartlyYesNo4.5
Hall et al[61], 2017RAI-ANoNoYesPartlyYesYesYes4.5
Subramaniam et al[62], 20185 Factor MFINoYesYesYesYesYesPartly5.5
Saito et al[39], 2019PPCSYesNoYesYesYesYesPartly5.5
Ohkura et al[40], 2020JNCDYesNoYesYesYesPartlyYes5.5
Arya et al[63], 2020RAI-revisedNoNoYesPartlyYesYesYes4.5
Wan et al[41], 2022RAI-revised (CC)NoNoYesPartlyYesYesYes4.5
Table 4 Methodological quality (overall performance) for preoperative models
Model
Study participation (out of 8)
Measurements (out of 4)
Analysis (out of 8)
Total (out of 20)
PNIN/AN/AN/AN/A
CCI64514
ACCI644.514.5
GNRI6.534.514
Cologne73414
Rotterdam7.54617.5
Philadelphia7.54516.5
Amsterdam83.5718.5
Original STS845.517.5
Ferguson7.54516.5
NSQIP SRC7.53.5616.5
Takeuchi83718
PNI multivariate84719
Revised STS844.516.5
PER74415
RAI-A746.517.5
5 Factor MFI6.536.516
PPCS745.516.5
JNCD846.518.5
RAI-revised74819
RAI-revised (CC)846.518.5
Table 5 Summary of the performance for all preoperative models in predicting perioperative mortality
Ref.
Predictive model
Discrimination
Calibration
Outcome
Bosch et al[44], 2011CCI (2)AUC = 0.567HL P value (0.659)Mortality
Filip et al[46], 2014AUC = 0.736Not reportedMortality
Bosch et al[44], 2011ACCI (2)AUC = 0.684HL P value (0.270)Mortality
Filip et al[46], 2014AUC = 0.744Not reportedMortality
Steyerberg et al[30], 2006Rotterdam score (3)AUC = 0.70“Excellent”Mortality
Zingg et al[42], 2009P value = 0.003HL P value (0.266)Mortality
D’Journo et al[52], 2017AUC = 0.64Fair (overpredicts)Mortality
Ra et al[32], 2008Philadelphia score (2)“Effective”“Good”Mortality
Zingg et al[42], 2009P value = 0.001HL P value (0.735)Mortality
Wright et al[33], 2009Original STS modelAUC = 0.621Not reportedMajor morbidity or mortality
Peng et al[54], 2020NSQIP SRC (2)AUC = 0.627O:E = 1.13Mortality
Ravindran et al[55], 2020AUC = 0.880Not reportedMortality
Takeuchi et al[35], 2014Takeuchi model (2)AUC = 0.766Not reportedMortality
Takeuchi et al[51], 2018AUC = 0.697Not reportedMortality
Raymond et al[37], 2016Revised STS modelAUC = 0.71Not reportedMortality
Reeh et al[38], 2016PER scoreP = 0.001Not reportedMortality
Wan et al[41], 2022RAI-AAUC = 0.58Not reportedMortality
5 Factor MFIAUC = 0.58Not reportedMortality
RAI-revisedAUC = 0.62Not reportedMortality
RAI-revised (CC)AUC = 0.60Not reportedMortality
Table 6 Summary of the performance for all preoperative models in predicting perioperative major morbidity
Ref.
Predictive model
Discrimination
Calibration
Outcome
Ferguson et al[45], 2011Amsterdam score AUC = 0.653Not statedMajor morbidity
Wright et al[33], 2009Original STS modelAUC = 0.621Not reportedMajor morbidity or mortality
Filip et al[36], 2015PNI multivariateAUC = 0.80HL P value (0.67)Major morbidity
Raymond et al[37], 2016Revised STS modelAUC = 0.63Not reportedMajor morbidity
Saito et al[39], 2019PPCS modelAUC = 0.798Not reportedMajor morbidity
Table 7 Summary of the performance for all preoperative models in predicting perioperative morbidity
Ref.
Predictive model
Discrimination
Calibration
Outcome
Filip et al[36], 2015PNIAUC = 0.65HL P value (0.85)Morbidity
Filip et al[36], 2015CCI AUC = 0.59Pearson P value (0.48)Morbidity
Filip et al[36], 2015ACCIAUC = 0.61Pearson P value (0.17)Morbidity
Schröder et al[29], 2006Cologne score (2)P value ≤ 0.001Not reportedMorbidity
Lindner et al[48], 2016P value = 0.010Not reportedMorbidity
Lagarde et al[31], 2008Amsterdam score (4)AUC = 0.65HL P value (0.366)Morbidity
Grotenhuis et al[43], 2010AUC = 0.64HL P value (0.84)Morbidity
Ferguson et al[45], 2011AUC = 0.639Not statedMorbidity
Filip et al[36], 2015AUC = 0.60HL P value (0.55)Morbidity
Gray et al[53], 2020NSQIP SRC (3)AUC = 0.553“Insufficient”Morbidity
Peng et al[54], 2020AUC = 0.600O:E = 1.89Morbidity
Ravindran et al[55], 2020AUC = 0.628Not reportedMorbidity
Reeh et al[38], 2016PER scoreP ≤ 0.001Not reportedMorbidity
Wan et al[41], 2022RAI-AAUC = 0.54Not reportedMorbidity
Wan et al[41], 20225 Factor MFIAUC = 0.57Not reportedMorbidity
Wan et al[41], 2022RAI-revisedAUC = 0.54Not reportedMorbidity
Wan et al[41], 2022RAI-revised (CC)AUC = 0.51Not reportedMorbidity
Table 8 Summary of the performance for all preoperative models in predicting respiratory complications, return to theatre, readmission and anastomotic leak
Ref.
Predictive model
Discrimination
Calibration
Outcome
Yamana et al[47], 2015PNIAUC = 0.609Not reportedRespiratory complications
Yamana et al[47], 2015GNRIAUC = 0.651Not reportedRespiratory complications
Ferguson et al[34], 2011Ferguson score (3)AUC = 0.708HL P value (0.16)Respiratory complications
Reinersman et al[49], 2016AUC = 0.726HL P value (0.2394)Respiratory complications
Xing et al[50], 2016AUC = 0.539Not reportedRespiratory complications
Gray et al[53], 2020NSQIP SRC (3)AUC = 0.533InsufficientReturn to theatre
AUC = 0.625InsufficientReadmission
Peng et al[54], 2020AUC = 0.558O:E = 0.48Return to theatre
AUC = 0.558O:E = 1.11Readmission
Ravindran et al[55], 2020AUC = 0.584Not reportedReturn to theatre
AUC = 0.767Not reportedReadmission
Ohkura et al[40], 2020JNCD modelAUC = 0.531Not reportedAnastomotic leak
Table 9 Summary of the preoperative models across the five categories
Ref.
Model
Clinical credibility (out of 7)
Methodological quality (out of 20)
Model performance (overall utility)
External validation
Clinical effectiveness
Onodera et al[56], 1984PNI6N/ANoYesNo
Charlson et al[57], 1987CCI514NoYesNo
Charlson et al[58], 1994ACCI514.5NoYesNo
Bouillanne et al[59], 2005GNRI5.514NoYesNo
Schröder et al[29], 2006Cologne4.514NoYesNo
Steyerberg et al[30], 2006Rotterdam617.5NoYesNo
Ra et al[32], 2008Philadelphia616.5NoYesNo
Lagarde et al[31], 2008Amsterdam618.5NoYesNo
Wright et al[33], 2009Original STS617.5NoNoNo
Ferguson et al[34], 2011Ferguson 5.516.5NoYesNo
Bilimoria et al[60], 2013NSQIP SRC5.516.5MortalityYesNo
Takeuchi et al[35], 2014Takeuchi5.518MortalityYesNo
Filip et al[36], 2015PNI multivariate5.519Major morbidityNoNo
Raymond et al[37], 2016Revised STS616.5MortalityNoNo
Reeh et al[38], 2016PER4.515NoNoNo
Hall et al[61], 2017RAI-A4.517.5NoYesNo
Subramaniam et al[62], 20185 Factor MFI5.516NoYesNo
Saito et al[39], 2019PPCS5.516.5Major morbidityNoNo
Ohkura et al[40], 2020JNCD 5.518.5NoNoNo
Arya et al[63], 2020RAI-revised4.519NoYesNo
Wan et al[41], 2022RAI-revised (CC)4.518.5NoYesNo