Basic Study
Copyright
©The Author(s) 2017. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Methodol. Mar 26, 2017; 7(1): 16-24
Published online Mar 26, 2017. doi: 10.5662/wjm.v7.i1.16
Table 1 Description of modified Delphi methodology
Delphi round 1 Systematic collection of online clinical score calculators Identified 176 externally validated online clinical score calculators Delphi round 2 Survey development Branching survey logic mapped score calculators to applicable specialties Survey distribution Academic and community based clinicians
Table 2 Survey respondent characteristics
Completion rate n of ScoresAnesthesia 2/5 (40%) 49 Cardiology 1/1 (100%) 37 Critical care 14/23 (61%) 75 Dermatology 0/0 1 Emergency medicine 4/6 (67%) 62 Family medicine 2/5 (40%) 107 Gastroenterology 3/3 (100%) 17 Hematology 1/1 (100%) 5 Infectious disease 2/2 (100%) 2 Internal medicine 14/25 (56%) 109 Nephrology 1/1 (100%) 6 Neurology 0/1 (0%) 23 OBGYN 1/1 (100%) 1 Oncology 1/2 (50%) 5 Orthopedics 0/0 3 Pediatric 7/13 (54%) 25 Pulmonology 4/6 (67%) 17 Surgery 2/3 (67%) 66
Table 3 Summary of importance of automation of specified clinical scores ranked by critical care and internal medicine physicians
Score name Year of creation n of variablesVery important Very important or nice to have Critical care APACHE II 1985 15 9/14 (64.3%) 12/14 (85.7%) SNAP II 2001 9 7/11 (63.6%) 9/11 (81.8%) NRDS scoring system 1998 5 7/12 (58.3%) 10/12 (83.3%) Post-anesthetic recovery score 1970 5 7/12 (58.3%) 9/12 (75%) Rotterdam score 1997 4 7/12 (58.3%) 8/12 (66.7%) SNAP 1993 27 7/12 (58.3%) 9/12 (75%) SNAP-PE 1993 30 7/12 (58.3%) 9/12 (75%) SNAP-PE II 2001 12 7/12 (58.3%) 9/12 (75%) Wells criteria for DVT 2006 9 7/12 (58.3%) 9/12 (75%) Wells criteria for PE 1998 7 7/12 (58.3%) 10/12 (83.3%) PAWS 2008 7 6/11 (54.5%) 8/11 (72.7%) CRIB 1993 5 6/12 (50%) 8/12 (66.7%) CRIB II 2003 5 6/12 (50%) 8/12 (66.7%) MSSS 2002 7 6/12 (50%) 8/12 (66.7%) PELOD score 1999 13 3/6 (50%) 4/6 (66.7%) SAPS II 1993 16 5/10 (50%) 7/10 (70%) TIMI risk index 2006 3 5/11 (45.5%) 8/11 (72.7%) TRISS 1987 9 4/9 (44.4%) 6/9 (66.7%) Children's coma score 1984 3 3/7 (42.9%) 4/7 (57.1%) PRISM score 1988 16 3/7 (42.9%) 5/7 (71.4%) CURB-65 2003 5 5/12 (41.7%) 8/12 (66.7%) SCORETEN scale 2000 6 5/12 (41.7%) 9/12 (75%) MEWS score 2006 6 4/10 (40%) 6/10 (60%) Rockall score 2008 11 3/8 (37.5%) 5/8 (62.5%) TRIOS score 2001 4 3/8 (37.5%) 5/8 (62.5%) Geneva score for PE 2006 9 4/11 (36.4%) 7/11 (63.6%) Injury Severity Score 1974 6 4/11 (36.4%) 8/11 (72.7%) Lung Injury score 1988 5 4/11 (36.4%) 8/11 (72.7%) MPMII - admission 1993 14 4/11 (36.4%) 6/11 (54.5%) MPMII - 24-48-72 1993 14 4/11 (36.4%) 6/11 (54.5%) LODS score 1996 12 3/9 (33.3%) 7/9 (77.8%) MEDS score 2003 10 3/9 (33.3%) 6/9 (66.7%) MESS score 1990 5 4/12 (33.3%) 7/12 (58.3%) Parsonnet Score 1989 14 4/12 (33.3%) 7/12 (58.3%) Pediatric coma scale 1988 3 2/6 (33.3%) 3/6 (50%) RAPS 1987 5 3/9 (33.3%) 7/9 (77.8%) Surgical Apgar score 2007 3 4/12 (33.3%) 8/12 (66.7%) ASCOT score 1990 8 4/13 (30.8%) 6/13 (46.2%) MELD score 2001 4 4/13 (30.8%) 12/13 (92.3%) PIM2 2003 8 2/7 (28.6%) 5/7 (71.4%) SWIFT score 2008 6 2/7 (28.6%) 4/7 (57.1%) Clinical Pulmonary Infection Score 1991 8 3/11 (27.3%) 9/11 (81.8%) MPM-24 h 1988 15 3/11 (27.3%) 6/11 (54.5%) Child-Pugh Score 1973 5 3/12 (25%) 11/12 (91.7%) Decaf score 2012 5 2/8 (25%) 4/8 (50%) ONTARIO score 1995 6 2/8 (25%) 4/8 (50%) AKICS score 2007 8 3/13 (23.1%) 7/13 (53.8%) AVPU scale 2004 4 2/9 (22.2%) 6/9 (66.7%) PERC rule for PE 2001 7 2/9 (22.2%) 6/9 (66.7%) RIETE score 1988 6 2/9 (22.2%) 6/9 (66.7%) BISAP score for pancreatitis mortality 2008 5 2/10 (20%) 4/10 (40%) Bleeding risk score 2007 4 2/10 (20%) 6/10 (60%) Clinical asthma evaluation score 1972 5 2/10 (20%) 6/10 (60%) PIRO score 2009 8 2/10 (20%) 7/10 (70%) ABC score for massive transfusion 2009 4 2/11 (18.2%) 6/11 (54.5%) ACLS score 1981 4 2/11 (18.2%) 7/11 (63.6%) MOD score 1995 7 2/11 (18.2%) 8/11 (72.7%) MPM - admission 1988 10 2/11 (18.2%) 6/11 (54.5%) sPESI 2010 8 2/11 (18.2%) 7/11 (63.6%) ABIC score 2008 4 2/12 (16.7%) 5/12 (41.7%) CRUSADE score 2009 8 2/12 (16.7%) 6/12 (50%) Pediatric trauma score 1988 6 1/6 (16.7%) 2/6 (33.3%) LRINEC Score for Necrotizing STI 2004 5 1/8 (12.5%) 4/8 (50%) Panc 3 score 2007 3 1/8 (12.5%) 3/8 (37.5%) Pancreatitis outcome score 2007 7 1/8 (12.5%) 3/8 (37.5%) TASH score 2006 7 1/8 (12.5%) 4/8 (50%) POSSUM score 1991 18 1/9 (11.1%) 3/9 (33.3%) Revised Trauma score 1981 3 1/9 (11.1%) 5/9 (55.6%) 24 h ICU trauma score 1992 4 1/10 (10%) 7/10 (70%) HIT Expert Probability Score 2010 11 1/11 (9.1%) 6/11 (54.5%) Bronchiectasis severity index 2014 10 1/12 (8.3%) 4/12 (33.3%) Oxygenation index 2005 3 1/13 (7.7%) 7/13 (53.8%) CT severity index 1990 1 0/12 (0%) 6/12 (50%) Glasgow coma scale 1974 3 0/13 (0%) 10/13 (76.9%) SOFA 2001 6 0/13 (0%) 8/13 (61.5%) Internal medicine Wells criteria for DVT 2006 9 10/14 (71.4%) 13/14 (92.9%) Wells criteria for PE 1998 7 10/14 (71.4%) 13/14 (92.9%) CHA2DS2-VASc 2010 7 9/14 (64.3%) 13/14 (92.9%) TIMI risk index 2006 3 9/14 (64.3%) 13/14 (92.9%) TIMI risk score for UA/NSTEMI 2000 7 9/14 (64.3%) 13/14 (92.9%) TIMI risk score for STEMI 2000 9 9/14 (64.3%) 13/14 (92.9%) CURB-65 2003 5 8/14 (57.1%) 13/14 (92.9%) STESS score 2008 4 8/14 (57.1%) 13/14 (92.9%) Duke criteria for IE 1994 8 6/13 (46.2%) 12/13 (92.3%) PESI 2006 11 7/12 (58.3%) 11/12 (91.7%) Revised cardiac risk index for pre-operative risk 1999 6 7/12 (58.3%) 11/12 (91.7%) SOFA 2001 6 6/12 (50%) 11/12 (91.7%) ABCD2 score 2006 5 5/12 (41.7%) 11/12 (91.7%) Charlson Comorbidity index 1987 1 2/12 (16.7%) 11/12 (91.7%) PERC rule for PE 2001 7 5/11 (45.5%) 10/11 (90.9%) sPESI 2010 8 4/11 (36.4%) 10/11 (90.9%) MOD score 1995 7 3/11 (27.3%) 10/11 (90.9%) MPM - 24 h 1988 15 4/10 (40%) 9/10 (90%) MPM - admission 1988 10 3/10 (30%) 9/10 (90%) MEDS score 2003 10 2/10 (20%) 9/10 (90%) PIRO score 2009 8 1/10 (10%) 9/10 (90%) SAPS II 1993 16 4/9 (44.4%) 8/9 (88.9%) SWIFT score 2008 6 2/8 (25%) 7/8 (87.5%) Panc 3 score 2007 3 1/8 (12.5%) 7/8 (87.5%) APACHE II 1985 15 9/14 (64.3%) 12/14 (85.7%) Parsonnett Score 1989 14 8/14 (57.1%) 12/14 (85.7%) HIT Expert Probability Score 2010 11 6/14 (42.9%) 12/14 (85.7%) Ranson's criteria 1974 11 6/14 (42.9%) 12/14 (85.7%) TRIOS score 2001 4 3/7 (42.9%) 6/7 (85.7%) 4Ts Score 2006 5 5/14 (35.7%) 12/14 (85.7%) Framingham coronary heart disease risk score 1998 7 5/14 (35.7%) 12/14 (85.7%) 30 d PCI readmission risk 2013 10 2/7 (28.6%) 6/7 (85.7%) Glasgow coma scale 1974 3 9/13 (69.2%) 11/13 (84.6%) Modified NIH Stroke Scale 2001 9 7/13 (53.9%) 11/13 (84.6%) King's College Criteria for Acetaminophen Toxicity 1989 6 4/12 (33.3%) 10/12 (83.3%) Glasgow-Blatchford Bleeding score 2000 9 3/12 (25%) 10/12 (83.3%) ATRIA bleeding risk score 2011 6 2/12 (16.7%) 10/12 (83.3%) Glasgow Alcoholic hepatitis score 2005 4 5/11 (45.5%) 9/11 (81.8%) MEWS score 2006 6 4/11 (36.4%) 9/11 (81.8%) Hemorr2hages score 2006 11 2/11 (18.2%) 9/11 (81.8%) Decaf score 2012 5 4/10 (40%) 8/10 (80%) MPMII - admission 1993 14 4/10 (40%) 8/10 (80%) MPMII - 24-48-72 1993 14 4/10 (40%) 8/10 (80%) Malnutrition universal screening tool (MUST) 2004 3 2/10 (20%) 8/10 (80%) ASTRAL score 2012 6 1/10 (10%) 8/10 (80%) GRACE ACS 2006 12 1/10 (10%) 8/10 (80%) CHADS2 2001 5 7/14 (50%) 11/14 (78.6%) Multidimensional frailty score 2014 9 7/14 (50%) 11/14 (78.6%) Geneva score for PE 2006 9 3/9 (33.3%) 7/9 (77.8%) Pittsburg knee rules 1994 3 3/9 (33.3%) 7/9 (77.8%) Mayo scoring system for assessment of ulcerative colitis activity 2005 4 1/9 (11.1%) 7/9 (77.8%) 4-yr mortality prognostic index 2006 12 1/9 (11.1%) 7/9 (77.8%) Rockall score 2008 11 1/9 (11.1%) 7/9 (77.8%) SHARF scoring system 2004 9 1/9 (11.1%) 7/9 (77.8%) HAS-BLED 2010 12 5/13 (38.5%) 10/13 (76.9%) ATRIA stroke risk score 2013 7 3/12 (25%) 9/12 (75%) Euroscore 1999 17 1/8 (12.5%) 6/8 (75%) Renal risk score 2011 6 1/8 (12.5%) 6/8 (75%) ROSE risk score 1996 7 1/8 (12.5%) 6/8 (75%) LRINEC Score for Necrotizing STI 2004 5 3/11 (27.3%) 8/11 (72.7%) Bleeding risk score 2007 4 2/11 (18.2%) 8/11 (72.7%) CT severity index 1990 1 1/11 (9.1%) 8/11 (72.7%) SCORETEN scale 2000 6 7/14 (50%) 10/14 (71.4%) REMS 2004 7 2/7 (28.6%) 5/7 (71.4%) Mayo CABG risk of inpatient death after MI 2007 7 1/7 (14.3%) 5/7 (71.4%) Mayo PCI risk of inpatient MACE 2007 7 1/7 (14.3%) 5/7 (71.4%) QMMI score 2001 11 1/7 (14.3%) 5/7 (71.4%) MELD score 2001 4 0/14 (0%) 10/14 (71.4%) Nexus criteria for C-spine imaging 1970 5 4/10 (40%) 7/10 (70%) Birmingham nutritional risk score 1995 7 2/10 (20%) 7/10 (70%) Canadian CT head rule 2001 9 2/10 (20%) 7/10 (70%) ACLS score 1981 4 1/10 (10%) 7/10 (70%) San Francisco syncope rule 2004 5 1/10 (10%) 7/10 (70%) Mannheim peritonitis index 1993 7 6/13 (46.2%) 9/13 (69.2%) HADO score 2006 4 3/9 (33.3%) 6/9 (66.7%) CARE score 2001 3 1/9 (11.1%) 6/9 (66.7%) ICH score 2001 5 1/9 (11.1%) 6/9 (66.7%) Adult appendicitis score 2014 8 6/14 (42.9%) 9/14 (64.3%) IMPACT score 2008 11 6/14 (42.9%) 9/14 (64.3%) CRUSADE score 2009 8 4/14 (28.6%) 9/14 (64.3%) PORT/PSI score 1997 20 2/14 (14.3%) 9/14 (64.3%) CIWA-Ar 1989 10 1/14 (7.1%) 9/14 (64.3%) LODS score 1996 12 3/8 (37.5%) 5/8 (62.5%) OESIL risk score 2003 4 2/8 (25%) 5/8 (62.5%) QRISK2 2010 14 2/8 (25%) 5/8 (62.5%) Qstroke score 2013 15 2/8 (25%) 5/8 (62.5%) RIETE score 1988 6 2/8 (25%) 5/8 (62.5%) EGSYS score 2008 6 1/8 (12.5%) 5/8 (62.5%) EHMRG 2012 10 1/8 (12.5%) 5/8 (62.5%) FOUR score 2005 4 1/8 (12.5%) 5/8 (62.5%) Pancreatitis outcome score 2007 7 1/8 (12.5%) 5/8 (62.5%) Prostate cancer prevention trial risk calculator 1993 6 6/13 (46.2%) 8/13 (61.5%) Alvarado score for acute appendicitis 1986 8 5/13 (38.5%) 8/13 (61.5%) DRAGON score 2012 6 1/10 (10%) 6/10 (60%) Bronchiectasis severity index 2014 10 3/14 (21.4%) 8/14 (57.1%) New Orleans head CT rule 2000 8 1/7 (14.3%) 4/7 (57.1%) POSSUM score 1991 18 1/7 (14.3%) 4/7 (57.1%) Child-Pugh Score 1973 5 0/14 (0%) 8/14 (57.1%) Lung Injury score 1988 5 4/9 (44.4%) 5/9 (55.6%) AVPU scale 2004 4 2/9 (22.2%) 5/9 (55.6%) Gupta perioperative cardiac risk 2011 5 2/9 (22.2%) 5/9 (55.6%) HEART score 2008 5 1/9 (11.1%) 5/9 (55.6%) IgA nephropathy score 2006 8 5/14 (35.7%) 7/14 (50%) ABIC score 2008 4 4/14 (28.6%) 7/14 (50%) CAMBS score 1993 4 4/14 (28.6%) 7/14 (50%) GAP risk assessment score 2012 4 2/8 (25%) 4/8 (50%) BISAP score for pancreatitis mortality 2008 5 2/10 (20%) 5/10 (50%) ONTARIO score 1995 6 1/8 (12.5%) 4/8 (50%) JAMA kidney failure risk equation 2011 7 4/13 (30.8%) 5/13 (38.5%)
Table 4 Predictors of desirability of score automation based on number of each variable type in each score
Automation: Very important/nice to have OR (95%CI) Critical care n of variables0.68 (0.23, 1.59) Clinical history 1.36 (0.36, 4.93) Vital sign 1.40 (0.53, 4.6) Medication 4.89 (0.10, 237.52) Clinical judgment 2.33 (0.76, 9.80) Examination 0.99 (0.36, 3.14) Laboratory value 1.48 (0.61, 4.41) Charted variable (non-vital) 2.26 (0.70, 8.93) Demographic value 0.20 (0.03, 1.00) Another score 2.07 (0.39, 12.13) Internal medicine n of variables0.64 (0.39, 1.04) Clinical history 2.34a (1.26, 4.67) Vital sign 1.88a (1.03, 3.68) Medication 2.89 (0.37, 63.17) Clinical judgment 1.41 (0.75, 2.74) Examination 1.56 (0.88, 2.87) Laboratory value 1.51 (0.90, 2.62) Charted variable (non-vital) 2.54 (0.85, 8.70) Demographic value 0.90 (0.41, 1.97) Another score 0.89 (0.30, 2.17)