Sequential organ failure assessment score is superior to other prognostic indices in acute pancreatitis

Abstract

BACKGROUND

Acute pancreatitis (AP) is a common surgical condition, with severe AP (SAP) potentially lethal. Many prognostic indices, including; acute physiology and chronic health evaluation II score (APACHE II), bedside index of severity in acute pancreatitis (BISAP), Glasgow score, harmless acute pancreatitis score (HAPS), Ranson’s score, and sequential organ failure assessment (SOFA) evaluate AP severity and predict mortality.

AIM

To evaluate these indices' utility in predicting severity, intensive care unit (ICU) admission, and mortality.

METHODS

A retrospective analysis of 653 patients with AP from July 2009 to September 2016 was performed. The demographic, clinical profile, and patient outcomes were collected. SAP was defined as per the revised Atlanta classification. Values for APACHE II score, BISAP, HAPS, and SOFA within 24 h of admission were retrospectively obtained based on laboratory results and patient evaluation recorded on a secure hospital-based online electronic platform. Data with < 10% missing data was imputed via mean substitution. Other patient information such as demographics, disease etiology, and patient outcomes were also derived from electronic medical records.

RESULTS

The mean age was 58.7 ± 17.5 years, with 58.7% males. Gallstones (n = 404, 61.9%), alcohol (n = 38, 5.8%), and hypertriglyceridemia (n = 19, 2.9%) were more common aetiologies. 81 (12.4%) patients developed SAP, 20 (3.1%) required ICU admission, and 12 (1.8%) deaths were attributed to SAP. Ranson’s score and APACHE-II demonstrated the highest sensitivity in predicting SAP (92.6%, 80.2% respectively), ICU admission (100%), and mortality (100%). While SOFA and BISAP demonstrated lowest sensitivity in predicting SAP (13.6%, 24.7% respectively), ICU admission (40.0%, 25.0% respectively) and mortality (50.0%, 25.5% respectively). However, SOFA demonstrated the highest specificity in predicting SAP (99.7%), ICU admission (99.2%), and mortality (98.9%). SOFA demonstrated the highest positive predictive value, positive likelihood ratio, diagnostic odds ratio, and overall accuracy in predicting SAP, ICU admission, and mortality. SOFA and Ranson’s score demonstrated the highest area under receiver-operator curves at 48 h in predicting SAP (0.966, 0.857 respectively), ICU admission (0.943, 0.946 respectively), and mortality (0.968, 0.917 respectively).

CONCLUSION

The SOFA and 48-h Ranson’s scores accurately predict severity, ICU admission, and mortality in AP, with more favorable statistics for the SOFA score.

Key Words: Pancreatitis; Severity scoring; Intensive care unit; Mortality; Sequential Organ Failure Assessment score; Ranson’s score

Core Tip: Acute pancreatitis is a common surgical emergency requiring quick evaluation of its severity to guide further management principles. Both the sequential organ failure assessment (SOFA) and 48-h Ranson scores accurately predict severity, intensive care unit admission, and mortality in acute pancreatitis (AP), with more favorable statistics for the SOFA score. Simple bedside scores such as bedside index of severity in AP and harmless AP score are practical and straightforward tests to screen out mild disease at the onset, allowing physicians to preferentially allocate resources for severe AP patients.

INTRODUCTION

Acute pancreatitis (AP) is a common surgical condition with an incidence of 50-80 per 100000 population[1-3]. Severe AP (SAP) occurs in 12%-20% of patients and has significant morbidity and mortality burden[4-6]. Early mortality (within the first two weeks) is attributed to cytokine storm and multisystem organ failure (OF). Delayed mortality (after two weeks) is attributed to infectious complications[7]. A primary concern for clinicians is the gross heterogeneity in clinical presentation and identifying patients predicted to manifest SAP and subsequent mortality risk. Therefore, an accurate scoring system on admission becomes critical to guide patient disposition and aggressiveness of treatment, resulting in better patient care and resource allocation. Though prevalent scoring systems have moderate to high accuracy, multiple laboratory variables are sometimes too cumbersome for routine clinical use[8,9]. The bedside index of severity in acute pancreatitis (BISAP)[10] and harmless acute pancreatitis score (HAPS)[11] are simple systems that can be computed using easily attained clinical parameters. The sequential organ failure assessment (SOFA) score developed initially by Vincent et al[12] was validated for use in AP[13]. The SOFA score is graded from 0 to 4 including markers PaO2/FiO2 ratio, Glasgow coma scale, mean arterial pressure or administration of vasopressors, bilirubin levels and platelet levels. While there have been studies that have compared the efficacy of these newer scores in predicting disease severity against classic scores such as the Ranson’s score and Glasgow score, such as the retrospective studies by Khanna et al[14] and Tan et al[15], these remain few and far between. Fewer still have reported their utility in predicting critical clinical outcomes such as intensive care unit (ICU) admission and AP mortality, as evidenced by the retrospective study by Shafiq et al[16] and Li et al[17]. This paper aims to evaluate the utility of six widely reported prognostic indices [acute physiology and chronic health evaluation II (APACHE-II), BISAP, Glasgow score, HAPS, Ranson’s score, SOFA)] in the prediction of three key determinants of disease outcomes: Severity of AP, the need for ICU admission, and mortality from AP.

MATERIALS AND METHODS

This is a retrospective cohort study of all patients admitted for AP under the Department of General Surgery at Tan Tock Seng Hospital, Singapore, between July 2009 and September 2016. Patients admitted under other departments were excluded from this study. As per departmental practice, all patients were scored using both the Ranson’s and Glasgow scores within the first 48 h of admission. Values for APACHE II score, BISAP, HAPS, and SOFA within 24 h of admission were retrospectively obtained based on laboratory results and patient evaluation recorded on a secure hospital-based online electronic platform. SOFA scores were only calculated on admission. Patients with grossly insufficient data to compute any of the six scorings were excluded from the study. On the occasion where laboratory values, particularly ventilator settings and blood gas data, were unavailable for patients not admitted to the ICU, no points were given for the missing values. Data with < 10% missing data was imputed via mean substitution. Other patient information such as demographics, disease etiology, and patient outcomes were also derived from electronic medical records. This study was approved by the institutional review board, reference number DSRB 2016/00825.

Definitions

Diagnosis and complications of AP: Definitions relating to AP diagnosis and complications were adopted from the Revised Atlanta classification[18]. Patients with any two out of the following three clinical parameters satisfied the diagnostic criteria for AP: (1) Characteristic abdominal pain, maximal pain over the epigastric area often with radiation to the back; (2) Biochemical features of AP, characterized as a measured serum lipase or amylase of > 3 times the upper limit of normal as defined by the local laboratory; and (3) Presence of characteristic radiological findings consistent with AP on contrast-enhanced computer tomography, magnetic resonance imaging or ultrasonography.

Complications of AP were categorized into local and systemic complications. Local complications (LC) include acute peripancreatic fluid collections, pancreatic pseudocysts, acute necrotic collections, walled-off necrosis, gastric outlet dysfunction, splenic and portal vein thrombosis, and colonic necrosis. Systemic complications were defined as exacerbation of pre-existing comorbidity by AP and distinct from persistent OF. OF, specifically renal, cardiovascular, or respiratory failure, was defined as per the modified Marshal scoring system (score of 2 or more for any of the above systems)[19].

Study outcomes

Severity stratification of AP: According to Revised Atlanta guidelines[18], AP can be graded as mild, moderately severe, or severe. The mild AP was defined in the absence of LC or OF. Mild AP is typically self-resolving within a week. Moderately severe disease was defined as AP in the presence of either LC or transient OF resolving within 48 h. SAP was defined as AP in the presence of persistent OF lasting more than 48 h.

ICU admission

Any patient admitted to the ICU for a minimum of 24 h was considered to have received care in ICU.

Mortality

Mortality was defined as the patient's death within the same hospital admission from any cause attributable to AP.

Prognostic scoring

Ranson’s score was the first developed to risk-stratify AP[8] and consists of 11 parameters, five scored at admission, and six scored at 48 h after admission. Glasgow score, otherwise known as the Glasgow-Imrie or Imrie score, was first described by Blamey et al[20] and consists of eight variables scored with values at 48 h after admission. The APACHE-II score was initially developed to predict survival in the ICU setting but was eventually proposed as a suitable assessment tool in AP[21-23]. APACHE-II consists of 15 laboratory variables measured at the time of admission. The BISAP score consists of five variables retrospectively derived from a large population-based study for the early prediction of mortality in AP[10], and values are scored upon admission. The HAPS was first described by Lankisch et al[11]. It was designed to rule out patients with AP requiring ICU treatment and scored within 30 min of admission. The SOFA score developed by Vincent et al[12] and validated for use in AP by Adam et al[13] in 2013 consists of five variables scored within 24 h of admission.

Statistical analysis

Statistical analysis was conducted using SPSS Statistics Version 23 (Armonk NY: IBM Corp). Categorical variables are presented as absolute numbers and proportions. Continuous variables are presented as mean ± standard deviation (SD). Variance within categorical variables was assessed using the Chi-square test or Fisher's exact test where appropriate. Variance within continuous variables was measured using the student's t-test. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive and negative likelihood ratios (LR+ and LR-), diagnostic odds ratio (DOR), and overall accuracy were calculated for each prognostic index with regards to disease severity, ICU admission, and mortality. Receiver operating characteristic (ROC) curves and areas under the curve (AUC) were calculated for each score. Pairwise comparisons between AUCs of each index's ROC were conducted using the nonparametric method described by DeLong et al[24] in 1988.

RESULTS

Patient characteristics

From July 2009 to September 2016, 675 patients were managed for AP. Four patients failed to satisfy the diagnostic criteria for AP, and two patients had missing global data. Of the remaining 669 patients, a total of 16 patients was excluded due to insufficient data to compute APACHE-II score (n = 16), HAPS score (n = 3), Ranson’s score (n = 4), and Glasgow score (n = 3). Altogether, 22 (3.3%) were excluded, and 653 patients were included.

The mean age ± SD of patients was 58.7 ± 17.5 years (range 20-98 years). There was a male predominance (n = 383, 58.7%). Hypertension (n = 339, 51.9%), hyperlipidemia (n = 235, 36%) and type 2 diabetes mellitus (T2DM) (n = 204, 31.2%) were common co-morbid conditions. 125 (19.1%) and 159 (24.4%) patients had a history of smoking and alcohol consumption, respectively. Gallstones was the most common aetiology (n = 404, 61.9%), followed by alcohol (n = 38, 5.8%) and hypertriglyceridemia (n = 19, 2.9%). 81 (12.4%) patients developed SAP, 20 (3.1%) patients required ICU admission, and 12 (1.8%) deaths were attributed to AP, all of whom had SAP.

Severity-stratified patient demographic and clinical profile is shown in Table 1. Patients with SAP were significantly older (64.2 vs 57.9, P = 0.002) and had higher prevalence of hypertension (69.1% vs 49.5%, P = 0.005), T2DM (44.4% vs 29.4%, P = 0.025) and ischaemic heart disease (22.2% vs 11.0%, P = 0.012). Asthma (4.9% vs 5.8%, P = 0.038) and smoking history (8.6% vs 20.5%, P = 0.042) were less prevalent among SAP patients. Most common interventions were cholecystectomy (n = 186, 28.5%), endoscopic retrograde pancreatography (n = 89, 13.6%) and endoscopic ultrasound (n = 12, 1.8%).

Table 1 Demographic and clinical profile of patients with acute pancreatitis n (%).

Characteristic	Overall study population (n = 653)	Mild to moderately severe AP (n = 572)	Severe AP (n = 81)	P value
Mean age at admission (Range)	58.7 ± 17.5 (20-98)	57.9 ± 17.0 (20-95)	64.2 ± 20.0 (20-98)	0.002a
Gender
Male	383 (58.7)	334 (58.4)	49 (60.5)	0.285
Ethnicity				0.099
Chinese	458 (70.1)	391 (68.4)	67 (82.7)
Malay	43 (6.6)	36 (6.3)	7 (8.6)
Indian	108 (16.5)	102 (17.8)	6 (7.4)
Others	44 (6.7)	43 (7.5)	1 (1.2)
Comorbidities
Hypertension	339 (51.9)	283 (49.5)	56 (69.1)	0.005a
T2DM	204 (31.2)	168 (29.4)	36 (44.4)	0.025a
Hyperlipidemia	235 (36)	198 (34.6)	37 (45.7)	0.373
Ischaemic heart disease	81 (12.4)	63 (11.0)	18 (22.2)	0.012a
Cerebrovascular disease	51 (7.8)	43 (7.5)	8 (9.9)	0.768
Renal impairment	42 (6.4)	33 (5.8)	9 (11.1)	0.195
COPD	13 (2.0)	9 (1.6)	4 (4.9)	0.217
Asthma	37 (5.7)	33 (5.8)	4 (4.9)	0.038a
Others	120 (18.4)	107 (18.7)	13 (16.0)	0.919
Medications
Immunosuppressed	2 (0.3)	2 (0.3)	0	0.467
Steroids	9 (1.4)	6 (1.0)	3 (3.7)	0.214
Anticoagulants	32 (4.9)	24 (4.2)	8 (9.9)	0.065
History of smoking	125 (19.1)	118 (20.5)	7 (8.6)	0.042a
History of alcohol consumption	159 (24.4)	145 (25.4)	14 (17.3)	0.454
Previous pancreatic disease	76 (11.6)	67 (11.7)	9 (11.1)	0.112
Chronic pancreatitis	30 (4.6)	29 (5.1)	1 (1.2)	0.098
Previous Cholecystectomy	44 (6.7)	38 (6.6)	6 (7.4)	0.809
Etiology
Gallstones	404 (61.9)	350 (61.2)	54 (66.7)	0.390
Alcohol	38 (5.8)	34 (5.9)	4 (4.9)	0.437
Idiopathic	61 (9.3)	52 (9.1)	9 (11.1)	0.634
Hypertriglyceridemia	19 (2.9)	14 (2.4)	5 (6.2)	0.161
Autoimmune	4 (0.6)	4 (0.7)	0	0.491
Hypercalcemia	3 (0.5)	2 (0.3)	1 (1.2)	0.235
Drug induced	6 (0.9)	3 (0.5)	3 (3.7)	0.065
Others	47(7.2)	44 (7.7)	3 (3.7)	0.343

^aP < 0.05. T2DM: Type 2 diabetes mellitus; COPD: Chronic obstructive pulmonary disease; AP: Acute pancreatitis; Idiopathic: Acute pancreatitis with no etiology despite extensive work up; Others: Etiologies of acute pancreatitis include trauma, pancreas cystic neoplasms, malignancy, iatrogenic causes such as endoscopic retrograde cholangiopancreatography.

Score comparison

Comparative characteristics of all six scores regarding the severity stratification, ICU admission, and mortality are shown in Table 2. AUC of the six scores in predicting SAP, ICU admission, and mortality are shown in Figures 1-3, respectively.

Figure 1 Area under receiver-operator curve for prognosticating severity in acute pancreatitis. ROC: Receiver operating characteristic; SOFA: Sequential Organ Failure Assessment; HAPS: Harmless acute pancreatitis score; BISAP: Bedside index of severity in acute pancreatitis; APACHE II: Acute physiology and chronic health evaluation–II.

Figure 2 Area under receiver-operator curve for prognosticating intensive care unit admission in acute pancreatitis. ROC: Receiver operating characteristic; SOFA: Sequential Organ Failure Assessment; HAPS: Harmless acute pancreatitis score; BISAP: Bedside index of severity in acute pancreatitis; APACHE II: Acute physiology and chronic health evaluation–II.

Figure 3 Area under receiver-operator curve for prognosticating mortality in acute pancreatitis. ROC: Receiver operating characteristic; SOFA: Sequential Organ Failure Assessment; HAPS: Harmless acute pancreatitis score; BISAP: Bedside index of severity in acute pancreatitis; APACHE II: Acute physiology and chronic health evaluation–II.

Table 2 Evaluation of prognostic indices for severe acute pancreatitis (n = 81), intensive care unit admission (n = 20), and mortality in acute pancreatitis (n = 12).

Score	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	LR+	LR-	Diagnostic odds ratio	Accuracy
SAP
HAPS ≥ 1	79.0	49.7	18.2	94.4	1.569	0.423	3.712	53.3
BISAP ≥ 3	24.7	95.3	42.6	89.9	5.231	0.790	6.618	86.5
APACHE II ≥ 8	80.2	63.3	23.6	95.8	2.186	0.312	7.003	65.4
Ranson’s ≥ 3	92.6	51.9	21.4	98.0	1.926	0.143	13.5	57.0
Glasgow ≥ 3	76.5	68.5	25.6	95.4	2.432	0.342	7.106	69.5
SOFA ≥ 7	13.6	99.7	84.6	89.1	38.84	0.867	44.786	89.0
ICU admission
HAPS ≥ 1	90.0	47.2	5.1	99.3	1.706	0.212	8.057	29.9
BISAP ≥ 3	25.0	93.4	10.6	97.5	3.768	0.803	4.690	91.3
APACHE II ≥ 8	100.0	59.6	6.6	100.0	2.473	0	Nil	60.5
Ranson ≥ 3	100.0	47.9	5.7	100.0	1.918	0	Nil	49.5
Glasgow ≥ 3	75.0	64.5	7.0	99.3	2.110	0.388	5.440	65.1
SOFA ≥ 7	40.0	99.2	61.5	98.1	50.64	0.605	83.733	97.4
Mortality in AP
HAPS ≥ 1	83.3	46.6	2.8	99.3	1.562	0.357	4.371	29.9
BISAP ≥ 3	25	93.1	6.4	98.5	3.642	0.805	4.523	91.9
APACHE II ≥ 8	100	58.7	3.6	100	2.419	0	Nil	59.1
Ranson’s ≥ 3	100	47.3	3.4	100	1.896	0	Nil	48.2
Glasgow ≥ 3	75	63.8	4.1	99.5	2.072	0.392	5.289	64.2
SOFA ≥ 7	50.0	98.9	46.2	99.1	45.786	0.506	90.571	98.0

HAPS: Harmless acute pancreatitis score; BISAP: Bedside index of severity in acute pancreatitis; APACHE II: Acute physiology and chronic health evaluation–II; PPV: Positive predictive value; NPV: Negative predictive value; SAP: Severe acute pancreatitis; ICU: Intensive care unit; AP: Acute pancreatitis. HAPS, BISAP, and APACHE II were calculated at admission, Ranson's and Glasgow's were calculated at 48 h post-admission.

Prediction of SAP

In predicting SAP, there was a significant variation between scores: Sensitivity (13.6%-92.6%) and specificity (49.7%-99.7%). Ranson’s score demonstrated the highest sensitivity (92.6%) but one of the lowest specificities (51.9%), only higher specificity than HAPS (49.7%). SOFA score demonstrated the lowest sensitivity (13.6%) but the highest specificity (99.7%). Positive predictive value (PPV) of all scores fell short of 50% aside from SOFA (84.6%). All scores demonstrated consistently high and comparable negative predictive values (NPV) in the prediction of severity. Ranson’s score had the highest NPV (98.0%). Of all scores, SOFA demonstrated the most significant positive likelihood ratio (LR+) (38.84), DOR (44.786), and overall accuracy (89.0%).

Figure 1 shows the area under receiver-operator curves (AUROC) of all scores for predicting SAP. SOFA (0.966) and 48-h Ranson’s score (0.857) demonstrated the highest AUROC. HAPS demonstrated the lowest AUROC (0.687). Nonparametric comparison of AUROC between SOFA and 48-h Ranson’s score revealed SOFA had significantly greater AUROC (difference 0.109, P < 0.0001). SOFA score had a significantly higher AUROC than all other scores (all other scores P < 0.0001). 48-h Ranson’s score had significantly higher AUROC as compared to APACHE-II (P = 0.0163), BISAP (P < 0.0001), Glasgow score (P = 0.0007), and HAPS (P < 0.0001).

ICU admission

In predicting ICU admission, sensitivity (25.0%-100%) and specificity (47.2%-99.2%) varied greatly among the various scores. APACHE-II and Ranson’s scores displayed 100.0% sensitivity for predicting ICU admission. While BISAP demonstrated the lowest sensitivity (25.0%), it displayed high specificity (93.4%). SOFA demonstrated the highest specificity (99.2%). PPV of all scores was low (5.1%-10.6%) except SOFA (61.5%). All scores demonstrated high and comparable NPV in predicting ICU admission (97.5-100.0%). Of all scores, SOFA demonstrated the greatest LR+ (50.64), DOR (83.73), and overall accuracy (97.4%).

Figure 2 shows the AUROC of all scores for predicting ICU admission. SOFA (0.943) and 48-h Ranson’s score (0.946) demonstrated the highest scores. Nonparametric comparison of AUROC of SOFA and 48-h Ranson’s score revealed no significant difference (difference 0.003, P = 0.933). SOFA score had significantly higher AUROC than scores of HAPS (P = 0.0009), BISAP (P < 0.0001), Glasgow (P = 0.0069), and APACHE-II (P = 0.001). 48-h Ranson’s score has significantly higher AUROC compared to other scores such as HAPS (P = 0.0001), BISAP (P < 0.0001), Glasgow (P = 0.0066), and APACHE-II (P = 0.0005).

Mortality in AP

In predicting mortality, variance in sensitivity (25.0%-100%) and specificity (47.2%-98.9%) were once again noted. APACHE-II and Ranson’s both displayed 100.0% sensitivity for predicting mortality. In contrast, BISAP demonstrated the lowest sensitivity (25.0%). SOFA score demonstrated the highest specificity (98.9%). PPV of all scores was low (2.8%-46.2%). All scores demonstrated high and comparable NPV in predicting mortality (98.5%-100.0%). Of all scores, the SOFA score displayed the highest LR+ (45.786), DOR (90.571), and overall accuracy (98.0%) in predicting mortality.

Figure 3 shows the AUROC of all scores for predicting mortality. SOFA (0.968) and 48-h Ranson’s score (0.917) demonstrated the highest scores. Nonparametric comparison of AUROC of SOFA and 48-h Ranson’s score revealed no significant difference (difference 0.051, P = 0.0.150). SOFA score had significantly higher AUROC than scores of HAPS (P = 0.0007), BISAP (P = 0.001), Glasgow (P = 0.0243), and APACHE-II (P = 0.0003). 48-h Ranson’s score has significantly higher AUROC compared to other scores such as HAPS (P = 0.00690), BISAP (P = 0.0037), and APACHE-II (P = 0.0203) but did not yield a significant difference when compared to Glasgow score (P = 0.139).

DISCUSSION

AP remains an important surgical condition, were determining its severity remains integral in guiding its management. We evaluated six standard prognostic scoring systems in predicting severity, ICU admission, and mortality. To our knowledge, this is the first study to compare the six prognostic scoring systems (APACHE-II, BISAP, Glasgow Score, HAPS, Ranson’s score, SOFA) in a single sitting. In our study, the SOFA score and 48-h Ranson’s score demonstrated a high correlation to predict the severity of AP, ICU admission, and mortality. The SOFA score had better statistical parameters and thus marginally outperformed 48-h Ranson’s score.

Patient characteristics

AP patients demonstrated a comorbidity profile similar to those in other studies[10,13] with a predominance of cardiovascular and metabolic conditions. Male predominance in AP is similarly reported in other studies[25,26]. Predominant etiologies of AP identified were gallstones (61.9%) and alcohol (5.8%), consistent with the reported trend in the American College of Gastroenterology Guidelines (40%-70% for gallstones, 25%-35% for alcohol)[27]. The lower prevalence of alcoholic pancreatitis in our population may reflect lower consumption rates in the Asian population[25,28].

Prediction of SAP

For the more established scoring systems of APACHE-II, Glasgow score, Ranson’s score, and BISAP, the high NPV corroborates current literature when predicting severity[26,29,30]. Simoes et al[25] present in their retrospective study of 126 patients the Ranson’s score to have the highest NPV (95.7%), followed by APACHE-II (91.4% at 48 h) and then Glasgow score (87.7%)[25]. Our study follows a similar trend of Ranson’s score having the highest NPV (98.0%). In a study by Cho et al[31] involving 161 patients, a high BISAP NPV (92.7%) was noted, which was consistent with our study's NPV as well (89.9%)[31]. Similarly, Gao et al[32] found that the 48-h Ranson’s score has a reasonably high AUROC (0.830), comparable to APACHE-II and BISAP[32]. Our study presents data to supplement the current literature on their NPV for determining SAP for the newer scoring systems of HAPS and SOFA score. To our knowledge, the NPV for HAPS in determining severity has only been validated by Ma et al[33] in 2020. In a prospective study involving 703 patients, Ma et al[33] reported high NPV for HAPS (97.7%), comparable to our results[33]. For the SOFA scoring system, a study by Zhou et al[34] involving 406 patients revealed that the NPV of SOFA (95.1%) was high, a finding consistent with our study (89.1%)[34]. Notably, even simple bedside scoring indices requiring five or fewer variables (HAPS, BISAP) have high NPVs. Zhou et al[34] even found the BISAP score to have the highest NPV (98.1%). Hence, these simple bedside scores' utility lies in their ability to screen out mild disease at the onset, allowing physicians to divert their focus to patients with SAP.

The incidence of SAP within our cohort (12.4%) is similar to that experienced internationally, with previously reported SAP rates ranging from 12%-20% of AP cases[4-6]. Risk factors we noted include older age, hypertension, T2DM, and ischemic heart disease. Zhou et al[34] also found similar trends with high incidence of T2DM (P = 0.004), but not cardiovascular disease (P = 0.123) and age (P = 0.162)[34]. This could be explained by variation in diagnostic criteria as well as the definition of comorbidities. Thus far, no large studies have determined an association between asthma and the severity of AP. In another retrospective study by Kim et al[35] involving 905 patients, risk factors for AP included smoking (P = 0.04, OR 7.22 for AP induced by gallstones, P = 0.05, OR 2.59 for AP induced by alcohol consumption)[35]. In our study, smoking and asthma have shown a protective effect on SAP. This could be due to variation in smoking history documentation, and these findings require prospective validation by others. Also, we pooled the data of moderately severe AP patients along with mild AP patients, and this could impact the results. Alcohol history and hyperlipidemia were not statistically significant risk factors for developing SAP. This could be due to the low prevalence of alcohol consumption and the small sample. While hyperlipidemia is a known etiology of AP, there has not been a difference detected in AP severity. In a prospective study by Balachandra et al[36] involving 43 patients, raised triglyceride levels did not correlate with higher APACHE-II scores (r² = 0.0015)[36]. However, at very high levels, a correlation may be possible. A univariate analysis done by Deng et al[37] involving 45 patients with SAP and hypertriglyceridemia (≥ 500 mg/dL) revealed that patients with hypertriglyceridemia tend to have more severe AP with higher APACHE-II scores and overall mortality[37]. Hence, more studies with higher power are necessary to determine hypertriglyceridemia's relationship with SAP.

The AUROC for prognosticating severity in AP was most remarkable for the SOFA score and 48-h Ranson’s score. This is in contrast with Zhou et al[34] study, which reported AUROC for determining severity as BISAP (0.841), Ranson’s (0.806), and SOFA score (0.806). Zhou et al[34] did not note any significant difference between pairwise comparisons of BISAP, SOFA, and 48-h Ranson’s score (BISAP vs SOFA, Z = 0.956, P = 0 .339; BISAP vs Ranson’s score, Z = 1.072, P = 0.284; SOFA vs Ranson’s score, Z = 0.000, P = 1.000). It is also worthy to note that a combination of red-cell distribution width was proposed as a combination of severity scoring with BISAP, which gave the highest AUROC in Zhou et al[34]'s study (0.872). However, it must be noted that the AUROC value was still inferior to the AUROC of SOFA score in our study (0.966). Contrasted to our study, it was noted that there were statistically significant differences in DeLong pairwise comparisons between SOFA and all five other scoring systems and between 48-h Ranson’s score and HAPS or BISAP scores. Another study by Hagjer et al[38] involving 60 patients noted the AUROC for determining the severity of AP for higher for BISAP score (0.875) than APACHE-II score (0.872)[38]. 48-h Ranson’s score had a slightly lower AUROC value (0.810). However, the study's low power suggests the need for more higher-powered studies to validate this claim.

ICU admission

The incidence in our study of ICU admissions (3.1%) also aligns to gross estimates in the literature, 3.7% in European cohorts[27,39,40]. However, variations between ICU admission criteria in various institutions should be taken into consideration. In our study, AUROC for 48-h Ranson’s score and SOFA score were the greatest for determining ICU admissions, while the BISAP score yielded a lower AUROC. This is directly compared to the study by Harshit Kumar et al[41], who described a similar trend where Ranson’s score (0.910) and APACHE-II (0.885) yielded good AUROC values, while the BISAP score yielded a better score than our study (0.877)[41]. However, Harshit Kumar et al[41]'s study had a small sample size and thus was not adequately powered. This is the first study to evaluate the utility of scoring indices for determining the likelihood of ICU admission for AP. Most of the literature extrapolate the need for ICU admission from the severity of the AP, akin to how Majdoub et al[26] inferred the need for ICU admission via APACHE-II, BISAP, Glasgow, and Ranson’s scoring systems by evaluating the AUROC predicting mortality and morbidity but did not directly measure the number of patients admitted to ICU[26]. In terms of NPV, both APACHE-II and 48-h Ranson’s scores yielded a 100% NPV rate for ICU admission.

Mortality in AP

The prediction of mortality using the six prognostic indices has been individually fairly well-reviewed in the literature. In a retrospective study by Zhang et al[42] involving 155 patients, the AUROC value for mortality in AP was best represented by the Ranson’s score (0.904), followed by the APACHE-II score (0.812) and the BISAP score (0.791)[42]. This directly contrasts the scores in our case where mortality was best represented by the AUROC values of the SOFA score (0.968) and 48-h Ranson’s score (0.917), followed by the APACHE-II score (0.779). The BISAP score yielded the lowest AUROC value (P = 0.647) in our study. While the general ranking of the scoring systems is similar, it must be essential to note that Zhang et al[42] noted alcohol as the primary etiology in AP (56.7%) and not gallstones (26.4%) explain the differences in AUROC values. Similarly, Khanna et al[14] noted in their retrospective study involving 72 patients, APACHE-II yielded the highest AUROC score for predicting mortality in AP (0.86, CI: 0.77-0.95) followed by Ranson’s score (0.84), Glasgow score (0.83) and BISAP score (0.83)[14]. Other studies corroborate the finding that BISAP scoring does not predict mortality and Ranson’s score as well[32], stating a lower sensitivity compared to Ranson’s score within 48 h of admission and lower specificity than 48-h Ranson’s score[43]. However, the literature has provided differing opinions on the best scoring system to predict mortality. Another retrospective study by Biberci Keskin et al[44] involving 690 patients reported AUROC values to predict in-hospital mortality to be highest when BISAP was used (0.92) when compared to HAPS (0.85) and Ranson’s score on admission (0.82)[44]. While the low AUROC value for Ranson’s score can be explained by the lack of 48-h Ranson’s score data, it is interesting to note the discrepancy in AURCO values for BISAP and HAPS scores compared to our data. Contrastingly, Mikó et al[45] noted in their meta-analysis on predicting mortality that the AUROC of APACHE-II (0.91) is superior to that of Ranson’s score (0.87), which is equivocal to that of BISAP score (0.87)[45]. Gao et al[32] reveal that Ranson’s score yielded the highest AUROC (0.92) among APACHE-II and BISAP[32]. Alternatively, Biberci Keskin et al[44] suggests using the Japanese Severity Score (JSS), which yielded the highest AUROC value for in-hospital mortality in their study (0.94). The discrepancy in scoring AUROC values could be due to the definition of in-hospital mortality used, where a 30-d cap was placed by Biberci Keskin et al[44] compared to our definition of death within the same hospital admission without a time limit. Thus, the evaluation of BISAP score for short-term mortality can be explored. Furthermore, the prognostic accuracy of JSS is heterogenous in the literature describing the JSS as both more accurate[44] and less accurate than 48-h Ranson’s score[43] in separate instances. In the same study by Hagjer et al[38] as mentioned above, the AUROC values for predicting mortality in AP is highest in both APACHE-II score (0.893) and BISAP (0.892) followed by 48-h Ranson’s score (0.803)[38], contrasting both our study and the study by Zhou et al[34]. However, given the small sample size of 60, more higher-powered studies can be considered before making a judgement as to why there is such a discrepancy.

Overall, despite the differences in AUROC values, the consensus in the literature support 48-h Ranson’s and APACHE-II scores as good predictors for mortality in AP. The SOFA score has yet to be studied aside from the initial study by Adam et al[13], where a mean SOFA score yielded an equivocally high AUROC score (AUROC = 0.904)[13]. Adam et al[13] also compared SOFA scores after ICU admission vs Ranson’s and APACHE II for prognosis of mortality. Authors reported that SOFA score trends after ICU admission were a good indicator for mortality prediction[13]. The study examined 39 patients with SAP in the ICU, with an overall mortality of 71%. SOFA scores correlated significantly with mortality, while APACHE II had no statistically significant association with mortality. Within the study, all patients with SOFA score ≥ 11 at any time during ICU stay had higher mortality (80% sensitivity, 79% specificity, AU 0.837). This is comparable to our study in patients with SOFA score ≥ 7 (50% sensitivity, 98.9% specificity, AUC 0.968 in the prognosis of mortality secondary to AP. Another related study by Tee et al[46] demonstrated the SOFA score on day seven to be reliable in predicting late mortality in AP[46]. Interestingly, SOFA score on admission (AUC = 0.67) and 48 h after admission (AUC = 0.765) had smaller AUROC compared with the APACHE II score (AUC = 0.821) in the prediction of mortality. However, the SOFA score on day seven was the best in predicting mortality (AUC = 0.858). The utility of SOFA in predicting disease outcomes is congruent with the underlying pathophysiology of SAP, with OF being recognized as the bridge to poor outcomes, as reported by Buter et al[47]. As the pancreas is a highly vascularized organ where both foregut and midgut vessels meet[48], bradykinin-mediated vasodilation and increase in vascular permeability cause further pancreatic ischemia, systemic hypotension, and subsequent OF[49]. Hence, the trending of SOFA scores throughout admission is a valuable tool to alert physicians to both the early critical phase due to systemic inflammatory response syndrome and the late critical phase, two weeks later, due to increased infection risks[50].

Limitations

Our study has several limitations. Firstly, this is a retrospective single-center study, and thus results cannot be generalized across the diverse demographic population in different geographic locations. Clinical variables such as the onset of abdominal pain rely on recall bias of patients and accuracy of clinical records, and these limitations can only be addressed by prospective study design. Though we had missing data, it was low (3.3%) and, in our opinion, is acceptable. Our study analyses prognostic indices at admission and not trends. It is known that response to resuscitation and daily trends are essential determinants to predict severity and mortality. Further studies can be done comparing the utility of trending such scores throughout inpatient stay. We do not routinely perform C-reactive protein, and thus, we could not include it in our analysis.

CONCLUSION

Overall, this study's six prognostic indices demonstrated high NPV in predicting severity, ICU admission, and mortality in AP. SOFA score and 48-h Ranson’s score are superior to other prognostic scorings (Glasgow score, APACHE II, BISAP, HAPS) in severity stratification, prediction of ICU admission, and mortality.

ARTICLE HIGHLIGHTS

Research background

Acute pancreatitis (AP) is a common surgical disease, and severe AP (SAP) can be fatal. Many prognostic indicators, including; acute physiology and chronic health evaluation II (APACHE II), bedside index of severity in acute pancreatitis (BISAP), Glasgow score, harmless acute pancreatitis score (HAPS), Ranson score, and sequential organ failure assessment (SOFA) assesses the severity of AP and predicts mortality.

Research motivation

An accurate scoring system on admission of AP is critical to guide patient disposition and aggressiveness of treatment, resulting in both better patient care as well as better distribution of resources for each institution. Few studies have compared the efficacy of these newer scores in predicting disease severity against classic scores such as Ranson's score and Glasgow score, and fewer still have reported their utility in predicting key clinical outcomes such as intensive care unit (ICU) admission and mortality in AP.

Research objectives

A major concern for clinicians is the gross heterogeneity in clinical presentation and identifying patients predicted to manifest SAP. We evaluated these indices' utility in predicting severity, ICU admission, and mortality.

Research methods

This is a retrospective cohort study. All patients were scored using Ranson and Glasgow scores within the first 48 h after admission. The APACHE II score, BISAP, HAPS, and SOFA values within 24 h of admission are retrospectively obtained based on laboratory results and patient evaluations recorded on a secure online electronic platform of the hospital. Data with missing data < 10% are extrapolated by means of replacement. Other patient information, such as demographics, disease causes, and patient results are also derived from electronic medical records.

Research results

The mean age was 58.7 ± 17.5 years, with 58.7% males. Gallstones (n = 404, 61.9%), alcohol (n = 38, 5.8%), and hypertriglyceridemia (n = 19, 2.9%) were more common aetiologies. 81 (12.4%) patients developed SAP, 20 (3.1%) required ICU admission, and 12 (1.8%) deaths were attributed to SAP. Ranson’s score and APACHE-II demonstrated the highest sensitivity in predicting SAP (92.6%, 80.2% respectively), ICU admission (100%), and mortality (100%). While SOFA and BISAP demonstrated lowest sensitivity in predicting SAP (13.6%, 24.7% respectively), ICU admission (40.0%, 25.0% respectively) and mortality (50.0%, 25.5% respectively). However, SOFA demonstrated the highest specificity in predicting SAP (99.7%), ICU admission (99.2%), and mortality (98.9%). SOFA demonstrated the highest positive predictive value, positive likelihood ratio, diagnostic odds ratio, and overall accuracy in predicting SAP, ICU admission, and mortality. SOFA and Ranson’s score demonstrated the highest area under receiver-operator curves at 48 h in predicting SAP (0.966, 0.857 respectively), ICU admission (0.943, 0.946 respectively), and mortality (0.968, 0.917 respectively).

Research conclusions

Overall, the six prognostic indices in this study demonstrated high negative predictive values in prediction of severity, ICU admission and mortality in AP. SOFA score and Ranson score at 48 h are superior to other prognostic scorings (Glasgow score, APACHE II, BISAP, HAPS) in severity stratification, prediction of ICU admission and mortality in AP.

Research perspectives

As we provide a retrospective single-center study, future renditions of this study could include multi-center analysis spanning across different countries to reduce bias. Further studies can also compare the utility of trending such scores throughout inpatient stay rather than retrospectively from patients’ results on admission.