Use of radiograph scoring systems to assess pulmonary disease severity in patients with COVID-19 pneumonia

Abstract

BACKGROUND

Severe acute respiratory syndrome coronavirus 2 causes pneumonia in most hospitalized patients, often leading to hypoxemia and the need for supplemental oxygen. While chest computed tomography is highly sensitive, chest radiographs (CXR) offer a practical alternative in high-volume settings. Scoring systems like Radiographic Assessment of Lung Edema (RALE) and BRIXIA standardize CXR interpretation and quantify severity, but their relationship with oxygen delivery requirements in coronavirus disease 2019 (COVID-19) patients remains unclear.

AIM

To evaluate whether the initial emergency department (ED) radiograph could predict subsequent oxygen support requirements. The secondary aim was to assess inter- and intra-rater agreement of the scoring systems.

METHODS

This retrospective cohort study examined consecutive COVID-19 patients presenting to a large tertiary hospital ED (May-June 2020) who required admission and underwent CXR within 24 hours of arrival. Infiltrate severity on ED radiographs was scored using the BRIXIA and RALE systems. Oxygen support was categorized by delivery device, and associations were examined using logistic regression.

RESULTS

Data was analyzed from 950 COVID-19 patients (90.6% male, mean age: 48.4 ± 12.3 years). Predictive performance showed notable variation: At ED admission, both BRIXIA and RALE scores had the highest discriminatory ability [area under the curve (AUC) = 0.74; 95% confidence interval (CI): 0.69-0.79] for predicting oxygen delivery via high flow nasal cannula/continuous positive airway pressure/Bi-level positive airway pressure. Prediction for non-rebreather mask yielded lower AUCs (BRIXIA: 0.65; RALE: 0.62), with nasal cannula use showing limited discrimination (BRIXIA: 0.56; RALE: 0.54). During hospitalization, predictive performance remained modest across all modalities. The AUCs for intubation were 0.63 (BRIXIA) and 0.62 (RALE), while for high flow nasal cannula/continuous positive airway pressure/Bi-level positive airway pressure, values dropped slightly to 0.62 and 0.59, respectively. Non-rebreather mask prediction maintained an AUC of 0.62 for both scores, and nasal cannula predictions remained low (BRIXIA: 0.56; RALE: 0.52). Inter- and intra-rater agreement was excellent in both scores, with inter-rater agreement at 95% (95%CI: 0.94-0.96) and intra-rater agreement at 97% (95%CI: 0.96-0.98) for BRIXIA and 98% (95%CI: 97-98) for RALE.

CONCLUSION

Both RALE and BRIXIA scores effectively predicted the need for advanced respiratory support in ED COVID-19 patients and demonstrated excellent inter-rater and intra-rater reliability. While their predictive power diminished during hospitalization, both scores remain valuable for initial triage, with BRIXIA particularly useful for ruling out the need for high-level oxygen support.

Key Words: COVID-19; SARS-CoV-2; BRIXIA; Radiographic Assessment of Lung Edema; Radiographic scoring; Reliability; Emergency department

Core Tip: Radiographic Assessment of Lung Edema and BRIXIA chest X-ray scoring systems are effective in predicting the need for advanced respiratory support in coronavirus disease 2019 patients presenting to the emergency department. Both methods show strong prognostic value for identifying patients who require advanced oxygen delivery devices in the emergency department and during their hospital stay. In addition, the two scoring systems demonstrated excellent inter- and intra-rater reliability among emergency medicine physicians, intensivists, and radiologists. Given their comparable performance in clinical practice, the choice between the Radiographic Assessment of Lung Edema and the BRIXIA scores can be determined by institutional preference.

INTRODUCTION

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was originally described in Wuhan, China, in late 2019[1,2]. The World Health Organization (WHO) declared the outbreak a pandemic on March 11, 2020[3]. Hospitals diagnosed pneumonia in around 91% of the admitted patients, with 5% requiring admission to intensive care facilities[4]. The most common presentation requiring hospital admission was hypoxemia, and the majority of hospitalized patients required oxygen therapy[5]. Chest computed tomography (CT) is the most sensitive modality for detecting early coronavirus disease 2019 (COVID-19)-related lung abnormalities[6,7]. Serial CT scans at 3-day to 7-day intervals effectively track disease progression from diagnosis through discharge[8-10]. However, high emergency department (ED) volumes limit its use. Chest radiographs (CXR) offer a practical alternative for pneumonia diagnosis[11], though with variable diagnostic accuracy[12-14]. This variability underscores the potential utility of structured scoring systems to standardize CXR interpretation and severity quantification.

The Radiographic Assessment of Lung Edema (RALE) score is a semi-quantitative scoring system[15] that evaluates lung edema severity by assessing the extent and density of alveolar opacities in each of the four lung quadrants on CXR, with scores ranging from 0 to 48. It has primarily been studied in respiratory distress syndrome patients and serves as a standardized tool for quantifying pulmonary opacification. Developed by Borghesi and Maroldi[16] for COVID-19 assessment, the BRIXIA score quantifies lung involvement on chest X-rays through a six-zone system (upper: I/IV; middle: II/V; lower: III/VI), with each zone scored 0-3 based on abnormality severity. The total score (0-18) correlates with parenchymal damage extent, disease severity, and mortality risk[16]. There is currently no published data correlating BRIXIA and RALE scores with the use of various oxygen delivery devices in COVID-19 patients, either at the time of admission to the ED or during the course of hospitalization. This study aims to investigate the ability of radiographs to predict oxygen support requirements and assess the inter- and intra-rater agreement of BRIXIA and RALE scores among clinicians.

MATERIALS AND METHODS

Study design and setting

This retrospective cohort study analyzed COVID-19 patients presenting to the 390-bed ED of a tertiary referral center in the Gulf region (Qatar) during the initial pandemic wave (May 15, 2020-June 30, 2020).

Study population

The study included consecutive adult patients who: (1) Received a chest radiography within 24 hours of ED assessment; (2) Had RT-PCR-confirmed SARS-CoV-2 infection; and (3) Required supplemental oxygen on admission or during hospitalization. Exclusion criteria consisted of: (1) Patients requiring pre-existing home oxygen supplementation; (2) Those who underwent endotracheal intubation or intercostal drain placement before initial radiography; and (3) Cases with suboptimal radiographic quality precluding accurate interpretation.

Data collection

Trained research assistants systematically extracted clinical data from the Cerner^® electronic health record system (Cerner Corporation, Kansas City, MO, United States), including demographic information, physiological parameters, biochemical markers, and oxygen delivery device specifications. To maintain data quality and accuracy, a randomly selected 20% subset of all records underwent independent dual review by senior researchers. All collected data were securely stored in password-protected Excel^® spreadsheets (Microsoft, Redmond, WA, United States), limiting access to only authorized study personnel throughout the research period. Any discrepancy was resolved by the leader of the data collection team.

Variables

The primary outcome was the requirement for supplemental oxygen, defined as the use of a nasal cannula, face mask, non-rebreather mask, high-flow nasal cannula, non-invasive ventilation, or invasive mechanical ventilation during the ED stay or hospitalization. Secondary outcomes included escalation in oxygen delivery mode. The main exposures were chest radiographic severity scores, specifically RALE and BRIXIA. Predictors included demographic characteristics (age and sex), comorbidities [no comorbidities, diabetes mellitus, hypertension, coronary artery disease/congestive heart failure, asthma/chronic obstructive pulmonary disease, chronic kidney disease, and others], physiological parameters (temperature, heart rate, respiratory rate, systolic blood pressure, and oxygen saturation), and laboratory biomarkers (C-reactive protein and white blood cell count).

Minimizing bias and standardizing radiograph interpretation

To minimize potential bias, all eligible patients meeting inclusion criteria were included to reduce selection bias. Radiographic assessment was standardized: Eight senior clinicians (five emergency medicine physicians, two critical care physicians, and one radiologist; Supplementary Table 1) received a 3-hour training program on the RALE and BRIXIA scoring systems. Each clinician interpreted 50 radiographs, with 30 reviewed twice to assess reliability. Cases were randomly assigned to clinicians, and scoring was blinded to reduce observer bias. Furthermore, all clinicians adhered to the standardized oxygen therapy guidelines from Hamad General Hospital (Supplementary Figure 1), ensuring consistency in clinical decision-making and eliminating bias arising from variable treatment protocols. Multivariable models were adjusted for potential confounders such as age and baseline oxygen delivery devices.

Statistical analysis

All analyses were conducted using STATA version 16.0 (StataCorp LLC, College Station, TX, United States), with continuous variables expressed as mean ± SD or median (interquartile range) based on distributional characteristics, and categorical variables reported as n (%). The predictive performance of radiographic scores for clinical outcomes was evaluated using receiver operating characteristic curve analysis, with area under the curve (AUC) calculations using the DeLong method[17]. Liu’s method was used to find the optimal CXR score cut-point for predicting oxygen delivery use by identifying the highest value of the product of sensitivity and specificity[18]. To ensure robustness, bootstrap resampling with 1000 replicates was utilized to generate robust confidence intervals for the selected cut-point. For categorical outcomes, including oxygen requirements, we performed logistic regression analyses, presenting both crude and adjusted odds ratios with 95% confidence intervals (CIs). Multivariable models incorporated sequential adjustment for potential confounders, beginning with a demographic factor (age) and progressively including clinical variables (initial oxygen delivery device in the ED on admission) to isolate the independent predictive value of radiographic findings.

To assess the consistency of radiographic scoring among clinicians, we evaluated both inter- and intra-rater reliability using complementary statistical measures. Weighted Scott’s/Fleiss’ Kappa[19] and Gwet’s AC[20] coefficients were calculated to quantify agreement. As a sensitivity analysis, we additionally computed agreement coefficients treating the scores as continuous variables with both linear and quadratic weighting schemes. All reliability estimates were reported with their corresponding 95%CIs. Bland-Altman analysis[21] was performed to evaluate individual clinician scoring consistency. Agreement was interpreted according to Gwet’s probabilistic benchmarking method: < 0 = poor, 0-0.2 = slight, 0.2-0.4 = fair, 0.4-0.6 = moderate, 0.6-0.8 = substantial, and 0.8-1.0 = almost perfect[20].

Ethical considerations

The Institutional Medical Research Council approved this study (No. MRC-05-233). Informed consent was waived, as the study involved a retrospective analysis of anonymized clinical data. All procedures complied with institutional and international ethical standards (Declaration of Helsinki, Good Clinical Practice, and Qatar Ministry of Public Health regulations). Protected health information remained secured in password-encrypted databases accessible only to study investigators.

RESULTS

Baseline characteristics

A final cohort of 950 participants with complete clinical data was obtained following exclusion of 137 datasets from an initial screening of 1087 (Figure 1). These participants comprised the study population, which exhibited a strong male predominance (90.6%), reflecting Qatar’s demographic profile. Patients demonstrated significantly higher oxygen support requirements during hospitalization than at initial ED presentation. Upon their arrival at the ED, the majority of patients (50.6%) were on room air and were able to maintain oxygen saturation levels of 90%-94% without the need for supplemental oxygen. Nasal cannulae were used in 35.6% of cases, while the use of non-rebreather mask (NRBM) (9.4%) and advanced support, such as high flow nasal cannula (HFNC), continuous positive airway pressure (CPAP), or Bi-level positive airway pressure (BiPAP) (2.1%), was limited. During hospitalization, oxygen support intensified: Nasal cannula 93.7%, NRBM 40.3%, and HFNC/CPAP/BiPAP 19.2%. Furthermore, 14.0% required intubation/tracheostomy. Table 1 displays the demographic, clinical, radiographic, and oxygen therapy characteristics of COVID-19 patients.

Figure 1 Patient selection. COVID-19: Coronavirus disease 2019.

Table 1 Summary of demographic characteristics, BRIXIA and Radiographic Assessment of Lung Edema scores, and oxygen use via oxygen delivery device, n (%).

Category	Variable	Summary
Total sample size	n	950 (100.0)
Demographics	Age (mean ± SD)	48.4 (13.6)
	Male	861 (90.6)
Chest X-ray scores, median (IQR)	BRIXIA	8.0 (6.0)
	RALE	12.0 (16.0)
Comorbidities	No comorbidities	475 (50.0)
	Diabetes mellitus	306 (32.2)
	Hypertension	293 (30.8)
	Coronary artery disease/CHF	59 (6.2)
	Asthma/COPD	35 (3.7)
	Chronic kidney disease	36 (3.7)
	Others	68 (7.1)
Clinical variables median (IQR)	Temperature	38.0 (1.3)
	Respiratory rate	22.0 (6.0)
	Heart rate	102.0 (24.3)
	Systolic blood pressure	129.0 (23.0)
	White blood cells	6.8 (4.0)
	C-reactive protein	71.5 (97.9)
Oxygen delivery device use in ED	Room air	481 (50.6)
	Nasal cannula	338 (35.6)
	Hudson mask	22 (2.3)
	NRBM	89 (9.4)
	HFNC/CPAP/BiPAP	20 (2.1)
Oxygen device used during hospitalization	Nasal cannula	890 (93.7)
	Hudson mask	14 (1.5)
	NRBM	383 (40.3)
	HFNC/CPAP/BiPAP	182 (19.2)
	Oral intubation/tracheostomy	133 (14.0)

n: Number of participants; IQR: Interquartile range; RALE: Radiographic Assessment of Lung Edema; CHF: Congestive heart failure; COPD: Chronic obstructive pulmonary disease; ED: Emergency department; NRBM: Non-rebreather mask; HFNC: High flow nasal cannula; CPAP: Continuous positive airway pressure; BiPAP: Bi-level positive airway pressure.

BRIXIA and RALE scores

As shown in Table 2, the diagnostic performance of the BRIXIA and RALE scores varied according to the type of oxygen delivery device used both on ED arrival and during hospitalization.

Table 2 Performance of BRIXIA and Radiographic Assessment of Lung Edema scores for predicting oxygen delivery device and discharge outcomes in the emergency department and during hospitalization.

BRIXIA score	Cut-off point (95%CI)	Sensitivity	Specificity	AUC at cut-point (95%CI)
Oxygen delivery device in ED
Room air	7 (6-8)	0.43	0.39	0.41 (0.38-0.44)
Nasal cannula	8 (7-9)	0.51	0.60	0.56 (0.52-0.59)
Hudson mask	9 (6-12)	0.36	0.61	0.49 (0.38-0.59)
NRBM	9 (8-10)	0.66	0.64	0.65 (0.60-0.70)
HFNC/CPAP/BiPAP	12 (9-15)	0.65	0.84	0.74 (0.64-0.85)
Oxygen delivery device during hospitalization
Nasal cannula	6 (4-8)	0.60	0.48	0.54 (0.47-0.61)
Hudson mask	13 (6-20)	0.29	0.86	0.57 (0.42-0.72)
NRBM	8 (7-9)	0.58	0.66	0.62 (0.59-0.65)
HFNC/CPAP/BiPAP	8 (7-9)	0.63	0.61	0.62 (0.58-0.66)
Oral intubation/tracheostomy	8 (6-10)	0.65	0.60	0.63 (0.58-0.67)
RALE score
Oxygen delivery device in ED
Room air	12 (8-16)	0.49	0.57	0.42 (0.39-0.45)
Nasal cannula	10 (7-13)	0.58	0.57	0.54 (0.51-0.57)
Hudson mask	14 (8-20)	0.41	0.61	0.54 (0.43-0.64)
NRBM	16 (10-22)	0.54	0.72	0.62 (0.57-0.68)
HFNC/CPAP/BiPAP	17 (12-22)	0.65	0.77	0.74 (0.65-0.83)
Oxygen delivery device during hospitalization
Nasal cannula	12 (6-18)	0.29	0.85	0.52 (0.46-0.59)
Hudson mask	13 (0-29)	0.58	0.65	0.55 (0.42-0.69)
NRBM	13 (10-16)	0.64	0.61	0.6 (0.57-0.63)
HFNC/CPAP/BiPAP	16 (11-21)	0.66	0.59	0.59 (0.55-0.64)
Oral intubation/tracheostomy	16 (12-20)	0.52	0.36	0.62 (0.57-0.66)

AUC: Area under the curve; CI: Confidence interval; ED: Emergency department; NRBM: Non-rebreather mask; HFNC: High flow nasal cannula; CPAP: Continuous positive airway pressure; BiPAP: Bi-level positive airway pressure; RALE: Radiographic Assessment of Lung Edema.

Oxygen delivery device in ED on arrival: For patients on room air, the BRIXIA score had a cut-off value of 7 (95%CI: 6-8), with a sensitivity of 0.43, specificity of 0.39, and an AUC of 0.41 (95%CI: 0.38-0.44). The RALE score for this group had a cut-off of 12 (95%CI: 8-16), showing slightly better specificity (0.57) and an AUC of 0.42 (95%CI: 0.39-0.45). Among patients receiving oxygen via nasal cannula in the ED, the BRIXIA score cut-off was 8 (95%CI: 7-9), yielding improved sensitivity (0.51), specificity (0.60), and an AUC of 0.56 (95%CI: 0.52-0.59). In the same group, the RALE score cut-off was 10 (95%CI: 7-13), with a sensitivity of 0.58, a specificity of 0.57, and an AUC of 0.54 (95%CI: 0.51-0.57). For patients using a Hudson mask, the BRIXIA score cut-off was 9 (95%CI: 6-12), with a sensitivity of 0.36, specificity of 0.61, and an AUC of 0.49 (95%CI: 0.38-0.59). The RALE score cut-off in this group was 14 (95%CI: 8-20), with an AUC of 0.54 (95%CI: 0.43-0.64). In patients requiring an NRBM, the BRIXIA score cut-off was 9 (95%CI: 8-10), showing a sensitivity of 0.66, a specificity of 0.64, and an AUC of 0.65 (95%CI: 0.60-0.70). The RALE score cut-off was 16 (95%CI: 10-22), with a sensitivity of 0.54, a specificity of 0.72, and an AUC of 0.62 (95%CI: 0.57-0.68).

Oxygen delivery device during hospitalization: For hospitalized patients requiring a nasal cannula, the BRIXIA score had a cut-off of 6 (95%CI: 4-8), with a sensitivity of 0.60, specificity of 0.48, and an AUC of 0.54 (95%CI: 0.47-0.61). The RALE score in this group had a cut-off with a sensitivity of 0.29, a specificity of 0.85, and an AUC of 0.52 (95%CI: 0.46-0.59). For patients needing a Hudson mask, the BRIXIA score had a cut-off of 13 (95%CI: 6-20), with a sensitivity of 0.29, a specificity of 0.86, and an AUC of 0.57 (95%CI: 0.42-0.72). The RALE score had a cut-off of 13 (95%CI: 0-29), a sensitivity of 0.58, a specificity of 0.65, and an AUC of 0.55 (95%CI: 0.42-0.69). For patients requiring oral intubation/tracheostomy, the BRIXIA score had a cut-off of 8 (95%CI: 6-10), with a sensitivity of 0.65, a specificity of 0.60, and an AUC of 0.63 (95%CI: 0.58-0.67). The RALE score had a cut-off of 16 (95%CI: 12-20), a sensitivity of 0.52, a specificity of 0.36, and an AUC of 0.62 (95%CI: 0.57-0.66). Receiver operating characteristic curves for BRIXIA and RALE scores, in relation to oxygen device use and length of hospital stay, are shown in Supplementary Figures 2 and 3.

Adjusted logistic regression outcomes: BRIXIA and RALE scores were also analyzed using three logistic regression models: Unadjusted, adjusted for age (a-odds ratio), and adjusted for both age and the initial oxygen delivery device used in the ED (a-odds ratio) (Table 3).

Table 3 Logistic regression analysis of oxygen delivery requirements by BRIXIA and Radiographic Assessment of Lung Edema scores.

	Crude OR (95%CI)	P value	a-OR1 (95%CI)	P value	a-OR2 (95%CI)	P value
BRIXIA score
Oxygen delivery device in ED
Model 1: Room air	0.9 (0.87-0.92)	< 0.001	0.9 (0.87-0.93)	< 0.001
Model 2: Nasal cannula	1.05 (1.02-1.08)	0.001	1.05 (1.02-1.08)	0.002
Model 3: Hudson mask	0.92 (0.84-1.02)	0.108	0.93 (0.84-1.02)	0.134
Model 4: NRBM	1.15 (1.1-1.21)	< 0.001	1.15 (1.09-1.21)	< 0.001
Model 5: HFNC/CPAP/BiPAP	1.27 (1.14-1.42)	< 0.001	1.27 (1.14-1.41)	< 0.001
Oxygen delivery device during hospitalization
Model 6: Nasal cannula	1.01 (0.95-1.07)	0.784	1 (0.95-1.06)	0.868	1.04 (0.97-1.1)	0.249
Model 7: Hudson mask	1.02 (0.91-1.14)	0.761	1.01 (0.9-1.14)	0.830	0.99 (0.88-1.12)	0.898
Model 8: NRBM	1.13 (1.1-1.16)	< 0.001	1.12 (1.09-1.16)	< 0.001	1.09 (1.06-1.13)	< 0.001
Model 9: HFNC/CPAP/BiPAP	1.12 (1.08-1.16)	< 0.001	1.11 (1.07-1.15)	< 0.001	1.05 (1.01-1.09)	0.018
Model 10: Oral intubation/tracheostomy	1.11 (1.07-1.16)	< 0.001	1.1 (1.06-1.15)	< 0.001	1.04 (1-1.09)	0.049
RALE score
Oxygen delivery device in ED
Model 14: Room air	0.96 (0.95-0.97)	< 0.001	0.96 (0.95-0.97)	< 0.001
Model 15: Nasal cannula	1.01 (1-1.02)	0.040	1.01 (1-1.02)	0.065
Model 16: Hudson mask	0.98 (0.94-1.02)	0.417	0.99 (0.95-1.03)	0.485
Model 17: NRBM	1.05 (1.04-1.07)	< 0.001	1.05 (1.03-1.07)	< 0.001
Model 18: HFNC/CPAP/BiPAP	1.07 (1.04-1.11)	< 0.001	1.07 (1.04-1.11)	< 0.001
Oxygen delivery device during hospitalization
Model 19: Nasal cannula	1 (0.97-1.02)	0.842	1 (0.97-1.02)	0.760	1.01 (0.99-1.04)	0.358
Model 20: Hudson mask	1.02 (0.98-1.07)	0.316	1.02 (0.98-1.07)	0.356	1.01 (0.96-1.06)	0.733
Model 21: NRBM	1.05 (1.04-1.06)	< 0.001	1.05 (1.03-1.06)	< 0.001	1.04 (1.02-1.05)	< 0.001
Model 22: HFNC/CPAP/BiPAP	1.04 (1.03-1.06)	< 0.001	1.04 (1.02-1.05)	< 0.001	1.02 (1-1.03)	0.037
Model 23: Oral intubation/tracheostomy	1.04 (1.03-1.06)	< 0.001	1.04 (1.02-1.05)	< 0.001	1.02 (1-1.04)	0.042

¹Adjusted for age.

²Adjusted for age and initial oxygen delivery device in the emergency department.

OR: Odds ratio; CI: Confidence interval; ED: Emergency department; NRBM: Non-rebreather mask; HFNC: High flow nasal cannula; CPAP: Continuous positive airway pressure; BiPAP: Bi-level positive airway pressure; RALE: Radiographic Assessment of Lung Edema.

Reliability of radiographic scores: The reliability analysis demonstrated a distinct pattern between inter- and intra-rater agreement. Inter-rater reliability for both BRIXIA and RALE scores was moderate (κ = 0.57-0.58), indicating notable variability between clinicians. The complete inter-rater statistical results are presented in Table 4, while Figure 2 displays score distributions across raters. Conversely, intra-rater reliability was substantially higher, ranging from substantial (BRIXIA κ = 0.77) to almost perfect (RALE κ = 0.85), demonstrating strong self-consistency over time. These intra-rater results are detailed in Table 5, with corresponding Bland-Altman plots shown in Figure 3.

Figure 2 BRIXIA and Radiographic Assessment of Lung Edema scores. A: Inter-rater distribution of BRIXIA scores; B: Inter-rater distribution of Radiographic Assessment of Lung Edema scores. RALE: Radiographic Assessment of Lung Edema.

Figure 3 BRIXIA and Radiographic Assessment of Lung Edema scores. A: Intra-rater agreement for BRIXIA scores; B: Intra-rater agreement of Radiographic Assessment of Lung Edema scores. RALE: Radiographic Assessment of Lung Edema.

Table 4 Inter-rater agreement coefficients for BRIXIA and Radiographic Assessment of Lung Edema scores (n = 44).

Weight type	Measure	BRIXIA (95%CI)	RALE (95%CI)	Benchmarking interval	Extent of agreement
Ordinal weights	Percent agreement	0.95 (0.94-0.96)	0.95 (0.94-0.96)	(0.80-1.00)	Almost perfect
	Scott/Fleiss’ Kappa	0.57 (0.46-0.67)	0.58 (0.46-0.70)	(0.40-0.60)	Moderate
	Gwet’s AC	0.74 (0.69-0.80)	0.75 (0.70-0.80)	(0.60-0.80)	Substantial
Linear weights	Percent agreement	0.83 (0.81-0.85)	0.85 (0.84-0.87)	(0.80-1.00)	Almost perfect
	Scott/Fleiss’ Kappa	0.38 (0.03-0.47)	0.37 (0.27-0.46)	(0.20-0.40)	Fair
	Gwet’s AC	0.54 (0.48-0.59)	0.54 (0.50-0.59)	(0.40-0.60)	Moderate
Quadratic weights	Percent agreement	0.95 (0.94-0.96)	0.96 (0.96-0.97)	(0.80-1.00)	Almost perfect
	Scott/Fleiss’ Kappa	0.59 (0.48-0.70)	0.59 (0.46-0.71)	(0.40-0.60)	Moderate
	Gwet’s AC	0.76 (0.71-0.82)	0.77 (0.72-0.83)	(0.60-0.80)	Substantial

CI: Confidence interval; RALE: Radiographic Assessment of Lung Edema; AC: Agreement coefficient.

Table 5 Intra-rater agreement coefficients for BRIXIA and Radiographic Assessment of Lung Edema scores (n = 220).

Score	Measure	Agreement coefficient (95%CI)	Benchmarking interval	Extent of agreement
BRIXIA	Percent agreement	0.97 (0.96-0.98)	(0.80-1.00)	Almost perfect
	Scott/Fleiss’ Kappa	0.77 (0.71-0.83)	(0.60-0.80)	Substantial
	Gwet’s AC	0.85 (0.81-0.89)	(0.80-1.00)	Almost perfect
RALE	Percent agreement	0.98 (0.97-0.98)	(0.80-1.00)	Almost perfect
	Scott/Fleiss’ Kappa	0.85 (0.81-0.90)	(0.80-1.00)	Almost perfect
	Gwet’s AC	0.88 (0.85-0.91)	(0.80-1.00)	Almost perfect

CI: Confidence interval; RALE: Radiographic Assessment of Lung Edema; AC: Agreement coefficient.

DISCUSSION

This study investigated the utility of two radiographic severity scores (RALE and BRIXIA) in quantifying pulmonary infiltrates on radiographs taken within 24 hours of admission for predicting oxygen support requirements in patients with COVID-19. The findings suggest that these scores are moderately predictive of patients requiring high levels of oxygen support in the ED and, to a lesser extent, in the subsequent hospital admission. Higher scores on either scoring system reflected increasingly dense and widespread infiltrates, which predicted higher levels of oxygen requirements. However, lower levels of infiltrate density did not reliably predict lower levels of oxygen support, especially during admission, and radiographs would not help identify patients suitable for transfer to facilities with less oxygen support.

The predictive accuracy of both scores, as measured by the AUC, increased with the intensity of required respiratory support. The AUC was modest for lower levels of support (BRIXIA AUC: 0.56; RALE AUC: 0.54) but significantly better for predicting the need for HFNC, CPAP, or BiPAP, with an identical AUC of 0.74. This indicates these scores are particularly valuable for identifying patients who will require advanced respiratory interventions rather than distinguishing between mild disease states. Additionally, a significant finding is the BRIXIA score’s high specificity (0.84) at its optimal cut-off of 12 for predicting HFNC/CPAP/BiPAP, suggesting a score below 12 can help exclude the immediate need for high-level support. The predictive power of both systems decreased during hospitalization (e.g., BRIXIA AUC dropped to 0.62), likely because clinical deterioration in inpatients is influenced by factors beyond the initial lung injury, such as secondary infections and comorbidities. Logistic regression confirmed a significant correlation between higher radiographic scores and the need for intensive respiratory support. Each one-point increase in the BRIXIA and RALE scores was associated with 27% and 7% higher odds of requiring HFNC/CPAP/BiPAP in the ED, respectively. This correlation held for patients requiring NRBMs and mechanical ventilation, confirming these scores accurately reflect the severity of pulmonary disease at a granular level.

The findings on the predictive value of CXR scores are strongly supported by existing literature. A study by Toussie et al[22] confirmed that a CXR severity score at initial ED presentation was a powerful predictor of critical outcomes, with a score of ≥ 2 associated with 6.2 times higher odds of hospitalization and a score of ≥ 3 among admitted patients associated with 4.7 times higher odds of intubation. Furthermore, the prognostic value of the specific scoring system we employed is well-established. The original BRIXIA score validation study by Borghesi et al[16] demonstrated that it was a significant independent predictor of in-hospital mortality, with each point increase associated with a 27% increase in the odds of death. This is further reinforced by recent research from Shaima et al[23], which validated specific CXR score cut-offs, finding that a score of ≥ 3 was an independent predictor of severe/critical disease and a score of ≥ 5 was a powerful predictor of in-hospital mortality.

The high inter-rater per cent agreement (95%) achieved in this study cannot be disentangled from the standardized 3-hour training program undertaken by the clinicians. This finding demonstrates that with a structured, concise training intervention, frontline clinicians across specialities (emergency medicine, critical care, and radiology) can achieve a strong consensus, making these scores viable for rapid implementation in clinical practice. The exceptional intra-rater agreement (97%-98%) further confirms that once trained, clinicians can apply these systems with high self-consistency, making them reliable for both initial triage and serial monitoring.

This study demonstrates that CXR-based scoring provides a rapid, pragmatic method for predicting a specific, pressing outcome in the ED: Oxygen requirement. This approach stands in contrast to other powerful but more resource-intensive approaches. For instance, Yucal et al[24] introduced the Clopidogrel for High Atherothrombotic Risk and Ischemic Stabilization, Management, and Avoidance score, which integrates CT imaging and the biomarker surfactant protein D for mortality prediction, representing a comprehensive but less readily available prognostic model. Their study found a strong correlation between serum surfactant protein D levels and the quantitative volume of pulmonary infiltration on CT (r = 0.960), and the Clopidogrel for High Atherothrombotic Risk and Ischemic Stabilization, Management, and Avoidance score demonstrated excellent predictive power for in-hospital mortality (AUC: 0.977). Similarly, purely clinical scores like the Colonoscopy Progression Score[25] effectively predict survival, demonstrating the prognostic value of synthesizing clinical and laboratory data. In this context, the primary advantage of RALE and BRIXIA scoring is its ability to provide immediate, objective risk stratification using a universally available, standard-of-care test, bypassing the need for specialized laboratory assays or CT imaging. In this context, the primary advantage of RALE and BRIXIA scoring lies in its unique balance of speed, availability, and objective quantification. Unlike CT-based models or specialized biomarker assays, it leverages a universally available, standard-of-care test to provide rapid, low-cost risk stratification. While not directly comparable in predicting long-term outcomes like mortality, its strong performance in predicting immediate respiratory support needs makes it an invaluable tool for frontline clinical decision-making.

The study is limited by its single-center, retrospective design and exclusion of chronic oxygen-dependent patients. Lack of standardized radiograph positioning and incomplete documentation of symptom onset and clinical severity may have introduced measurement bias. Mortality data were unavailable, and only age and the initial ED oxygen device were considered as predictors, omitting other potentially relevant variables. Future multicenter studies with standardized imaging, comprehensive outcomes including mortality, and integration of scoring systems with clinical and biomarker data are recommended.

CONCLUSION

This study validates the clinical utility of both RALE and BRIXIA scoring systems as prognostic tools for COVID-19 patients in the ED. Demonstrating excellent inter-rater and intra-rater reliability, both scores effectively predicted the need for advanced respiratory support, with higher scores strongly correlating with increased oxygen support requirements. The BRIXIA score showed particular value in ruling out the need for high-level support due to its high specificity. However, the predictive power of both systems diminished during hospitalization. For emergency practice, the high reliability and predictive utility of either scoring system can provide valuable prognostic information, with the choice between them appropriately based on local preference and workflow considerations. Future research could focus on exploring the integration of these radiographic scores with clinical biomarkers and severity scales to enhance predictive accuracy across the care continuum.

ACKNOWLEDGEMENTS

The authors would like to express their deepest gratitude to all frontline workers in Qatar who dedicated their time, expertise, and unwavering commitment during the COVID-19 pandemic. Their courage, resilience, and selfless service played a vital role in protecting the community and sustaining essential services throughout this challenging period.