To systematically assess the robustness of meta-analyses based on randomised controlled trials (RCTs) in vascular surgery using the Fragility Index (FI).

Cross-sectional study.

Meta-analyses published in English from January 2019 to April 2025, identified from EMBASE, PubMed and Web of Science.

67 articles, with 291 meta-analyses involving RCTs evaluating vascular surgical interventions, covering venous, aortic, peripheral arterial, vascular access and other relevant fields.

FI, defined as the minimum number of event changes required to alter the statistical significance of meta-analysis results, and its association with sample size and total number of events, analysed using frequency distribution histograms and restricted cubic spline models.

The median FI was 7, with considerable variation across different fields. Aortic meta-analyses demonstrated higher robustness compared with venous and vascular access meta-analyses. FI showed a non-linear relationship with sample size and total number of events, indicating robustness improved only up to specific thresholds, beyond which robustness declined or plateaued.

Overall robustness of meta-analyses in vascular surgery was moderate, with notable variability among research areas. FI provides valuable insight into the stability of synthesised evidence, suggesting the need for improved methodological quality and advocating broader adoption of FI in meta-analytical research.

The p-value has changed from a versatile tool for scientific reasoning to a strict judge of medical information, with the usual 0.05 cutoff frequently deciding a study's significance and subsequent clinical use. Through an examination of five critical care interventions that demonstrated meaningful treatment effects yet narrowly missed conventional statistical significance, this paper illustrates how rigid adherence to p-value thresholds may obscure therapeutically beneficial findings. By providing a clear, step-by-step illustration of a basic Bayesian calculation, we demonstrate that clinical importance can remain undetected when relying solely on p-values. These observations challenge current statistical paradigms and advocate for hybrid approaches-including both frequentist and Bayesian methodologies-to provide a more comprehensive understanding of clinical data, ultimately leading to better-informed medical decisions.

To determine the robustness of randomized controlled trials (RCTs) supporting the current rhinosinusitis guideline; International Consensus Statement on Allergy and Rhinology: rhinosinusitis (ICAR-RS).

RCTs referenced by ICAR-RS with primary dichotomous outcomes were analyzed. The Fragility Index (FI) was calculated for trials with statistically significant findings. Trial characteristics, the FI, and FI minus number lost to follow-up (LTF) were assessed for associations.

A total of 317 RCTs were identified, with 38 trials possessing a primary dichotomous outcome. Thirty-one percent evaluated surgical interventions and 24 % were industry-sponsored. The mean sample size was 116 with 9 patients, on average, LTF. Sixty-three percent were eligible for FI calculation and had a median FI of 2.5 (IQR 1, 4.25). Sixty-seven percent of trials had an FI ≤ 3, indicating low robustness. No difference in FI was observed between trials with and without industry support (p = 0.577). The FI was less than or equal to the number of patients LTF in 33 % of trials (n = 8). Higher FI was strongly correlated with higher sample size, total number of events, p-value, and grade of recommendation (p < 0.001). After adjusting for covariates, higher sample size and total number of events were associated with higher FI.

The RCTs used to support the ICAR-RS have an overall low robustness and future rhinosinusitis trials should report FI measures to provide improved context of their results.

Esophageal cancer remains a significant global health challenge. Several treatment modalities were explored in randomized controlled trials (RCTs) in recent decades. This study evaluates the robustness of RCTs focusing on esophageal cancer treatment using the fragility index (FI) and reverse fragility index (RFI).

A systematic review of RCTs studying different treatment modalities for esophageal cancer from 2000 to 2023 was conducted. The FI and RFI were utilized to gauge the robustness of statistically significant and non-significant outcomes, respectively. The FI represents the minimal number of patient outcomes that would need to alter to overturn a trial's statistical significance, while RFI indicates the minimal changes required to achieve significance in non-significant results.

Out of 4028 studies retrieved, 21 RCTs were included for final analysis. The studies spanned 2001 to 2023 with a mean followup of 66 months (range, 29-108 months) and median number of patients of 194 (range, 45-802). The most common treatment modalities examined in these studies were neoadjuvant chemoradiotherapy (n = 7, 33.3%), neoadjuvant chemotherapy (n = 4, 19.0%), and neoadjuvant immunotherapy (n = 2, 9.5%). Only 5 studies (23.8%) had a statistically significant primary outcome result with a median FI of 6 (IQR, 2.5-8.5). Non-significant primary outcomes were seen in 16 studies (76.2%) with a median RFI of 4 (IQR 1-11) and lost to followup of 0 (IQR 0-4). In the study with the highest FI (10), the FI was lower than the number of patients lost to followup (13).

Our findings demonstrate that most RCTs on esophageal cancer treatments did not report significant primary outcomes. The few studies that reported significant results had a low fragility index, suggesting a vulnerability in their findings.

The fragility index (FI) of a meta-analysis evaluates the extent that the statistical significance can be changed by modifying the event status of individuals from included trials. Understanding the FI improves the interpretation of the results of meta-analyses and can help to inform changes to clinical practice. This review determined the fragility of ophthalmology-related meta-analyses.

Meta-analyses of randomized controlled trials with binary outcomes published in a journal classified as 'Ophthalmology' according to the Journal Citation Report or an Ophthalmology-related Cochrane Review were included. An iterative process determined the FI of each meta-analysis. Multivariable linear regression modeling evaluated the relationship between the FI and potential predictive factors in statistically significant and non-significant meta-analyses.

175 meta-analyses were included. The median FI was 6 (Q1-Q3: 3-12). This meant that moving 6 outcomes from one group to another would reverse the study's findings. The FI was 1 for 18 (10.2%) of the included meta-analyses and was ≤5 for 75 (42.4%) of the included meta-analyses. The number of events (p < 0.001) and the p-value (p < 0.001) were the best predictors of the FI in both significant and non-significant meta-analyses.

The statistical significance of meta-analyses in ophthalmology often hinges on the outcome of a few patients. The number of events and the p-value are the most important factors in determining the fragility of the evidence. The FI is an easily interpretable measure that can supplement the reader's understanding of the strength of the evidence being presented.

CRD42022377589.

Acute Respiratory Distress Syndrome (ARDS) necessitates personalized treatment strategies due to its heterogeneity, aiming to mitigate Ventilator-Induced Lung Injury (VILI). Advanced monitoring techniques, including imaging, driving pressure, transpulmonary pressure, and mechanical power, present potential avenues for tailored interventions.

To review some of the most important techniques for achieving greater personalization of mechanical ventilation in ARDS patients as evaluated in randomized clinical trials, by analyzing their effect on three clinically relevant aspects: mortality, ventilator-free days, and gas exchange.

Following PRISMA guidelines, we conducted a systematic review and meta-analysis of Randomized Clinical Trials (RCTs) involving adult ARDS patients undergoing personalized ventilation adjustments. Outcomes were mortality (primary end-point), ventilator-free days, and oxygenation improvement.

Among 493 identified studies, 13 RCTs (n = 1255) met inclusion criteria. No personalized ventilation strategy demonstrated superior outcomes compared to traditional protocols. Meta-analysis revealed no significant reduction in mortality with image-guided (RR 0.88, 95 % CI 0.70-1.11), driving pressure-guided (RR 0.61, 95 % CI 0.29-1.30), or transpulmonary pressure-guided (RR 0.85, 95 % CI 0.58-1.24) strategies. Ventilator-free days and oxygenation outcomes showed no significant differences.

Our study does not support the superiority of personalized ventilation techniques over traditional protocols in ARDS patients. Further research is needed to standardize ventilation strategies and determine their impact on mechanical ventilation outcomes.

With the vast increase in spinal surgery research and accessibility, critical evaluation of studies is paramount. Historically, P values and confidence intervals have been the gold standard, but more recently, the inclusion of the Fragility Index has brought a more holistic approach. The Fragility Index aims to communicate the robustness of a trial and how tenuous statistical significance may be. It can be used in conjunction with more traditional methods for evaluating research.

To familiarize clinicians with the emerging concepts in critical care research of Bayesian thinking and personalized medicine through phenotyping and explain their clinical relevance by highlighting how they address the issues of frequent negative trials and heterogeneity of treatment effect.

The past decades have seen many negative (effect-neutral) critical care trials of promising interventions, culminating in calls to improve the field's research through adopting Bayesian thinking and increasing personalization of critical care medicine through phenotyping. Bayesian analyses add interpretive power for clinicians as they summarize treatment effects based on probabilities of benefit or harm, contrasting with conventional frequentist statistics that either affirm or reject a null hypothesis. Critical care trials are beginning to include prospective Bayesian analyses, and many trials have undergone reanalysis with Bayesian methods. Phenotyping seeks to identify treatable traits to target interventions to patients expected to derive benefit. Phenotyping and subphenotyping have gained prominence in the most syndromic and heterogenous critical care disease states, acute respiratory distress syndrome and sepsis. Grouping of patients has been informative across a spectrum of clinically observable physiological parameters, biomarkers, and genomic data. Bayesian thinking and phenotyping are emerging as elements of adaptive clinical trials and predictive enrichment, paving the way for a new era of high-quality evidence. These concepts share a common goal, sifting through the noise of heterogeneity in critical care to increase the value of existing and future research.

The future of critical care medicine will inevitably involve modification of statistical methods through Bayesian analyses and targeted therapeutics via phenotyping. Clinicians must be familiar with these systems that support recommendations to improve decision-making in the gray areas of critical care practice.

Overactive bladder (OAB) syndrome significantly impairs quality of life, often necessitating pharmacological interventions with associated risks. The fragility of OAB trial outcomes, as measured by the fragility index (FI: smallest number of event changes to reverse statistical significance) and quotient (FQ: FI divided by total sample size expressed as a percentage), is critical yet unstudied.

We conducted a systematic search for randomized controlled trials on OAB medications published between January 2000 and August 2023. Inclusion criteria were trials with two parallel arms reporting binary outcomes related to OAB medications. We extracted trial details, outcomes, and statistical tests employed. We calculated FI and FQ, analyzing associations with trial characteristics through linear regression.

We included 57 trials with a median sample size of 211 participants and a 12% median lost to follow-up. Most studies investigated anticholinergics (37/57, 65%). The median FI/FQ was 5/3.5%. Larger trials were less fragile (median FI 8; FQ 1.0%) compared to medium (FI: 4; FQ 2.5%) and small trials (FI: 4; FQ 8.3%). Double-blinded studies exhibited higher FQs (median 2.9%) than unblinded trials (6.7%). Primary and secondary outcomes had higher FIs (median 5 and 6, respectively) than adverse events (FI: 4). Each increase in 10 participants was associated with a +0.19 increase in FI (p < 0.001).

A change in outcome for a median of five participants, or 3.5% of the total sample size, could reverse the direction of statistical significance in OAB trials. Studies with larger sample sizes and efficacy outcomes from blinded trials were less fragile.

The fragility index (FI), which is the minimum number of changes in status from "event" to "non-event" resulting in a loss of statistical significance, serves as a significant supplementary indicator for clinical physicians in interpreting clinical trial results and aids in understanding the outcomes of randomized controlled trials (RCTs). In this systematic literature survey, we evaluated the FI for RCTs evaluating Chinese herbal medicine (CHM) for irritable bowel syndrome (IBS), and explored potential associations between study characteristics and the robustness of RCTs.

A comprehensive search was conducted in four databases in Chinese and four databases in English from their inception to January 1, 2023. RCTs encompassed 1:1 ratio into two parallel arms and reported at least one binary outcome that demonstrated statistical significance were included. FI was calculated by the iterative reduction of a target outcome event in the treatment group and concomitant subtraction of a non-target event from that group, until positive significance (defined as P < 0.05 by Fisher's exact test) is lost. The lower the FI (minimum 1) of a trial outcome, the more fragile the positive result of the outcome was. Linear regression models were adopted to explore influence factors of the value of FI.

A total of 30 trials from 2 4118 potentially relevant citations were finally included. The median FI of total trials included was 1.5 (interquartile range [IQR], 1-5), and half of the trials (n = 15) had a FI equal to 1. In 12 trials (40%), the total number of participants lost to follow-up surpassed the respective FI. The study also identified that increased FI was significantly associated with no TCM syndrome differentiation for inclusion criteria of the patients, larger total sample size, low risk of bias, and larger numbers of events.

The majority of CHM IBS RCTs with positive results were found to be fragile. Ensuring adequate sample size, scientifically rigorous study design, proper control of confounding factors, and a quality control calibration for consistency of TCM diagnostic results among clinicians should be addressed to increase the robustness of the RCTs. We recommend reporting the FI as one of the components of sensitivity analysis in future RCTs to facilitate the assessment of the fragility of trials.

Evidence-based care relies on robust research. The fragility index (FI) is used to assess the robustness of statistically significant findings in randomized controlled trials (RCTs). While the traditional FI is limited to dichotomous outcomes, a novel tool, the continuous fragility index (CFI), allows for the assessment of the robustness of continuous outcomes.

To calculate the CFI of statistically significant continuous outcomes in RCTs evaluating interventions for managing anterior shoulder instability (ASI).

Meta-analysis; Level of evidence, 2.

A search was conducted across the MEDLINE, Embase, and CENTRAL databases for RCTs assessing management strategies for ASI from inception to October 6, 2022. Studies that reported a statistically significant difference between study groups in ≥1 continuous outcome were included. The CFI was calculated and applied to all available RCTs reporting interventions for ASI. Multivariable linear regression was performed between the CFI and various study characteristics as predictors.

There were 27 RCTs, with a total of 1846 shoulders, included. The median sample size was 61 shoulders (IQR, 43). The median CFI across 27 RCTs was 8.2 (IQR, 17.2; 95% CI, 3.6-15.4). The median CFI was 7.9 (IQR, 21; 95% CI, 1-22) for 11 studies comparing surgical methods, 22.6 (IQR, 16; 95% CI, 8.2-30.4) for 6 studies comparing nonsurgical reduction interventions, 2.8 for 3 studies comparing immobilization methods, and 2.4 for 3 studies comparing surgical versus nonsurgical interventions. Significantly, 22 of 57 included outcomes (38.6%) from studies with completed follow-up data had a loss to follow-up exceeding their CFI. Multivariable regression demonstrated that there was a statistically significant positive correlation between a trial's sample size and the CFI of its outcomes (r = 0.23 [95% CI, 0.13-0.33]; P < .001).

More than a third of continuous outcomes in ASI trials had a CFI less than the reported loss to follow-up. This carries the significant risk of reversing trial findings and should be considered when evaluating available RCT data. We recommend including the FI, CFI, and loss to follow-up in the abstracts of future RCTs.

The fragility index (FI) was conceived as an adjunct to the P value, signifying the strength of statistically significant results. The index states the minimal number of patients whose outcome must be changed from "event" to "nonevent" for the results to be statistically nonsignificant. The FI was applied in various medical specialties to assess the robustness of results presented in studies. We aim to assess the robustness of statistically significant results in studies on plastic surgery of the breast and determine factors correlated with studies deemed fragile.

A systematic literature review of PubMed databases using designated keywords was performed. Background characteristics were extracted from the studies, alongside the significance of outcomes. FI and fragility quotient were calculated for each analyzed outcome and correlated with various baseline characteristics.

FI and fragility quotient were both significantly correlated only with the P value of the analyzed outcomes. However, grouping studies based on the P value into three categories did not demonstrate a difference in FI. Comparisons of fragile and robust studies did not demonstrate a statistically significant change in terms of baseline variables, except for the mean P value of the outcome.

Statistically significant results of randomized controlled trials in plastic surgery of the breast suffer from extensive fragility, and researchers should critically implement their conclusions in their practice.

Fragility analysis supplements the p-value and risk of bias assessment in the interpretation of results of randomised controlled trials. In this systematic review we determine the fragility index (FI) and fragility quotient (FQ) of randomised trials in aneurysmal subarachnoid haemorrhage.

This is a systematic review registered with PROSPERO (ID: CRD42020173604). Randomised controlled trials in adults with aneurysmal subarachnoid haemorrhage were analysed if they reported a statistically significant primary outcome of mortality, function (e.g. modified Rankin Scale), vasospasm or delayed neurological deterioration.

We identified 4825 records with 18 randomised trials selected for analysis. The median fragility index was 2.5 (inter-quartile range 0.25-5) and the median fragility quotient was 0.015 (IQR 0.02-0.039). Five of 20 trial outcomes (25%) had a fragility index of 0. In seven trials (39.0%), the number of participants lost to follow-up was greater than or equal to the fragility index. Only 16.7% of trials are at low risk of bias.

Randomised controlled trial evidence supporting management of aneurysmal subarachnoid haemorrhage is weaker than indicated by conventional analysis using p-values alone. Increased use of fragility analysis by clinicians and researchers could improve the translation of evidence to practice.

Biologics and small molecules have been increasingly applied in Crohn's disease (CD) and ulcerative colitis (UC). But the robustness of their trials has not been evaluated.

We initially collected all the approved biologics or small molecules for CD or UC up to December 1, 2022. Databases were then queried by keywords in chemical name and CD or UC. Randomized controlled trials (RCTs) in the two-arm, 1:1 design were included. Fragility index (FI) and fragility quotient (FQ) were subsequently calculated.

We included twenty-eight RCTs, including nine pivotal trials listed in approval labels, nineteen non-pivotal trials not included in the labels. The median sample size was 99 [IQR, 60-262] and the median number of loss-of-follow-up (LFU) was 14 [IQR, 8-43]. Pivotal trials in the labels had the median FI of 8 [IQR, 4-14, n = 6] that was marginally higher than non-pivotal trials (3 [IQR, 2-4], p = 0.08). The median FQ was 0.0330 [IQR, 0.1220-0.0466] and 0.0310 [IQR, 0.0129-0.0540] for pivotal and non-pivotal trials, respectively (p = 1.0). The sample size and FI were significantly correlated (Spearman correlation coefficient [r] = 0.56, 95 %CI 0.21-0.78, p = 0.003). The number of total events was also significantly correlated with FI (r = 0.53, 95 %CI 0.17-0.77, p = 0.006). Study p-values were significantly associated with FI (p = 0.01): trials with p-values < 0.001 had the highest median FI of 10 [IQR, 6-17]. No factor was found strongly correlated with FQ.

Results from trials assessing administration-approved biologics or small molecules for treating CD or UC were vulnerable to small changes by measuring FI or FQ. Pivotal studies contributing to regulatory approvals exhibited a relatively higher degree of resilience compared to non-pivotal trials.

It is unclear how often survival benefits observed in single-center randomized controlled trials (sRCTs) involving critically ill patients are confirmed by subsequent multicenter randomized controlled trials (mRCTs). We aimed to perform a systemic literature review of sRCTs with a statistically significant mortality reduction and to evaluate whether subsequent mRCTs confirmed such reduction.

We searched PubMed for sRCTs published in the New England Journal of Medicine, JAMA, or Lancet, from inception until December 31, 2016. We selected studies reporting a statistically significant mortality decrease using any intervention (drug, technique, or strategy) in adult critically ill patients. We then searched for subsequent mRCTs addressing the same research question tested by the sRCT. We compared the concordance of results between sRCTs and mRCTs when any mRCT was available. We registered this systematic review in the PROSPERO International Prospective Register of Systematic Reviews (CRD42023455362).

We identified 19 sRCTs reporting a significant mortality reduction in adult critically ill patients. For 16 sRCTs, we identified at least one subsequent mRCT (24 trials in total), while the interventions from three sRCTs have not yet been addressed in a subsequent mRCT. Only one out of 16 sRCTs (6%) was followed by a mRCT replicating a significant mortality reduction; 14 (88%) were followed by mRCTs with no mortality difference. The positive finding of one sRCT (6%) on intensive glycemic control was contradicted by a subsequent mRCT showing a significant mortality increase. Of the 14 sRCTs referenced at least once in international guidelines, six (43%) have since been either removed or suggested against in the most recent versions of relevant guidelines.

Mortality reduction shown by sRCTs is typically not replicated by mRCTs. The findings of sRCTs should be considered hypothesis-generating and should not contribute to guidelines.

The purpose of this study was to determine the stability of statistical findings among sham surgery randomized controlled trials (RCTs) in orthopaedic surgery using fragility analysis.

PubMed systematic review was conducted to include studies reporting dichotomous outcomes pertaining to sham surgery. The final review included eight RCTs involving only partial meniscectomies and vertebroplasties from 2009 to 2020. With a fixed sample size with dichotomous outcome measures (events versus non-events), the Total Fragility Index (TFI), which is composed of the fragility index (FI) and reverse fragility index (RFI), was calculated by altering the ratio of events to non-events in an iterative fashion until results were reversed from significant to nonsignificant findings (FI) or vice versa (RFI). The TFI, FI, and RFI were divided by their sample sizes to obtain the respective total fragility quotient, fragility quotient (FQ), and reverse fragility quotient. Median fragility indices and quotients were reported for all studies.

The eight RCTs included 50 dichotomous outcomes involving either partial meniscectomies or vertebroplasties, with a median TFI and total fragility quotient of 5 [interquartile range (IQR) 4 to 6] and 0.035 (IQR 0.028 to 0.048), respectively, indicating that a median of five total patients or 3.5 per 100 patients would need to experience a different outcome to reverse significant or insignificant findings for each of the eight trials. Among the 8 statistically significant ( P < 0.05) outcome events (16%), the respective FI and FQ were 2 (IQR 1 to 5) and 0.018 (IQR 0.010 to 0.044). Among the 42 statistically insignificant outcome events (84%), the respective RFI and reverse fragility quotient were 5 (IQR 4 to 6) and 0.04 (IQR 0.034 to 0.048). The median number of patients lost to follow-up was 1.5 (IQR 0.5 to 2).

The unstable findings in partial meniscectomy and vertebroplasty sham surgical RCTs undermine their study conclusions and recommendations. We recommend using fragility analysis in future sham surgical RCTs to contextualize statistical findings.

Level IV; Systematic Review.

Determine the statistical stability of RCTs examining primary and secondary prevention of ankle sprains.

Databases were searched to August 2023. We included parallel design RCTs, using conservative interventions for preventing ankle sprain, reporting dichotomous injury event outcomes. Statistical stability was quantified using Fragility Index (FI) and Fragility Quotient (FQ). Subgroup analyses were undertaken to test if FI varied based on by study objective, original approach to analysis (frequency vs time to event), follow-up duration, and pre-registration.

3559 studies were screened with 45 RCTs included. The median number of events required to change the statistical significance (FI) was 4 (IQR 1-6). FI was similar regardless of study objective, original analysis, follow-up duration, and pre-registration status. Median (IQR) FQ was 0.015 (0.005-0.046), therefore reversing events <2 patients/100 would alter significance. In 80% of studies the number of patients lost to follow-up was greater than the FI.

RCTs informing primary and secondary prevention of ankle sprain are fragile. Only a small percentage of outcome event reversals would reverse study significance, and this is often exceeded by the number of drop outs. Robust reporting of dichotomous outcomes requires the use P values and key metrics such as FI or FQ.

Our purpose was to conduct a bibliometric study investigating the prevalence of underpowered randomized controlled trials (RCTs) in trauma surgery.

A medical librarian conducted a search of RCTs in trauma published from 2000 to 2021. Data extracted included study type, sample size calculation, and power analyses. Post hoc calculations were performed using a power of 80% and an alpha level of 0.05. A CONSORT checklist was then tabulated from each study as well as a fragility index for studies with statistical significance.

In total 187 RCTs from multiple continents and 60 journals were examined. A total of 133 (71%) were found to have "positive" findings consistent with their hypothesis. When evaluating their methods, 51.3% of articles did not report how they calculated their intended sample size. Of those that did, 25 (27%) did not meet their target enrollment. When examining post hoc power, 46%, 57%, and 65% were adequately powered to detect small, medium, and large effect sizes, respectively. Only 11% of RCTs had complete adherence with CONSORT reporting guidelines and the average CONSORT score was 19 out of 25. For positive superiority trials with binary outcomes, the fragility index median (interquartile range) was 2 (2 to 8).

A concerningly large proportion of recently published RCTs in trauma surgery do not report a priori sample size calculations, do not meet enrollment targets, and are not adequately powered to detect even large effect sizes. There exists opportunity for improvement of trauma surgery study design, conduct, and reporting.

The P value is a widely used measure of statistical importance but has many drawbacks and limitations, one being that it does not reflect the robustness of the results of a clinical trial. The Fragility Index (FI) was developed as a measure of how many outcome events would need to change to nonevents to render a significant P value nonsignificant (P ≥ 0.05). The FI of trials from other medical specialties is typically < 5. We aimed to determine the FI of pediatric anesthesiology randomized controlled trials (RCT) and to test for association with various characteristics of the included trials.

We conducted a comprehensive systematic search of high-impact anesthesia, surgical, and medical journals from the last 25 years for trials comparing an intervention between two groups with a statistically significant P value (< 0.05) for a dichotomous outcome. We also compared FI values for variables that reflect the quality and importance of a trial.

The median [interquartile range] FI was 3 [1-7] and correlated positively with the number of participants (rS = 0.41; P < 0.001) and events (rS = 0.42; P < 0.001), and negatively with the P value (rPB = -0.36; P < 0.001). Other measures of trial quality and impact or importance were not strongly associated with the FI.

The FI of published trials in pediatric anesthesiology is similarly low as in other medical specialties. Larger trials with more events and P values ≤ 0.01 were associated with a higher FI.

The aim was to systematically evaluate risks and benefits of proton pump inhibitor (PPI) use for stress ulcer prophylaxis in the critically ill patient.

Whether PPIs increase mortality in the critically ill patient remains controversial.

Systematic review and meta-analysis of randomized controlled trials (RCTs) and cohort studies with trial sequential analysis, Bayesian sensitivity analysis, and fragility index analysis.

A total of 31 studies in 78,009 critically ill adults receiving PPIs versus any comparator were included. PPI use was associated with an increased mortality risk in all studies [19.6% PPI vs. 17.5% comparator; RR: 1.10; 95% confidence interval (CI): 1.02-1.20; P =0.01], in the subgroup of RCTs (19.4% vs. 18.7%; RR: 1.05; 95% CI: 1.0-1.09, P =0.04), but not cohort studies (19.9% vs. 16.7%; RR: 1.12; 95% CI: 0.98-1.28, P =0.09). Results were maintained with a Bayesian sensitivity analysis (RR: 1.13; 95% credible interval: 1.035-1.227) and a fragility index analysis, but not sequential analysis ( P =0.16). RCTs with a higher baseline severity of illness revealed the greatest mortality risk with PPI use (32.1% PPI vs. 29.4% comparator; RR: 1.09; 95% CI: 1.04-1.14; P <0.001). PPI use reduced clinically important bleeding in RCTs (1.4% PPI vs. 2.1% comparator; RR: 0.67; 95% CI: 0.5-0.9; P =0.009) but increased bleeding in cohort studies (2.7% PPI vs. 1.2% comparator; RR: 2.05; 95% CI: 1.2-3.52; P =0.009). PPI use was not associated with a lower incidence of clinically important bleeding when compared with histamine-2 receptor antagonists (1.3% vs. 1.9%; RR: 0.59; 95% CI: 0.28-1.25, P =0.09).

This meta-analysis demonstrated an association between PPI use and an increased risk of mortality.

The fragility index (FI), i.e., theminimum number of best survivors reassigned to the control group required to revert the statistically significant result of a clinical trial to non-significant, is a metric to evaluate the robustness of randomized controlled trials (RCTs). We aimed to assess the FI in the field of HCC.

This is a retrospective analysis of phase 2 and 3 RCTs for the treatment of HCC published between 2002 and 2022. We included two-arm studies with 1:1 randomization and significant positive results for a primary time-to-event endpoint for the FI calculation, which involves the iterative addition of a best survivor from the experimental group to the control group, until positive significance (p <0,05, Log-rank test) is lost.

We identified 51 phase 2 and 3 positive RCTs, of which 29 (57%) were eligible for fragility index calculation. After reconstruction of the Kaplan-Meier curves, 25/29 studies remained significant, among which the analysis was performed. The median (interquartile range (IQR)) FI was 5 (2-10) and Fragility Quotient (FQ) was 3% (1%-6%). Ten trials (40%) had a FI of 2 or less. FI was positively correlated to the blind assessment of the primary endpoint (median FI 9 with blind assessment versus 2 without, p = 0.01), the number of reported events in the control arm (RS = 0.45, p = 0.02) and to impact factor (RS = 0.58, p = 0.003).

Several phases 2 and 3 RCTs in HCC have a low fragility index, underlying the limited robustness on the conclusion of their superiority over control treatments. The fragility index might provide an additional tool to assess the robustness of clinical trial data in HCC.

The fragility index is a method to assess robustness of a clinical trial and is defined the minimum number of best survivors reassigned to the control group required to revert the statistically significant result of a clinical trial to non-significant. Among 25 randomised controlled trials in HCC, the median fragility index was 5, and 10 trials among 25 (40%) had a fragility index of 2 or less, indicating an important fragility.

The fragility index (FI) measures the robustness of randomized controlled trials (RCTs). It complements the P value by taking into account the number of outcome events. In this study, the authors measured the FI for major interventional radiology RCTs.

Interventional radiology RCTs published between January 2010 and December 2022 relating to trans-jugular intrahepatic portosystemic shunt, trans-arterial chemoembolization, needle biopsy, angiography, angioplasty, thrombolysis, and nephrostomy tube insertion were analyzed to measure the FI and robustness of the studies.

A total of 34 RCTs were included. The median FI of those studies was 4.5 (range 1-68). Seven trials (20.6%) had a number of patients lost to follow-up that was higher than their FI, and 15 (44.1%) had a FI of 1-3.

The median FI, and hence the reproducibility of interventional radiology RCTs, is low compared to other medical fields, with some having a FI of 1, which should be interrupted cautiously.

Utility of robotic over laparoscopic approach has been an area of debate across all surgical specialties over the past decade. The fragility index (FI) is a metric that evaluates the frailty of randomized controlled trials (RCTs) findings by altering the status of patients from an event to non-event until significance is lost. This study aims to evaluate the robustness of RCTs comparing laparoscopic and robotic abdominopelvic surgeries through the FI.

A search was conducted in MEDLINE and EMBASE for RCTs with dichotomous outcomes comparing laparoscopic and robot-assisted surgery in general surgery, gynecology, and urology. The FI and reverse fragility Index (RFI) metrics were used to assess the strength of findings reported by RCTs, and bivariate correlation was conducted to analyze relationships between FI and trial characteristics.

A total of 21 RCTs were included, with a median sample size of 89 participants (Interquartile range [IQR] 62-126). The median FI was 2 (IQR 0-15) and median RFI 5.5 (IQR 4-8.5). The median FI was 3 (IQR 1-15) for general surgery (n = 7), 2 (0.5-3.5) for gynecology (n = 4), and 0 (IQR 0-8.5) for urology RCTs (n = 4). Correlation was found between increasing FI and decreasing p-value, but not sample size, number of outcome events, journal impact factor, loss to follow-up, or risk of bias.

RCTs comparing laparoscopic and robotic abdominal surgery did not prove to be very robust. While possible advantages of robotic surgery may be emphasized, it remains novel and requires further concrete RCT data.

The fragility index (FI) was proposed as a simplified way to communicate robustness of statistically significant results and their susceptibility to a change of a handful number of events. While this index is intuitive, it is not anchored by a cut-off or a guide for interpretation. We identified cardiovascular trials published in six high impact journals from 2007 to 2021 (500 or more participants and a dichotomous statistically significant primary outcome). We estimated area under curve (AUC) to determine FI value that best predicts whether the treatment effect was precise, defined as adequately powered for a plausible relative risk reduction (RRR) of 25% or 30% or having a CI that is sufficiently narrow to exclude a risk reduction that is too small (close to the null, <0.05). The median FI of 201 included cardiovascular trials was 13 (range 1-172). FI exceeded the number of patients lost to follow-up in 46/201 (22.89%) trials. FI values of 19 and 22 predicted that trials would be precise (powered for RRR of 30% and 25%; respectively, combined with CI that excluded risk reduction <0.05). AUC for meeting these precision criteria was 0.90 (0.86-0.94). In conclusion, FI values that range 19-22 may meet various definitions of precision and can be used as a rule of thumb to suggest that a treatment effect is likely precise and less susceptible to random error. The number of patients lost to follow-up should be presented alongside FI to better illustrate fragility.

Randomized controlled trials (RCTs) are regarded as high-level evidence, but the strength of their P values can be difficult to ascertain. The Fragility Index (FI) is a novel metric that evaluates the frailty of trial findings. It is defined as the minimum number of patients required to change from a non-event to event for the findings to lose statistical significance. This study aims to characterize the robustness of bariatric surgery RCTs by examining their FIs.

A search was conducted in MEDLINE, EMBASE, and CENTRAL from January 2000 to February 2022 for RCTs comparing two bariatric surgeries with statistically significant dichotomous outcomes. Bivariate correlation was conducted to identify associations between FI and trial characteristics.

A total of 35 RCTs were included with a median sample size of 80 patients (Interquartile range [IQR] 58-109). The median FI was 2 (IQR 0-5), indicating that altering the status of two patients in one treatment arm would overturn the statistical significance of results. Subgroup analyses of RCTs evaluating diabetes-related outcomes produced a FI of 4 (IQR 2-6.5), while RCTs comparing Roux-en-Y gastric bypass and sleeve gastrectomy had an FI of 2 (IQR 0.5-5). Increasing FI was found to be correlated with decreasing P value, increasing sample size, increasing number of events, and increasing journal impact factor.

Bariatric surgery RCTs are fragile, with only a few patients required to change from non-events to events to reverse the statistical significance of most trials. Future research should examine the use of FI in trial design.

While randomized controlled trials (RCTs) are regarded as one of the highest forms of clinical research, the robustness of their P values can be difficult to ascertain. Defined as the minimum number of patients in a study arm that would need to be changed from a non-event to event for the findings to lose significance, the Fragility Index is a method for evaluating results from these trials. This study aims to calculate the Fragility Index for trials evaluating perioperative esophagectomy-related interventions to determine the strength of RCTs in this field. MEDLINE and EMBASE were searched for RCTs related to esophagectomy that reported a significant dichotomous outcome. Two reviewers independently screened articles and performed the data extractions with risk of bias assessment. The Fragility Index was calculated using a two-tailed Fisher's exact test. Bivariate correlation was conducted to evaluate associations between the Fragility Index and study characteristics. 41 RCTs were included, and the median sample size was 80 patients [Interquartile range (IQR) 60-161]. Of the included outcomes, 29 (71%) were primary, and 12 (29%) were secondary. The median Fragility Index was 1 (IQR 1-3), meaning that by changing one patient from a non-event to event, the results would become non-significant. Fragility Index was correlated with P value, number of events, and journal impact factor. The RCTs related to esophagectomy did not prove to be robust, as the significance of their results could be changed by altering the outcome status of a handful of patients in one study arm.

Open fractures represent a major source of morbidity. Surgical site infections (SSIs) after open fractures are associated with a high rate of reoperations and hospitalizations, which are associated with a lower health-related quality of life. Early antibiotic delivery, typically chosen through an assessment of the size and contamination of the wound, has been shown to be an effective technique to reduce the risk of SSI in open fractures. The Gustilo-Anderson classification (GAC) was devised as a grading system of open fractures after a complete operative debridement of the wound had been undertaken but is commonly used preoperatively to help with the choice of initial antibiotics. Incorrect preoperative GAC, leading to less aggressive initial management, may influence the risk of SSI after open fracture. The objectives of this study were to determine (1) how often the GAC changed from the initial to definitive grading, (2) the injury and patient characteristics associated with increases and decreases of the GAC, and (3) whether a change in GAC was associated with an increased risk of SSI.

Using data from the FLOW trial, a large multicenter randomized study, we used descriptive statistics to quantify how frequently the GAC changed from the initial to definitive grading. We used regression models to determine which injury and patient characteristics were associated with increases and decreases in GAC and whether a change in GAC was associated with SSI.

Of the 2420 participants included, 305 participants had their preoperative GAC change (12.6%). The factors associated with upgrading the GAC (from preoperative score to the definitive assessment) included fracture sites other than the tibia, bone loss at presentation, width of wound, length of wound, and skin loss at presentation. However, initial misclassification of type III fractures as type II fractures was not associated with an increased risk of SSI (P = 0.14).

When treating patients with open fracture wounds, surgeons should consider that 12% of all injuries may initially be misclassified when using the GAC, particularly fractures that have bone loss at presentation or those located in sites different than the tibia. However, even in misclassified fractures, it did not seem to increase the risk of SSI.

This study aimed to reassess the effect of prophylactic transcervical amnioinfusion for intrapartum meconium-stained amniotic fluid on meconium aspiration syndrome and other adverse neonatal and maternal outcomes.

From inception to November 2021, a systematic search of the literature was performed in PubMed, Embase, Web of Science, and Scopus databases and gray literature sources.

We identified randomized controlled trials of patients with intrapartum moderate to thick meconium-stained amniotic fluid that evaluated the effect of amnioinfusion on adverse neonatal and maternal outcomes.

Of note, 2 reviewers independently abstracted data and gauged study quality by assigning a modified Jadad score. Meconium aspiration syndrome constituted the primary outcome. The secondary outcomes were meconium below the cords, Apgar scores of <7 at 5 minutes, neonatal acidosis, cesarean delivery, cesarean delivery for fetal heart rate abnormalities, neonatal intensive care unit admission, and postpartum endometritis. This study calculated the odds ratios with 95% confidence intervals for categorical outcomes and weighted mean differences with 95% confidence intervals for continuous outcomes.

A total of 24 randomized studies with 5994 participants met the inclusion criteria. The overall odds of meconium aspiration syndrome was reduced by 67% in the amnioinfusion group (pooled odds ratio, 0.33; 95% confidence interval, 0.21-0.51). Except for postpartum endometritis, amnioinfusion was associated with a significant reduction in all secondary outcomes.

Our study found that the use of intrapartum amnioinfusion in the setting of meconium-stained amniotic fluid significantly reduces the odds of meconium aspiration syndrome and other adverse neonatal outcomes.

Given variable frequency of misleading reports and the potential for spin (a way of describing results that can mislead readers) to influence interpretation of randomised controlled trials (RCTs), we have undertaken a spin reassessment. We evaluated the quality of recent literature in anaesthesia journals by assessing the presence of spin and calculating the fragility index.

This systematic review of randomised trials was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. We searched via PubMed® from January 1, 2019 to January 1, 2021 to identify all RCTs published in one of the 20 anaesthesia journals with the highest journal impact factors during this time. Four pairs of reviewers assessed articles independently for eligibility using a piloted electronic data extraction form. They assessed the presence of spin in statistically negative RCTs and calculated the fragility index for statistically positive RCTs.

Of the 802 screened records, 162 (20%) articles were analysed for spin, and 65 (8%) trials were analysed for fragility index. For the statistically negative studies, 66 articles (40%) presented spin; 89% of these occurrences of spin were described in the conclusion of the abstract. The primary type of spin was the highlight of secondary outcomes (67%). For statistically positive trials, the median fragility index was 4 [1-8].

This systematic review showed that 40% of statistically negative trials in high-impact anaesthesia journals could mislead readers. For statistically positive RCTs, the results relied on few subjects, with a median fragility index of 4 [1-8]. Efforts must be continued to reduce spin and fragility in the medical literature.

Significant results of randomized controlled trials (RCTs) should be properly weighed. This study adopted fragility index (FI) to evaluate the robustness of significant dichotomous outcomes from RCTs on coronavirus disease 2019 (COVID-19) treatment.

ClinicalTrials.gov and PubMed were searched from inception to July 31, 2021. FIs were calculated and their distribution was depicted. FI's categorical influential factors were analyzed. Spearman correlation coefficient (r _s) was reported for the relationship between FI and the continuous characteristics of RCTs.

Fifty RCTs with 120 outcomes in 7869 patients were included. The FI distribution was abnormal with median 3 (interquartile range 1-7, P = 0.0001). The FIs and robustness were affected by the outcomes of interest, various patient populations, and interventions (T = 18.215,16.667, 23.107; P = 0.02,0.0001, 0.001, respectively). A cubic relationship between the FIs and absolute difference of events between groups with R square of 0.848 (T = 215.828, P = 0.0001, R square = 0.865) was observed. A strong negative logarithmic relationship existed between FI and the P value with R square = - 0.834.

The robustness of significant dichotomous outcomes of COVID-19 treatments was fragile and affected by the outcomes of interest, patients, interventions, P value, and absolute difference of events between the groups. FI was an useful quantitative metric for the binary significant outcomes on COVID-19 treatments.

PROSPERO (CRD42021272455).

The online version contains supplementary material available at 10.1007/s44231-022-00027-y.

There is controversy regarding the optimal statistical method to interpret how robust is a statistically significant result. The fragility index (FI) and the reverse fragility index (RFI) are quantitative measures that can facilitate the appraisal of a clinical trial's robustness. This study was performed to evaluate the FI and RFI of randomized controlled trials (RCTs) examining nutritional interventions in patients with diabetes mellitus, focusing on cardiovascular outcomes.

A systematic search was conducted and relevant RCTs were identified in three databases. RCTs examining nutritional interventions (supplements or dietary patterns) in patients with DM with dichotomous primary endpoints involving cardiovascular outcomes were eligible. Data were extracted to compose 2 × 2 event tables and the FI and RFI were calculated for each comparison, using Fisher's exact test. Risk of bias (RoB) of the included RCTs was assessed with the Cochrane RoB 2.0 tool.

A total of 14,315 records were screened and 10 RCTs were included in the analyses. The median FI of the paired comparisons was 3 (IQR: 2-4) and the median RFI was 8 (IQR: 4.5-17). RoB and heterogeneity were low.

RCTs examining nutritional interventions and cardiovascular outcomes among patients with diabetes mellitus appear to be statistically fragile. Τhe FI and the RFI can be reported and interpreted as an additional perspective of a trial's robustness.

• In the evidence-healthcare era, assessing how robust statistically significant results are remains a matter of controversy. • Recently, the fragility index (FI) and reverse fragility index (RFI) were proposed to assess the robustness of randomized controlled trials (RCTs) with 2 × 2 comparisons. • When applying the FI and RFI, RCTs examining nutritional interventions and cardiovascular outcomes among patients with diabetes mellitus (DM) appear to be statistically fragile. • Τhe FI and the RFI can be reported and interpreted as an additional perspective of a trial's robustness. • RCTs implementing nutrition interventions among patients with DM can improve their methodology.

Fragility analysis is increasingly utilized to evaluate the robustness of results within the orthopaedic literature and has frequently revealed instability of reported outcomes.

The purpose of this investigation was to utilize a fragility analysis to evaluate the stability of reported results in the patellofemoral instability (PFI) literature. We hypothesized the demonstration of significant fragility in patellofemoral research to be similar to that identified throughout other areas of the orthopaedic literature.

Systematic review; Level of evidence, 4.

The PubMed database was queried from January 1, 2000, to October 10, 2020 for comparative trials in 10 prominent orthopaedic journals that reported dichotomous outcomes related to the management of PFI. The fragility index (FI) and the fragility quotient (FQ) were calculated for each individual outcome event, and the overall FI and FQ were determined for all included studies.

A total of 22 comparative studies comprising 11 randomized controlled trials and 11 nonrandomized trials were included for the analysis. A total of 75 outcome events underwent a fragility analysis and revealed a median FI and FQ of 3 (interquartile range [IQR], 1-5) and 0.043 (IQR, 0.018-0.081), respectively. Also 27% of included studies reported loss to follow-up greater than the overall FI, therefore suggesting the maintenance of the follow-up may have resulted in the reversal of significance.

The result of the comprehensive fragility analysis demonstrated a lack of robustness in PFI research with the alteration of only a few outcome events required to reverse statistical significance. We therefore recommend the triple reporting of the P value, the FI, and the FQ to aid in the interpretation of the statistical integrity of future comparative trials in the PFI literature.

Inhibition of the renin-angiotensin-aldosterone system (RAAS) is broadly recommended in many nephrological guidelines to prevent chronic kidney disease (CKD) progression. This work aimed to analyze the robustness of randomized controlled trials (RCTs) investigating the renal and cardiovascular outcomes in CKD stages 3-5 patients treated with RAAS inhibitors (RAASi). We searched for RCTs in MEDLINE (PubMed), EMBASE databases, and the Cochrane register. Fragility indexes (FIs) for every primary and secondary outcome were calculated according to Walsh et al., who first described this novel metric, suggesting 8 as the cut-off to consider a study robust. Spearman coefficient was calculated to correlate FI to p value and sample size of statistically significant primary and secondary outcomes. Twenty-two studies met the inclusion criteria, including 80,455 patients. Sample size considerably varied among the studies (median: 1693.5, range: 73-17,276). The median follow-up was 38 months (range 24-58). The overall median of both primary and secondary outcomes was 0 (range 0-117 and range 0-55, respectively). The median of FI for primary and secondary outcomes with a p value lower than 0.05 was 6 (range: 1-117) and 7.5 (range: 1-55), respectively. The medians of the FI for primary outcomes with a p value lower than 0.05 in CKD and no CKD patients were 5.5 (range 1-117) and 22 (range 1-80), respectively. Only a few RCTs have been shown to be robust. Our analysis underlined the need for further research with appropriate sample sizes and study design to explore the real potentialities of RAASi in the progression of CKD.

There is increasing awareness throughout biomedical science that many results do not withstand the trials of repeat investigation. The growing abundance of medical literature has only increased the urgent need for tools to gauge the robustness and trustworthiness of published science. Dichotomous outcome designs are vital in randomized clinical trials, cohort studies, and observational data for ascertaining differences between experimental and control arms. It has however been shown with tools like the fragility index (FI) that many ostensibly impactful results fail to materialize when even small numbers of patients or subjects in either the control or experimental arms are recoded from event to non-event. Critics of this metric counter that there is no objective means to determine a meaningful FI. As currently used, FI is not multidimensional and is computationally expensive. In this work, a conceptually similar geometrical approach is introduced, the ellipse of insignificance. This method yields precise deterministic values for the degree of manipulation or miscoding that can be tolerated simultaneously in both control and experimental arms, allowing for the derivation of objective measures of experimental robustness. More than this, the tool is intimately connected with sensitivity and specificity of the event/non-event tests, and is readily combined with knowledge of test parameters to reject unsound results. The method is outlined here, with illustrative clinical examples.

Proximal humeral fracture represents an increasingly common pathology with evaluation and treatment often guided by evidence from randomized controlled trials (RCTs), but the strength of an RCT must be considered in this process. The purpose of this study was to evaluate the strength of outcomes in RCTs on the management of proximal humeral fractures using the fragility index (FI), a method used with statistically significant dichotomous outcomes to assess the number of patients that would change an outcome measure from significant (P ≤ .05) to nonsignificant if the patient outcome changed. We also aimed to correlate the FI with other measures of study strength.

A systematic review was performed using Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines to evaluate RCTs on the management of proximal humeral fractures. The PubMed, Ovid MEDLINE, Web of Science, and Embase databases were searched from database inception to May 2021. RCTs with at least 1 statistically significant (P ≤ .05) dichotomous outcome were included. The FI was calculated for each included trial using the Fisher exact test. The FI was correlated with the study sample size and journal impact factor.

Ten RCTs reporting on 656 patients and published between 2011 and 2020 were included. The median patient sample size was 67 (mean, 65.6; range, 40-86). Complications were the most commonly reported dichotomous statistically significant outcome. The median FI was 1 (mean, 2.6; range, 0-18), with 4 studies having an FI of 0. A median FI of 1 indicates that 1 patient experiencing an alternative outcome or having not been lost to follow-up could have changed the pertinent conclusions of the trial for a given outcome. The median number of patients lost to follow-up was 3 (mean, 4.9; range, 0-16) and exceeded the FI in 50% of studies. There was no correlation between the FI and sample size (Spearman coefficient = 0.0592, P = .865) or between the FI and journal impact factor (Spearman coefficient = -0.0229, P = .522).

In most studies of proximal humeral fractures, only 1 or 2 patients experiencing an alternative outcome or lost to follow-up would change the conclusions for the dichotomous outcome studied. Although the FI cannot be used to assess continuous variables, which are often the primary outcome variables of RCTs, it does offer an additional unique measure of study strength that surgeons should consider when evaluating RCTs.

The fragility index has been gaining ground in the evaluation of comparative clinical studies. Many scientists evaluated trials in their fields and deemed them to be fragile, although there is no consensus on the definition of fragility. We aimed to calculate the fragility index and its permutations for paediatric surgical trials.

We searched pubmed for prospectively conducted paediatric surgical trials with intervention and control group without limitations and calculated their (reverse) fragility indices and respective quotients along with posthoc-power. Relationships between variables were evaluated using Spearman's ρ. We also calculated S values by negative log transformation base-2 of P values.

Of 516 retrieved records, we included 87. The median fragility index was 1.5 (interquartile range: 0-4) and the median reverse fragility index was 3 (interquartile range: 2-4), although they were statistically not different (Mood's test: χ² = 0.557, df = 1, P = 0.4556). P values and fragility indices were strongly inversely correlated (ρ = - 0.71, 95% confidence interval: - 0.53 to - 0.85, P < 0.0001), while reverse fragility indices were moderately correlated to P values (ρ = 0.5, 95% confidence interval: 0.37-0.62, P < 0.0001). A fragility index of 1 resulted from P values between 0.039 and 0.003, which resulted in S values between 4 and 8.

Fragility indices, reverse fragility indices, and their respective fragility quotients of paediatric surgical trials are low. The fragility index can be viewed as no more than a transformed P value with even more substantial limitations. Its inherent penalisation of small studies irrespective of their clinical relevance is particularly harmful for paediatric surgery. Consequently, the fragility index should be avoided.

With the growing concerns about research reproducibility and replicability, the assessment of scientific results' fragility (or robustness) has been of increasing interest. The fragility index was proposed to quantify the robustness of statistical significance of clinical studies with binary outcomes. It is defined as the minimal event status modifications that can alter statistical significance. It helps clinicians evaluate the reliability of the conclusions. Many factors may affect the fragility index, including the treatment groups in which event status is modified, the statistical methods used for testing for the association between treatments and outcomes, and the pre-specified significance level. In addition to assessing the fragility of individual studies, the fragility index was recently extended to both conventional pairwise meta-analyses and network meta-analyses of multiple treatment comparisons. It is not straightforward for clinicians to calculate these measures and visualize the results. We have developed an R package called "fragility" to offer user-friendly functions for such purposes. This article provides an overview of methods for assessing and visualizing the fragility of individual studies as well as pairwise and network meta-analyses, introduces the usage of the "fragility" package, and illustrates the implementations with several worked examples.

Debate centering on the management of anterior shoulder instability (ASI) in recent years has led to many randomized controlled trials (RCTs) being published on the topic. The fragility index (FI) has subsequently emerged as a novel method of assessing significant findings reported in RCTs, particularly those with small sample sizes.

To evaluate the FI of statistically significant findings in RCTs that reported the outcomes of management strategies of patients with ASI.

Systematic review; Level of evidence, 1.

Using PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, 2 independent reviewers performed a systematic review of RCTs focusing on the outcomes of management strategies of patients with ASI. There were 3 main categories of RCTs included: (1) nonoperative management in internal rotation (IR) versus external rotation (ER), (2) nonoperative management versus a surgical intervention, and (3) surgical management with arthroscopic Bankart repair versus open Bankart repair. The Fisher exact test was utilized to calculate the FI for the reversal of statistical significance in all statistically significant dichotomous outcomes.

A total of 21 RCTs were included, including 1589 shoulders (mean age, 29.4 years) with a mean follow-up of 26.8 months. There were 10 RCTs (831 shoulders) that reported outcomes after the nonoperative management of ASI in IR versus ER, with a mean FI of 6.8. There were 5 RCTs (324 shoulders) that reported outcomes comparing the nonoperative and operative management of ASI, with a mean FI of 3.5. There were 6 RCTs (434 shoulders) that reported outcomes after the operative management of ASI with either arthroscopic Bankart repair or open Bankart repair, with a mean FI of 9.6.

The overall FI of RCTs reporting the outcomes of management strategies for patients with ASI was high, suggesting a moderate fragility of statistically significant outcomes including recurrence, revision stabilization, and return to play.

Interpreting results from randomized clinical trials (RCTs) for COVID-19, which have been published rapidly and in vast numbers, is challenging during a pandemic.

To evaluate the robustness of statistically significant findings from RCTs for COVID-19 using the fragility index.

This cross-sectional study included COVID-19 trial articles that randomly assigned patients 1:1 into 2 parallel groups and reported at least 1 binary outcome as significant in the abstract. A systematic search was conducted using PubMed to identify RCTs on COVID-19 published until August 7, 2021.

Trial characteristics, such as type of intervention (treatment drug, vaccine, or others), number of outcome events, and sample size.

Fragility index.

Of the 47 RCTs for COVID-19 included, 36 (77%) were studies of the effects of treatment drugs, 5 (11%) were studies of vaccines, and 6 (13%) were of other interventions. A total of 138 235 participants were included in these trials. The median (IQR) fragility index of the included trials was 4 (1-11). The medians (IQRs) of the fragility indexes of RCTs of treatment drugs, vaccines, and other interventions were 2.5 (1-6), 119 (61-139), and 4.5 (1-18), respectively. The fragility index among more than half of the studies was less than 1% of each sample size, although the fragility index as a proportion of events needing to change would be much higher.

This cross-sectional study found a relatively small number of events (a median of 4) would be required to change the results of COVID-19 RCTs from statistically significant to not significant. These findings suggest that health care professionals and policy makers should not rely heavily on individual results of RCTs for COVID-19.

We aimed to determine the post-hoc power of randomized controlled trials (RCTs) in critical care, and describe the implications for long-term positive (PPV) and negative predictive value (NPV) of statistically significant and non-significant findings respectively in the research field.

We reviewed three cohorts of RCTs. "Adult-RCTs" were 216 multicenter RCTs with a mortality outcome from a published systematic review. "Pediatric-RCTs" were 120 RCTs with a mortality outcome, obtained by search of picutrials.net. "Consecutive-RCTs" were 90 recent RCTs obtained by screening publications in 6 journals. Post-hoc power for each study was calculated at α 0.05 and 0.005, for measures of small, medium, and large effect-size, using G*Power software. Long-run expected PPV and NPV of critical care research field findings were then calculated.

With α 0.05, post-hoc power for small effect-size was very low in all RCT-cohorts (eg, median 24% in Adult-RCTs). For medium effect-size, post-hoc power was low, except for Adult-RCTs (eg, median 9% in Pediatric-RCTs). For large effect-size, post-hoc power for non-human-animal Consecutive-RCTs was low (median 32%). With α 0.005, post-hoc power was even lower. The corollary was that both PPV and NPV were poor for small effect-size, unless α 0.005 was used. Even with α 0.005, with realistic (vs. optimistic) prior probability of the alternative hypothesis, the PPV was low (eg, in Adult-RCTs 57.1% vs. 92.3%). Adding mild bias (0.1) reduced the PPV even further. For medium effect-size both PPV and NPV were better; nevertheless, with α 0.05 and realistic prior probability of the alternative hypothesis the PPV was poor, and with α 0.005 and mild bias (0.1) the PPV was very low (eg, Adult-RCTs median 44.1%).

To improve the predictive value of findings in the critical care research field, RCTs should be designed to have 80% power for realistic effect-size at α 0.005.

The American Statistical Association, among others, has called for the use of statistical methods beyond p ≤ 0.05. The fragility index is a statistical metric defined as the minimum number of patients for whom if an event rather than a nonevent occurred, then the p value would increase to ≥0.05. Previous reviews have demonstrated that many randomized controlled trials have a low fragility index, suggesting they may not be robust.

The purpose of this study was to review the fragility indices of randomized controlled trials in colorectal surgery.

A PubMed search was performed.

Colorectal surgery randomized controlled trials with a dichotomous primary outcome p ≤ 0.05 and publication between 2016 and 2018 were systematically identified.

All procedural interventions related to colorectal surgery were included.

The main measures were the fragility index and the number of patients lost to follow-up for each trial. The percentage of trials with the number of patients lost to follow-up greater than the fragility index was calculated.

In total, 712 abstracts were reviewed, with 90 trials meeting the inclusion criteria. The median fragility index was 3 (interquartile range of 1 to 10). In 51 of the 90 trials (57%), the number of patients lost to follow-up was greater than the fragility index.

The fragility index is only one measure of the robustness of a randomized clinical trial.

Most colorectal surgery randomized controlled trials have a low fragility index. In 57% of trials, more patients were lost to follow-up than would be required to change the outcome of the trial from "significant" to "nonsignificant" based on the p value. This emphasizes the importance of assessing the robustness of clinical trials when considering their clinical application, rather than relying solely on the p value. See Video Abstract at http://links.lww.com/DCR/B741.CUANDO EL VALOR-P ES INSUFICIENTE: ÍNDICE DE FRAGILIDAD APLICADO EN ESTUDIOS ALEATORIOS CONTROLADOS EN CIRUGÍA COLORECTAL.

La Sociedad Estadounidense de Estadística, entre otros, ha pedido el uso de métodos estadísticos más allá de p <0,05. El índice de fragilidad es una medida estadística definida como el número de desenlaces que podrían cambiar para revertir, o conseguir, la significación estadística, así el valor p aumentaría a ≥ 0,05. Las revisiones anteriores han demostrado que muchos estudios aleatorios controlados tienen un índice de fragilidad bajo, lo que sugiere que pueden poco sólidos.

El propósito de la présente investigación fué de revisar los índices de fragilidad de los estudios aleatorios controlados en cirugía colorrectal.

PubMed.

Se identificaron sistemáticamente estudios aleatorios controlados de cirugía colorrectal con un resultado primario dicotómico, valor de p ≤ 0,05 y publicados entre 2016-2018.

Se incluyeron todas aquellas intervenciones con procedimientos relacionados con la cirugía colorrectal.

Las principales medidas fueron: el índice de fragilidad y el número de pacientes perdidos durante el seguimiento en cada estudio. Se calculó el el índice de fragilidad en porcentaje de estudios con el mayor número de pacientes perdidos durante el seguimiento mas prolongado.

En total, se revisaron 712 resúmenes con 90 ensayos que cumplieron con los criterios de inclusión. La mediana del índice de fragilidad fue de 3 (rango intercuartíl de 1 a 10). En 51 de los 90 estudios (57%), el número de pacientes perdidos durante el seguimiento fue mayor que el índice de fragilidad.

El índice de fragilidad es solo una medida de la robustez de un estúdio clínico aleatorio.

La mayoría de los estudios aleatorios y controlados en cirugía colorrectal tienen un índice de fragilidad bajo. En el 57% de los estudios, se perdieron más pacientes durante el seguimiento de los que se necesitarían para cambiar el resultado del estudios de grado "significativo" a un grado "no significativo" según el valor-p. Este concepto enfatiza la importancia de evaluar la robustez de los estudios clínicos al considerar su aplicación verdadera aplicación clínica, en lugar de depender únicamente del valor-p. Consulte Video Resumen en http://links.lww.com/DCR/B741. (Traducción-Dr. Xavier Delgadillo).

As a metric to determine the robustness of trial results, the fragility index (FI) is the number indicating how many patients would be required to reverse the significant results. This study aimed to calculate the FI in randomized controlled trials (RCTs) involving premature.

Trials were included if they had a 1:1 study design, reported statistically significant dichotomous outcomes, and had an explicitly stated sample size or power calculation. The FI was calculated for binary outcomes using Fisher's exact test, and the FIs of subgroups were compared. Spearman's correlation was applied to determine correlations between the FI and study characteristics.

Finally, 66 RCTs were included in the analyses. The median FI for these trials was 3.00 (interquartile range [IQR]: 1.00-5.00), with a median fragility quotient of 0.014 (IQR: 0.008-0.028). FI was ≤ 3 in 42 of these 66 RCTs (63.6%), and in 42.4% (28/66) of the studies, the number of patients lost to follow-up was greater than that of the FI. Significant differences were found in the FI among journals (p = 0.011). We observed that FI was associated with the sample size, total number of events, and reported p-values (r _s = 0.437, 0.495, and -0.857, respectively; all p < 0.001).

For RCTs in the premature population, a median of only three events was needed to change from a "non-event" to "event" to render a significant result non-significant, indicating that the significance may hinge on a small number of events.