Rationalising animal research synthesis in orthopaedic literature

Abstract

Systematic reviews in orthopaedic literature are frequently criticised for offering inconsistent conclusions. On top of that, high-quality randomized human evidence on crucial orthopaedic topics is more often than not lacking. In this situation, pooling animal literature could offer an excellent insight into unanswered critical clinical questions, thus potentially improving healthcare. In this paper, we sought to present the rationale and basic principles governing meta-analysis of animal research. More specifically, we elaborated on the available evidence-based methods to achieve a scientifically sound animal data synthesis. In addition, we discussed result interpretation, strength of recommendations and clinical implications based on the results of these meta-analytic modalities.

Key Words: Meta-analysis; Animal research; Evidence synthesis; in vivo; Orthopaedics

Core Tip: Relying on the findings of properly conducted meta-analyses of animal research is crucial, particularly in the paucity of human evidence on crucial orthopaedic topics. It is an undeniable fact that authors tend to encounter a great many challenges when conducting this type of research as they have to address several potential sources of bias. For that reason, we advocate that readers should critically appraise the findings of animal syntheses papers.

INTRODUCTION

Given the nature and rarity of many orthopaedic diseases, conducting high-quality double-blind randomized control trials is not always feasible. This is particularly true when it comes to addressing a particular orthopaedic surgical intervention. Hence, crucial research questions remain unanswered due to the fact that safe conclusions cannot be drawn purely based on a few underpowered and low-quality individual studies. In this situation, animal evidence could offer valuable information towards delineating the potential of a prevention and/or therapeutic orthopaedic intervention.

The rationale behind synthesizing animal literature is to avoid the potential bias which is commonly detected in narrative literature reviews. To elaborate, selective presentation of individual study findings and incorrect weighting of conclusions can exert a negative impact on the credibility of a systematic review. Rather, by summarizing the results of multiple individual studies a researcher could potentially produce more valid results provided that guidelines governing meta-analyses of animal papers are respected. In this paper, we sought to present the key elements for conducting a high-quality meta-analysis of animal research which could provide a useful insight into unanswered clinical questions in orthopaedics.

PROSPECTIVE ANIMAL REVIEW REGISTRATION AND REPORTING GUIDELINES. ARE THEY NECESSARY?

Regardless of the nature of the subjects utilised in an in vivo evidence synthesis, it is strongly advocated that systematic reviews be prospectively registered with a valid database (e.g., PROSPERO). The main reason behind this protocol registration is to increase transparency in reporting and prevent selective outcome reporting issues.

On top of that, abiding by published guidelines for systematic reviews (e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is of utmost importance given the fact that poor reporting diminishes accuracy and potential usefulness of an animal meta-analysis[1].

CONTROLLED VS UNCONTROLLED DATA SYNTHESIS: IS THERE ANY DIFFERENCE?

From a methodological standpoint, if properly controlled homogenous groups are available, then standard head-to-head meta-analysis can be safely undertaken by using a readily available piece of statistical software [e.g., Review Manager (RevMan)][2]. However, synthesising uncontrolled research represents a different task which can be achieved by means of proportional meta-analysis[3]. It is underlined that although indirect comparisons could be made by comparing overlapping of confidence intervals in the aforementioned type of meta-analysis, safe conclusions on the comparative efficacy of interventions cannot be reached and therefore this approach is not generally recommended.

LUMPING INTERVENTION GROUPS IN META-ANALYSES OF ANIMAL RESEARCH

One frequently encountered methodological issue in pair-wise meta-analyses is the limited statistical power precluding reliable conclusions to be drawn[4]. To address this issue, lumping intervention groups into valid subgroups with respect to literature classifications[5,6] is recommended. By and large, a crucial point authors need to pay attention to when they elect for the subgroup pathway is the trade-off between statistical power and precision in reporting. We advocate that as long as published guidelines have been followed prior to creating subgroups and sensitivity analysis has been conducted to investigate the impact of subgrouping on the data synthesis, the validity of the findings is not severely compromised.

POOLING DICHOTOMOUS AND CONTINUOUS DATA MEASURING THE SAME OUTCOME. IS IT POSSIBLE?

Encountering a situation where information for the same outcome is presented in some studies as dichotomous data and in other papers by means of a continuous variable is a common phenomenon in animal research. To address this issue, re-expressing standardized mean differences to odds ratios (or the vice versa) is recommended[7]. Subsequently the generic inverse variance model in RevMan can be utilised to pool those converted data together[7] (Figure 1). Although we recognise this could be a challenging task for a researcher to accomplish, the problem of missing information which may compromise the validity of the meta-analysis results can be overcome[5].

Figure 1 Forest plot of standardised mean differences with multiple subgroup analyses is demonstrated. The methicillin-resistant Staphylococcus aureus infection prevention potential is assessed by means pair-wise meta-analysis in inverse variance mode to consider not only continuous but also dichotomous data in the analysis. CI： Confidence interval; CSA: Cationic steroidal antimicrobial; HA: Hydroxyapatite; SMD: Standardised mean difference; IV: Inverse variance; PA: Periapatite; PLEX: Polymer-lipid encapsulation matrix; SE: Standard error; TiO2 = Titanium dioxide. Citation: Tsikopoulos K, Sidiropoulos K, Kitridis D, Hassan A, Drago L, Mavrogenis A, McBride D. Is coating of titanium implants effective at preventing Staphylococcus aureus infections? A meta-analysis of animal model studies. Int Orthop 2020. Copyright© The Author(s) 2020. Published by Springer Nature Publishing Group[5]. The authors have obtained the permission for figure using from the Springer Nature Publishing Group (Supplementary material).

FEASIBILITY OF EXTRACTING QUANTITATIVE DATA FROM GRAPHICAL PRESENTATIONS

Meticulous data extraction is a crucial element in performing a satisfactory systematic review and meta-analysis. It is a common phenomenon in original papers published a long time ago to present their findings in a graphical manner with no corresponding numerical data. In this situation, taking advantage of the use of an appropriate software tool (e.g., Plot Digitizer and Getdata Graph Digitizer)[8] which allows for reliable digitization of graphs and/or plots is recommended to abstract and subsequently synthesise the required information.

QUALITY ASSESSMENT IN SYSTEMATIC REVIEWS OF ANIMAL PAPERS

Quality appraisal of individual animal studies performed by means well-established tool such as the SYRCLE’s Risk of Bias tool[9], ensures consistency and prevents discrepancies in assessing risk of bias in systematic reviews of animal intervention studies. SYRCLE’s Risk of Bias tool is an adaptation of the Cochrane Risk of Bias tool which could potentially facilitate transition of animal research into clinical practice. On top of that, due to the relatively standardised use of this instrument in the existing literature, the necessity of improving particular methodological aspects of animal studies can be easily stressed[9]. It should be noted that a graphical quantification of the risk of bias summarising the assessments for each domain could be of essence (Figure 2)[5].

Figure 2 Quantification of risk of bias assessment enables not only summarising quality appraisal results but also making judgments as to what the future studies should look at. Citation: Tsikopoulos K, Sidiropoulos K, Kitridis D, Hassan A, Drago L, Mavrogenis A, McBride D. Is coating of titanium implants effective at preventing Staphylococcus aureus infections? A meta-analysis of animal model studies. Int Orthop 2020. Copyright© The Author(s) 2020. Published by Springer Nature Publishing Group[5]. The authors have obtained the permission for figure using from the Springer Nature Publishing Group (Supplementary material).

IS PUBLICATION BIAS A COMMON THREAT TO VALIDITY IN LABORATORY ANIMAL RESEARCH?

It is an undeniable fact that “negative” laboratory animal results more often than not remain unpublished[10]. Therefore, exploration of selective reporting in animal papers appears to be critical. In other words, merely relying on statistical significance may introduce bias in the results of the statistical analysis and potentially threaten the validity of the meta-analysis findings.

HIERARCHY OF EVIDENCE-BASED MEDICINE AND BIAS ASSESSMENT

It is highlighted that while a systematic review is generally better than an individual study, a meta-analysis of animal studies should not be placed at the top of the hierarchy in a pyramid that depicts validity[11]. This is because a meta-analysis is as good as the studies identified and included[12]. Nevertheless, in the absence of high-quality evidence, relying on the results of a meta-analysis of animal models is advisable provided that caution is exercised due to potential bias.

INTERPRETING RESULTS AND DRAWING CONCLUSIONS

It is worthy of note that prior to drawing meta-analysis conclusions, sample size of the included comparison groups, quality rating of the involved studies, effect sizes, and statistical heterogeneity should be taken into account. On top of that, investigating the impact of various sources of clinical heterogeneity by means of a sensitivity analysis (i.e., exclusion of one or more papers from the analysis to assess the impact of a particular confounding factor on the findings of the study) with a view to verify the meta-analysis results is strongly advocated.

CONCLUSION

Despite the abundance of literature on developing meta-analytic skills relating to human data, methodological papers dealing with animal data synthesis are lacking. In the current article, we focused on the technicalities and implications of pooling animal literature which could be particularly useful when investigating the results of orthopaedic surgical interventions in the absence of human evidence. It is worthy of note that due to the experimental nature of animal papers, a certain amount of uncertainty in the meta-analysis conclusions is anticipated. For that reason, we advise caution when it comes to extrapolating the results of this type of research back to human biology.