Low back pain-related meta-analysis: Caution is needed when interpreting published research results

Abstract

The systematic reviews (SRs) including a meta-analysis are considered as the top level of evidence. Although the existence of more than a hundred of low back pain (LBP)-related SRs seems very appealing and might therefore suggest significant evidence on the topic, a deep analysis indicates that several of these SRs included only very few studies. Other SRs raise concerns because they included some randomized controlled trials which had a low methodological quality, or some studies which differed significantly regarding the studied populations and/or the experimental procedure. The sometimes controversial results of different SRs conducted on the same topic also highlight the significant influence of the inclusion/exclusion criteria used in the SRs to select the articles. To conclude, although meta-analysis is at the top of the evidence pyramid and have several strengths, the conclusions drawn from SRs should always be interpreted with caution because they can also have weaknesses. This is true, whether it be for LBP-related SRs including a meta-analysis, or any other. Therefore a critical analysis of any SR is always needed before integrating the results of the SR in its own clinical practice. Furthermore, clinical reasoning remains crucial, especially to consider the potential differences between one’s patient and the patients included in the meta-analysis.

Key Words: Meta-analysis, Systematic review, Spine, Back pain, Limitations, Recommendations, Evidence-based practice

Core tip: Although meta-analysis are at the top of the evidence pyramid and have several strengths, the conclusions drawn from systematic reviews combined to a meta-analysis should always be interpreted with caution because they can also have weaknesses. This is true, whether it be for low back pain-related systematic reviews including a meta-analysis, or any other. Therefore, a critical analysis of a systematic review is always needed before integrating the results in its own clinical practice. Furthermore, clinical reasoning remains crucial, especially to consider the potential differences between one’s patient and the patients included in the meta-analysis.

INTRODUCTION

Nowadays, the relevance and necessity to treat patients according to the evidence-based medicine is widely recognized[1]. This approach aims to integrate clinical expertise, patient values and the best research evidence[1,2]. Regarding the latter component, it has become impossible for clinicians, and even scientists, to read all the papers on a specific topic considering the constant increasing number of scientific studies conducted every year[3]. Therefore, narrative and systematic reviews (SRs) have become very popular “easy access” methods for clinicians and researchers to help them to overview the scientific literature[1]. In contrast to the narrative reviews, which are rather qualitative summaries based on the knowledge of an experienced author, the SRs are performed according to a complex but very transparent process of search[4]. Furthermore, its combination with a meta-analysis provides a statistical summary of the evidence (e.g., treatment effect, diagnostic method, prognosis, etc.) regarding a given topic, thereby facilitating the integration of the best evidence into practice. Meta-analysis also allow increasing the power, improving precision, and analyzing the consistency of effects[4,5]. Because the systematic reviews with a meta-analysis are considered as the top level of evidence, their results can greatly influence the guidelines and decisions related to a specific topic.

The number of meta-analysis published in the scientific literature is increasing in an exponential way[6]; the low back pain (LBP) field is no exception. This is well illustrated by a quick search of Pubmed with the generic terms “low back pain” and “meta-analysis” at the beginning of January 2015 which resulted in 377 hits, among which half of them (n = 166) were published over the last five years. An analysis of the hits reveals that two thirds of these 166 hits were effectively SRs combined to a meta-analysis related to LBP. The main topics concerned surgery, exercises, injections, pharmacological treatments, risk factors, spinal manual/manipulative therapies and imaging. Most of them were meta-analyses of randomized controlled trials (RCTs). There were also an overview of systematic reviews[7] and a SR of systematic reviews[8] reflecting the expansion of the SRs.

Although the existence of more than a hundred of LBP-related SRs seems very appealing and might therefore suggest significant evidence on the topic, a deep analysis indicates that several of these SRs combined to a meta-analysis included only very few studies (because of the low number of studies on the topic and/or of the low methodological quality of several studies) (e.g.,[9,10]). In some SRs, a meta-analysis was planned but was not conducted due to a lack of articles on the topic (e.g.,[11]) or a high clinical heterogeneity between studies regarding the intervention (e.g.,[12,13]) or the functional outcome measures (e.g.,[14]). The choice to conduct or not a meta-analysis appears subjective and differed between authors (e.g., Hansen et al[11] decided to perform it only if the literature search resulted in at least 5 RCTs meeting the predefined inclusion criteria and if they were homogeneous whereas other performed a meta-analysis based only on three studies (e.g.,[10]).

Other SRs raise concerns because they included some RCTs which had a low methodological quality (e.g., 5 out of the 7 RCTs included in the review of Ebadi et al[15] on the effectiveness of therapeutic ultrasound for chronic LBP had a score < 6/12 when using the 12 criteria recommended by the Cochrane Back Review Group) or some studies which differed significantly regarding the experimental procedure (e.g., treatment provided). The pooling of spinal mobilizations and spinal manipulations[16], which are two different techniques regarding the indications, effects, etc., or of a specific technique (manipulation) and a manual therapy concept (combination of techniques)[17], are good examples of mixing heterogeneous studies[18]. Another example of not ideal pooling is taken from meta-analysis regarding the effectiveness of “exercises” for LBP. Indeed, some conducted a calculation of a summary estimate[19] although there exists so many different types of exercise and although some parameters might influence the treatment outcomes[20] (e.g., the number of sessions[21]). Another review on the topic tried to distinguish the different types of exercises[22], but the pooling was not always relevant[23]. Thus, reviewing the effectiveness of some LBP-related treatments (e.g., physical therapy) is much more complex than for other treatments (e.g., drug therapy) which are less heterogeneous between studies.

The characteristics of the participants are other crucial parameters to consider when conducting and interpreting a SR related to LBP. Indeed, in contrast to specific diseases (e.g., influenza), “non-specific LBP” has no identifiable cause and is rather a symptom. Furthermore, according to the bio-psycho-social model and the literature on the topic, numerous (individual, psychosocial, work-related) factors influence the outcome/prognosis of musculoskeletal pain and should be taken into account when treating a patient, especially in case of chronic pain[24,25]. Therefore, lots of subgroups of patients with LBP have been described in literature[26]. A meta-analysis about the effectiveness of classification-based interventions reveals that such interventions seem more effective for reducing pain and disability than “standard” treatments. This highlights the potential bias when pooling studies with different populations. To solve this problem, a subgroup analysis can be conducted to study the influence of several parameters. However, one has to keep in mind that an effect of chance can occur when analyzing too many subgroups[27]; moreover, a low number of studies included in the SRs prevents to achieve such an analysis.

The inclusion/exclusion criteria used in the SRs to select the articles are also important to consider when interpreting the results of a meta-analysis because they can have a significant influence[6]. Indeed, a SR of the SRs conducted on the effectiveness of the Pilates exercises in patients with chronic LBP revealed that the 5 SRs which had been published on the topic by that time had different conclusions although they had similar research objectives[8]; furthermore, only two out of the ten primary studies were included in the 5 papers[8]. Besides, some SRs considered only quantitative results from questionnaires while ignoring qualitative studies which might therefore introduced also a bias[28].

Another point to be noticed about the LBP-related meta-analyses is that most of them have been conducted on aggregate-level data whereas only a few have been performed on individual data (e.g.,[29]), although the latter analysis appears very relevant[30].

To conclude, although meta-analyses are at the top of the evidence pyramid and have several strengths, the conclusions drawn from SRs should always be interpreted with caution because they can also have weaknesses. This is true, whether it be for LBP-related SRs including a meta-analysis, or any other. Interpreting results of a meta-analysis is not easy, as evidenced by the letters to the editor[18,31] related to the SR of Licciardone et al[17]. Using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis checklist can help readers for critical appraisal purposes. Besides, analyzing the methodology of the SR, examining the studies included (e.g., population, intervention, outcome) and thinking about the possible sources of heterogeneity of the results (in case it occurs) are really necessary before integrating the results of the SR in its own clinical practice[2,6,27]. Furthermore, clinical reasoning remains crucial[32], especially to consider the potential differences between one’s patient and the patients included in the meta-analysis[27]. Only reading the abstract of a SR combined to a meta-analysis is clearly not good enough to do so.