Use of the varying coefficient model in an exercise and depression meta-analysis

Abstract

AIM: Use a recently developed varying coefficient model to determine the effects of exercise in adults with depression.

METHODS: Data from a recent meta-analysis addressing the effects of exercise on depression in adults were used. Studies were limited to randomized controlled intervention trials of any type of chronic exercise (for example, walking and jogging) in adults greater than or equal to 18 years of age with a diagnosis of depression. For each study, the standardized mean difference (exercise minus control) effect size for depression, adjusted for small-sample bias, was calculated. Variance statistics for each effect size and pooling of results were calculated using the recently proposed varying coefficient (VC) model for standardized mean differences. Standardized effect-sizes of 0.20, 0.50 and 0.80 were considered to represent small, medium and large effects. Results were considered statistically significant if the 95% confidence intervals did not cross 0, with negative results indicative of reductions in depression. These findings were then compared with results using traditional random-effects (RE) models.

RESULTS: A total of 23 studies representing 907 men and women (476 exercise, 431 control) were pooled for analysis. Both RE and VC models resulted in large, statistically significant improvements in depression as a result of exercise in adults. However, the VC model resulted in a larger overall effect size as well as confidence intervals that were narrower than previously reported using the RE model. The overall mean effect size for the RE model was -0.82 with a 95% confidence interval of -1.12 to -0.51. For the VC model, overall mean effect size was -0.88 with a 95% confidence interval of -1.08 to -0.68. The relative difference between the RE and VC approaches was 7.3%.

CONCLUSION: The VC model, a potentially preferable model, confirms the positive effects of exercise on depression in adults.

Key Words: Random-effects; Meta-analysis; Varying coefficient; Exercise; Depression

INTRODUCTION

Depression is a major public health problem affecting an estimated 9.0% of adults in the United States[1]. A recent meta-analysis using the traditional random-effects (RE) model reported statistically significant improvements in depressive symptoms as a result of any exercise modality in adults with depression[2]. However, since that time, a potentially more valid varying coefficient (VC) model has been proposed[3]. Given the importance of reaching accurate conclusions regarding a body of research, the objective of this brief report was to confirm the previously reported effects of exercise on depression in adults using the newly proposed VC model.

MATERIALS AND METHODS

Data source

Data from a recent meta-analysis addressing the effects of exercise on depression in adults, details of which have been described elsewhere, were used[2]. Briefly, studies were limited to randomized controlled intervention trials of any type of chronic exercise in adults ≥ 18 years of age with a diagnosis of depression, as defined by trial authors[2].

Calculation and pooling of effect sizes for VC model

For each study, the standardized mean difference (exercise minus control) ES for depression, adjusted for small-sample bias, was calculated[4,5]. Variance statistics for each ES and pooling of results were calculated using the recently proposed VC model for standardized mean differences, a model that makes no assumptions with respect to a common ES or random sampling from a normally distributed super-population of studies[3]. The only assumption is that a random sample is obtained from each study population.

The mean unweighted estimate of the population standardized mean difference ES was calculated as follows: δ¯=m−1∑i=1mbiδ^i

where δ¯ is the pooled mean standardized ES, m represents each study, δ^is the standardized ES from each study, and b_i is Hedge’s small sample-size adjustment for ESs, calculated as b_i = 3/[4(/n_i +n₂)-9][5]. The independent samples variance for each study was calculated as: var(δ^i)=[δ^i2(σ^i14/dfi1+σ^i24/dfi2)/8σ^i4+(σ^i12/dfi1+σ^i22/dfi2)σ^i2]

where df_ij = n_ij - 1 and σ^i=[(σ^i12+σ^i22)/2]1/2

The 95% confidence interval around the pooled standardized mean ES is calculated as: δ¯±zα/2[m−2∑i=1mbi2var(δ^i)]1/2

When compared to fixed-effect (FE) and RE approaches, the VC model has been shown to have superior coverage probabilities for standardized ESs[3]. Generally, standardized ESs of 0.20, 0.50 and 0.80 represent small, medium and large effects[6]. Results were considered statistically significant if the 95% confidence intervals did not cross 0, with negative results indicative of reductions in depression. All VC data were analyzed using Synthesizer 1.0[7].

RESULTS

A total of 23 studies representing 907 men and women (476 exercise, 431 control) were pooled for analysis[2]. Both RE and VC models resulted in large, statistically significant improvements in depression as a result of exercise in adults [RE, X¯, 95% CI: -0.82 (-1.12 to -0.51); VC: -0.88 (-1.08 to -0.68)]. However, the VC model resulted in a larger overall ES of 7.3% as well as confidence intervals that were narrower than previously reported using the RE model[2].

DISCUSSION

Exercise, a nonpharmacologic intervention that can be implemented in the public health setting, has been reported to reduce depression in adults[2]. Using a recently developed and novel VC model[3], the findings of the current investigation support the overall results of a previous RE meta-analysis in which exercise resulted in improvements in depression among adults diagnosed with depression[2]. Confirmation of these findings is important given the prevalence and costs, estimated to be more than $83.1 billion in the year 2000, associated with depression in adults[1,8].

The two most common methods currently used to pool data in a meta-analysis are the FE model and the RE model. A FE model assumes that all studies share the same common ES and that any differences in effects are limited to sampling error. Consequently, study weights are limited to controlling for within-study error with no attention paid to between-study error. The FE model is usually appropriate if the studies included in the meta-analysis are identical and if the purpose is to calculate a common ES for the included studies and not generalize to other populations[9]. However, rarely are the studies included in a meta-analysis identical. In addition, one is usually interested in generalizing the findings to other populations. In contrast to the FE model, the RE model assumes that the true ES varies from study to study and they represent a random sample of true effects. Consequently, in a RE model, study weights are assigned to try and minimize both within and between-study variance[9]. Given the fact that RE models try to account for both within and between study variance, they are usually the preferred model to use. However, a major limitation of the RE model is the unrealistic assumption that each ES from the included studies is a random-sample of true effects. To try and address the problems associated with fixed and RE models, Bonett has recently proposed the VC model used in the current report[3]. As previously described, this model does not assume a common ES or random sampling of study populations and has been shown to have superior coverage probabilities to the fixed and RE models[3]. Importantly, simulation results suggest that confidence intervals based on the FE model are too narrow while confidence intervals based on the RE model are too wide[3].

While the results for both RE and VC models in the current study were similar, the use of the VC model may be preferable given the greater coverage probabilities reported as well as the fact that the assumptions underlying the VC model may be more realistic in the meta-analytic setting[3]. For example, as previously mentioned, no assumptions with respect to a common ES or random sampling from a normally distributed population of studies are made with the VC model[3]. Thus, the larger ES and narrower confidence intervals from the VC model in this brief report may be a more accurate reflection of the effects of exercise in the treatment of depression among adults. Given the former, it is recommended that the VC model be considered when conducting meta-analytic research on depression and other mental health conditions. However, regardless of the model chosen, it’s important to understand that the external validity of a meta-analysis depends not only on the statistical models used but also on the quality and representativeness of the individual studies included[7], factors not considered in this brief report. In conclusion, using the apparently more valid VC model, this brief report confirms the results of a previous RE meta-analysis with respect to exercise reducing depression in men and women diagnosed with depression[2].

COMMENTS

Background

Meta-analysis is a quantitative approach for synthesizing information from multiple studies and for reaching general conclusions regarding a body of research. The use of accurate statistical models for reaching such conclusions is important.

Research frontiers

A major area of current research with respect to meta-analytic methods is the development of more accurate methods for pooling results for meta-analysis. Currently, the random-effects (RE) model is the most preferred approach for pooling meta-analytic results. However, the RE model is based on the unrealistic assumption that results from the included studies are a random-sample of true effects.

Innovations and breakthroughs

Recently, the varying coefficient (VC) model has been proposed as an alternative model for meta-analysis. This model does not assume a common effect size or random sampling of study populations. It has been shown to perform better than both fixed and RE approaches. In this study, the authors compared the results of the VC model to the RE model using data from a meta-analysis on exercise and depression in adults, a major public health problem. While both models confirmed that exercise reduced depression in adults, the VC yielded a larger overall effect size and narrower confidence intervals.

Applications

The results of this study confirmed that exercise reduced depression in adults and that the VC may be preferred over the RE model.

Peer review

The results of the study confirm what it is already known with a new model for meta-analysis. The manuscript is well presented and easy to read. The title and the abstract reflect the topic and contents of the study. Methods are well presented and results provide sufficient experimental evidence to draw scientific conclusions.