Improving the conduct of meta-analyses of observational studies

Abstract

The author, who has published numerous meta-analyses of epidemiological studies, particularly on tobacco, comments on various aspects of their content. While such meta-analyses, even when well conducted, are more difficult to draw inferences from than are meta-analyses of clinical trials, they allow greater insight into an association than do simple qualitative reviews. This editorial starts with a discussion of some problems relating to hypothesis definition. These include the definition of the outcome, the exposure and the population to be considered, as well as the study inclusion and exclusion criteria. Under literature searching, the author argues against restriction to studies published in peer-reviewed journals, emphasising the fact that relevant data may be available from other sources. Problems of identifying studies and double counting are discussed, as are various issues in regard to data entry. The need to check published effect estimates is emphasised, and techniques to calculate estimates from material provided in the source publication are described. Once the data have been collected and an overall effect estimate obtained, tests for heterogeneity should be conducted in relation to different study characteristics. Though some meta-analysts recommend classifying studies by an overall index of study quality, the author prefers to separately investigate heterogeneity by those factors which contribute to the assessment of quality. Reasons why an association may not actually reflect a true causal relationship are also discussed, with the editorial describing techniques for investigating the relevance of confounding, and referring to problems resulting from misclassification of key variables. Misclassification of disease, exposure and confounding variables can all produce a spurious association, as can misclassification of the variable used to determine whether an individual can enter the study, and the author points to techniques to adjust for this. Issues relating to publication bias and the interpretation of “statistically significant” results are also discussed. The editorial should give the reader insight into the difficulties of producing a good meta-analysis.

Keywords: Hypothesis definition; Literature searching; Heterogeneity; Publication bias; Misclassification; Confounding; Meta-analysis

Core tip: The author has published many meta-analyses of epidemiological studies, particularly on smoking, and the editorial comments on various aspects of their conduct. Areas covered include the definition of the hypothesis to be tested, literature searching and data entry, as well as methods to test for heterogeneity and investigate such issues as confounding, misclassification and publication bias. The need for well conducted meta-analyses and the difficulty in determining whether a “statistically significant” association is actually indicative of a causal relationship are discussed. The editorial should be helpful to readers inexperienced with the conduct of meta-analyses.