Integrating contextual variables in meta-analyses

Abstract

Meta-analysis is an important statistical tool, and it is often used to solve clinical problems. However inevitably when conducting a meta-analysis, the included studies often have heterogeneity. This paper suggests the inclusion of relevant background data or contextual variables into the model. The contextual variables are those variables not explicitly measured in the studies included in a meta-analysis; thus, these must be very well-described and justified as parameters for analyses.

Key Words: Covariates; Moderator; Meta-analysis; Contextual; Subgroup analysis; Meta regression

Core Tip: This letter call for the use of contextual variables, that are typically not in use for covariate analyses. Contextual variables are introduced and defined as variables not immediately/directly measured by the original studies in the meta-analysis but rather can be estimated knowing the background of each study. For example in a meta analysis of clinical trails one might want to adjust for studies from high income vs low income countries or studies that were funded vs independent.

TO THE EDITOR

Meta-analysis is a statistical procedure for combining data from several studies about a specific problem[1]. When performing meta-analyses, it is important to determine the level of homogeneity between studies[2]. When effect size is consistent between the studies, meta-analysis approach can be used to identify this common effect by combining all of the studies[1,2]. However, problems are not always simple and often researchers face the issue of heterogeneity in meta-analysis and therefore employ moderator’s analysis techniques. A moderator is an independent variable that conditions the relations between two others variables[1,3]. Because effect sizes are the relationship between two variables, any variable that predicts the effect sizes is a moderator. When performing a moderator analysis, the moderator data type determines the type of analysis used. For categorical moderators (e.g., males vs females) usually a subgroup analysis is performed. For continuous moderators (e.g., age) usually a meta-regression is performed, it must be note here that continuous moderators can be either integer or decimal[1,3]. Meta-analysis researchers often extract these moderator variables from the studies being reviewed. Available moderator variables might be limited to basic demographics or irrelevant to the core analysis at hand.

The purpose of this paper is to define and describe “contextual variables” and provide reasons for integrating them in meta-analysis studies. Brief section describes a worked example of contextual variables in a moderator analysis. The strengths and limitations of incorporating a contextual variable in moderator analyses are discussed as well as approaches to routinely define and include these variables in future meta-analyses.

A contextual variable is defined as a variable that was not explicitly measured in the studies included in a meta-analysis but rather is inferred or computed for the included studies. Contextual variables are variables that characterize the study setting, methodology or conduct. By adding a contextual variable meta-analysis researcher are encouraged to incorporate other relevant background data into the model. This extends meta-regression to a higher level. These variables would include things like the population sampled, the response rate, the geographical setting, the design, sponsorship etc. An example of contextual variable is to include an analysis of therapeutic trials by source of funding (industry, independent, mixed) which has repeatedly shown the presence of publication bias.

To illustrate this, a recent meta-analysis of ours was undertaken to examine the effect of Ramadan intermitting fasting on metabolic syndrome components[4]. The MetS components analyzed were: Waist circumference (WC), systolic blood pressure (SBP), fasting glucose (FG), triglycerides (TG), and high-density lipoprotein (HDL) cholesterol. We identified 85 studies (4326 participants in total) that were conducted in 23 countries between 1982 and 2019. For illustration purposes, we will use WC. Ramadan fasting induced effect sizes for WC was small (K = 24, N = 1557) Hedge’s g = -0.312, 95%CI: -0.387 to -0.236). To better understand the effects of Ramadan fasting on WC we performed moderator analysis for age and sex (proportion of male subjects) in the form of meta-regression. These simple demographic data (age and sex) were available for all of the included 24 studies. Meta-regression results revealed that sex was significant in explaining variation in WC (β = -0.20, P = 0.03), but age was not significant moderator.

As researchers we were also interested in examining the effect of fasting time/day in explaining changes in WC. Unfortunately, the included studies did not include this variable in their results section. However, by having some information about the study e.g. location and year of data collection we were able to use compute the contextual variable of fasting time/day (in minutes) which was defined according to Ramadan as time between sunrise and sunset, see https://www.sunrise-and-sunset.com/. Other plausible examples, would be metrological data such as mean temperature or relative humidity which can be obtained easily from historical weather platform available online.

Based on our experience of including contextual variables, this paper proposes the following three conditions to be included in models. First, ensuring that the data available are obtained during the same period of the original included study time. For example, for Ramadan fasting study that was included from Bahrain, 2000 it was crucial that sunrise and sunset were calculated for the same timeframe. Second, Gather data for all of the included studies from the same reliable/trustworthy source. For example, the calculated data on daytime length for a study from Bahrain, 2000 and study from the United Kingdom, 2018 are obtained from https://www.sunrise-and-sunset.com. Third, authors need to give detailed descriptions on how contextual variables were computed, summarized and analyzed.

The inclusion of contextual has high potentials in improving the methodology of meta-analysis and the benefits of including contextual variables can be summarized as follow: (1) Increase generalisability of results, as it would be important to demonstrate that effect size is consistent among variables of interest; (2) Facilitate the identification of iatrogenic effects in subgroups; (3) Modelling improvement, by including meaningful variables that would give scientifically interpretable results; and (4) Have practical and research implications by generating future hypotheses for practice or further studies.

There are few limitations of including contextual variables which include the following: (1) Instability of models due to collinearity e.g., fasting time and country of origin results might yield same results; (2) Difficulty in obtaining some variables with high level of precision; and (3) Interpreting the results might be challenging.

In conclusion, this paper propose that contextual variables are encouraged in meta-analyses to further understand and interpret the data and enrich study findings and inform future studies. These variables must meet a minimum set of conditions before inclusion.