Methodological challenges to control for immortal time bias in addressing drug effects in type 2 diabetes

Abstract

There are multiple biases in using observational studies to examine treatment effects such as those from prevalent drug users, immortal time and drug indications. We used renin angiotensin system (RAS) inhibitors and statins as reference drugs with proven efficacies in randomized clinical trials (RCTs) and examined their effectiveness in the prospective Hong Kong Diabetes Registry using adjustment methods proposed in the literature. Using time-dependent exposures to drug treatments yielded greatly inflated hazard ratios (HR) regarding the treatment effects of these drugs for cardiovascular disease (CVD) in type 2 diabetes. These errors were probably due to changing indications to use these drugs during follow up periods, especially at the time of drug commencement making time-dependent analysis extremely problematic. Using time-fixed analysis with exclusion of immortal time and adjustment for confounders at baseline and/or during follow-up periods, the HR of RAS inhibitors for CVD was comparable to that in RCT. The result supported the use of the Registry for performing pharmacoepidemiological analysis which revealed an attenuated low low-density lipoprotein cholesterol related cancer risk with RAS inhibitors. On the other hand, time-fixed analysis with including immortal time and adjustment for confounders at baseline and/or during follow-up periods, the HR of statins for CVD was similar to that in the RCT. Our results highlight the complexity and difficulty in removing these biases. We call for validations of the methods to cope with immortal time and drug use indications before applying them to particular research questions, so to avoid making erroneous conclusions.

Key Words: Pharmacoepidemiological analysis; Immortal time bias; Drug effects; Prevalent drug user bias; Drug indication bias; Type 2 diabetes

Core tip: There are multiple biases in using observational studies to examine treatment effects. These biases include those due to prevalent drug users, immortal time and drug indications that must be taken into consideration. In this regard, we used drugs with proven effects in randomized controlled trials and applied those proposed methods by other groups to estimate their effects in a prospective cohort of patients with type 2 diabetes. Our results highlighted the importance of validating adjustment methods for immortal time and drug use indications before applying them to addressing research questions, so to avoid making erroneous conclusions.

INTRODUCTION

In pharmacoepidemiological analysis, there are multiple biases in using observational studies to examine treatment effects. These biases may be due to prevalent drug users, immortal time and drug indications[1]. Given these biases, the question to ask is, do we have a way to judge whether the quality of the database of an observational study or the method of analysis is free from these three major biases?

The prevalent user bias is easy to discern and can be readily excluded during data analysis. In type 2 diabetes (T2D), biases due to drug indication depend upon whether the subphenotypes associated with the drug usage (indications) may have selected a patient subgroup inherently at high risk for a clinical outcome, e.g., cardiovascular disease (CVD) or cancer. Immortal time bias is not difficult to detect because we can suspect such a bias as long as immortal time is reported, i.e., non-drug exposure periods have been classified as exposure periods. Several methods have been proposed to control for immortal bias but it is uncertain whether these methods can adequately remove these biases[2-4]. Our recent work has shed light on these important issues[5,6].

INDICATIONS OF DRUG USE AND CONFOUNDERS

Hyperglycemia is the reason why a person is prescribed an antidiabetic regimen which can include various combinations of oral antidiabetic drugs (OAD), insulin and other injectables, such as glucagon like peptide 1[7]. Besides, many factors such as disease severity, predominant disease mechanisms (e.g., insulin deficiency versus insulin resistance), prescribing habits, formulary restrictions, willingness to pay for or accept treatment, referral and volunteer biases can also affect the choices of drug combinations as first, second or third lines of treatments during the clinical course. Of note, some of these factors which can influence drug choices may not be captured in observational studies. Hence, randomized clinical trials (RCT) remain to be the gold standard by evenly distributing these unmeasured confounders in different experimental and control groups to reduce these biases.

Several studies including ours have reported an association of hyperglycemia with cancer in diabetes[8,9]. Our group also reported a linear association between glycated hemoglobin (HbA_1c) and all-site cancer in T2D with 1% increase in HbA_1c associated with 18% increase in the risk of cancer[10]. These observations were supported by a meta-analysis of RCT data where 0.5% reduction in HbA_1c was associated with a non-significant hazard ratio (HR) of 0.91 for cancer risk in T2D[11]. In a recent large randomized trial[12], treatment with saxagliptin, a dipeptidyl peptidase 4, was associated with 0.3% reduction in HbA_1c accompanied by a 50% reduction in the risk of pancreatic cancer, albeit short of significance[12]. Although the underlying mechanism linking hyperglycemia and cancer remains to be elucidated, the overarching premise is that users of OADs and insulin are high risk subjects for cancer. Unless these drug indications are captured and removed, these drug users are likely to be found to increase cancer risk, which might be erroneously attributed to drug effects.

In epidemiological analysis, propensity score is often used to control for indications of drug use[13]. The robustness of these scores in removing selection bias is indicated by the area under receiver’s operating characteristics curve (AUC) where values ≥ 0.90, ≥ 0.80 to < 0.90, and ≥ 0.70 to < 0.80 indicate excellent, good and fair performance, respectively[1]. Apart from including propensity score, multivariable analysis with inclusion of subphenotypes associated with a clinical event, e.g., cancer, can also attenuate bias due to drug indications[1]. In prospective cohort analysis of the Hong Kong Diabetes Registry, we had identified a group of subphenotypes for cancer risk in T2D[14], in addition to age and hyperglycemia. These included (1) body mass index ≥ 27.6 kg/m² and < 24 kg/m^2[15]; (2) low-density lipoprotein cholesterol (LDL-C) ≥ 3.8 mmol/L[16]; (3) co-presence of LDL-C < 2.8 mmol/L and albuminuria[17] which was further enhanced in the presence of increased high density lipoprotein cholesterol (HDL-C) ≥ 1.0 mmol/L[18]; (4) co-presence of LDL-C < 2.8 mmol/L and triglyceride < 1.7 mmol/L[18]; and (5) HDL-C < 1.0 mmol/L[19].

IMMORTAL TIME AND IMMORTAL TIME BIAS

Immortal time refers to a period in cohort studies when non-exposure to a drug treatment from the baseline to the time of initiation of the drug treatment in the “drug exposure group” is misclassified as exposure to the drug treatment[1]. This misclassification may lead to a deflated HR of the treatment for the endpoint due to addition of the non-drug exposure period into the drug exposure period. This can lead to a false conclusion that the drug reduces the risk of the event of interest. Several researchers recommended the use of time-varying or time-dependent drug exposure Cox proportional hazard regression to cope with immortal time bias[3,4]. The use of this method assumes that exposure to the drug treatment or the drug commencement is at random[20]. However, this is rarely the case in real world practice since patients usually start on a drug treatment for a new or changed indication which may not be systematically captured in analysis of cohort study data. If these confounders are not available, the use of a time-dependent model may lead to an increased HR of the treatment for the endpoint (Figure 1).

Figure 1 Schematic diagram to explain the multiple biases in epidemiological analysis of cardiovascular disease with use of statins in observational cohorts. While baseline variables may be used to derive a propensity score to adjust for drug indications, this will not apply to prevalent drug users whose risk profile has already been influenced by drug use (blue panel). For patients who were started on the drug after a period of immortal time (non-drug exposure), insufficient data capturing during the observation period may not allow full adjustment for confounders resulting in an inflated hazard ratio of cardiovascular disease, while inclusion of immortal time in a time-dependent Cox model may lead to a reduced hazard ratio by including a period of non-drug exposure during estimation of event rates (red panel). In patients with detailed documentation of risk factors followed by drug commencement as in a clinical trial setting, time fixed analysis may yield hazards unbiased by immortal time (green panel). ECG: Electrocardiogram.

In order to test the validity of these methods[3,4], we used a referent drug with proven benefits [e.g., statin or renin angiotensin system (RAS) inhibitors] and applied the methods to the Hong Kong Diabetes Registry[5,6] to find out if the estimated effect size fell within the bounds of that reported in RCTs with regard to their associations with CVD. We tested various combinations of exclusion/inclusion of immortal time and adjustment for drug indications at baseline or at the end of immortal time when drug was commenced Consistently, time-dependent drug-exposure Cox models severely inflated the HR of these two drugs for CVD risk[5,6], despite their proven cardioprotective effects in RCTs. In the statin-CVD validation[6], compared to a HR of 0.63 (95%CI: 0.48 to 0.83) in a RCT[21], exclusion of immortal time and adjustment for estimated covariables at the end of the immortal time when statins were commenced, resulted in a 52.3% inflation in the HR of statins for CVD (0.96, 0.72 to 1.27), which was above the higher bound of the 95%CI. On the other hand, inclusion of immortal time, i.e., ignoring immortal time bias, and adjustment for covariables at baseline generated the least inflated HR of 0.64 (0.48 to 0.84) which was within the HR estimates in clinical trial and inflated by 1.59% compared to the absolute HR of 0.63.

In the RAS inhibitors-CVD validation[5], exclusion of immortal time and adjustment for covariables when RAS inhibitors was commenced resulted in a HR of 0.89 (0.68 to 1.17) which was within the estimates of 0.92 (0.84-1.0) reported in RCTs with 3% deflated risk compared to the absolute hazard. By contrast, inclusion of immortal time and adjustment for covariables at baseline yielded a HR of 0.66 (0.51 to 0.86) which was outside the estimates with a 28.3% deflation rate (or inflation rate: -28.3%) compared to the absolute HR.

In most observational or administrative databases, the events preceding the commencement of drugs like statins (e.g., high LDL-C, angina, abnormal imaging) were often not available in the dataset. In a time-dependent model which includes immortal time, inadequate adjustment for indications at the time of drug commencement can lead to overinflated hazards. In this situation, a non-time-dependent analysis but ignoring immortal time and adjusting for propensity score using covariables at baseline might yield the least bias. It is also possible that inflated hazards due to inadequate removal of drug indications and reduced hazards by including the immortal time might have cancelled out one another, giving a HR close to that in a RCT. In the case of drugs with more general indications such as RAS inhibitors, a time-fixed Cox model with exclusion of immortal time and adjustment of covariables at the end of the immortal time, estimated from the baseline variables, might remove most, if not all, of these biases.

Our results highlight the challenges in removing bias from drug indications and immortal time simultaneously if these biases have not been systematically captured. In this new era of big data, clearly, more research is needed to develop methods for removing immortal time bias. This is especially relevant to drugs such as statins and insulin, often prescribed for clinical conditions (e.g., angina, poor glycemic control), the information of which may not be documented in the database. Pending better methodologies, we recommend the use of non-time-dependent model with exclusion of immortal time and adjustment for propensity score or subphenotypes associated with the event of interest to reduce potential biases.

On the other hand, by selecting high quality datasets with documentation of drug usage and prognostic variables, pharmacoepidemiological analysis may uncover novel hypothesis for further testing. In an analysis of the diabetes-cancer link, in light of the phenotypic heterogeneity, we first used multivariable analysis to identify risk factors or subphenotypes associated with cancer. By adjusting for a low LDL-C related cancer-subphenotype at drug commencement, we discovered a novel drug-subphenotype interaction where RAS inhibitors specifically attenuated low LDL-C related cancer risk in T2D[22]. These pharmacoepidemiological findings, coupled with pathophysiological knowledge and evidence from mechanistic investigations, have provided the basis for a hypothesis where the complex cross-talk between the RAS and the insulin-like growth factor 1-cholesterol pathway might explain the diabetes-cancer link, for further testing[14].

CONCLUSION

In pharmacoepidemiological analysis, there are methodological challenges in removing biases from immortal time and drug indications simultaneously. Hence, risk associations between drug use and clinical events based on observational studies must be interpreted with great caution. To avoid misinterpretation, researchers should take these biases into consideration at the stage of study design, e.g., by documenting indications or variables at the time when drugs are introduced or changed. Our validation studies indicated that exclusion of immortal time in an analysis testing effects of RAS inhibitors while inclusion of immortal time in an analysis testing effects of statins on CVD, respectively yielded effect sizes in T2D close to those obtained in RCTs. Our findings call for further research in developing methodology to simultaneously remove immortal time bias and drug use indication bias. Meanwhile, in the absence of methods which can address effects of different drugs in multiple databases, it will be prudent to use reference drugs and test the quality of databases and adjustment methods for immortal time and drug indications before testing of other drug associations with clinical outcomes to avoid erroneous conclusions.