Comparison of interaction and subgroup effects in observational studies and randomized clinical trials: Three empirical examples

Abstract

Background: Compared to RCTs, observational (nonrandomized) studies often comprise larger sample sizes, which gives adequate power to study interaction. Observational studies, however, are prone to confounding. Objectives: To determine the validity of subgroup and interaction effects (differences between subgroups) for different study designs. Methods: We compared effects of medical interventions based on observational studies, RCTs, and Individual Patient Data Meta-Analysis of RCTs (IPDMAs; reference) on three different clinical topics: (1) mammography screening effects on breast cancer mortality; (2) CABG and all-cause mortality; (3) statins and the incidence of major coronary events. Main, subgroup, and interaction effects were compared. Results: Main and subgroup effects were comparable with respect to the direction of effects for IPDMAs, RCTs, and observational studies. Small differences in the magnitude of subgroup effects in observational studies yielded different interactions compared to IPDMA. In the mammography example the Ratio of Risk Ratio's (RRR) (i.e., interaction effect) among observational studies was 1.46 (95% CI 1.09;1.96) compared to an IPDMA effect of 110 (95% CI 0.89;1.37). For the CABG studies the observational RRR was 1.03 (95% CI 0.84;1.26), whereas in the IPDMA this was 1.40 (95% CI 1.08;11.81). Finally, in the statin example the RRR was 1.35 (95% CI 113;1.61) for observational studies, in the IPDMA this was 0.90 (95% CI 0.84;0.97). Conclusions: Main and subgroup effects based on observational data are in line with main and subgroup effects in IPDMAs based on RCTs, yet interactions may differ substantially.