Addressing Missing Data of Dichotomous Outcomes in Systematic Reviews

Abstract

Missing outcome data of trial participants is a frequent phenomenon in RCTs and may represent a serious potential source of bias if not reported and handled appropriately. The potential effect of bias associated with missing outcome data- attrition bias- is that invalid conclusions about efficacy and safety of studied interventions may be reached and ultimately impact clinical practice. The poor reporting and handling of missing outcome data in RCTs contribute to the inadequate reporting and handling of missing outcome data in systematic reviews. In chapter 2, we assessed how authors of 100 Cochrane and non-Cochrane systematic review authors report and address categories of participants with who might have missing outcome data. The methodological survey showed that most systematic reviews do not explicitly report sufficient information on categories of trial participants with potential missing outcome data or address missing outcome data in their primary analyses. Chapter 3 surveyed all RCT reports included in the 100 systematic reviews included in chapter 2 to describe how RCT authors (1) report on different categories of participants that might have missing outcome data, (2) handle these categories in the analysis, and (3) judge the risk of bias associated with missing outcome data. The survey showed that the majority of trials did not clearly report on whether different categories of participants that might have missing outcome data have been followed-up or not. The median percentage of participants who were explicitly not followed-up was 5.8% (IQR 2.2-14.8%). When one also includes participants with unclear follow-up status, the total value rises to 11.7% (IQR 5.6-23.7%). In addition, most trials did not specify how they handled missing outcome data in their analysis. Chapter 4 presented guidance for authors of systematic reviews on how to identify participants with missing outcome data in trial reports, especially when trial reporting is not clear. Our approach was based on how trial authors report on categories of participants who might have missing outcome data and how they handle them in their analyses. Chapter 5 explores the potential impact of missing outcome data on effect estimates of the 100 meta-analyses included in chapter 2. When applying plausible assumptions to the outcomes of participants with definite missing outcome data, up to a quarter of meta-analyses lost statistical significance. When applying implausible but commonly assumptions, the percentage of systematic reviews that lost significance was as high as 60% with the worst-case scenario. Chapter 6 showed that systematic review authors were inconsistent in their methods of handling missing data across their eligible primary trials. Moreover, most systematic review authors did not explicitly report their methods to handle missing data. Of the seven reviews that did explicitly report on their methods, none applied that method consistently across the included trials. Finally, in Chapter 7, we discussed the main findings of each chapter, discussed the strengths and limitations, and provided implications for practice for systematic review authors, for trialists, for journal editors, and for research.