Where No Reviewer Has Gone Before: Exploring the Potential of Mixed Studies Reviewing

Abstract

In mixed studies reviews, findings from quantitative and qualitative studies are synthesized in one review study. Methods for the inclusion of both types of evidence are available, but relatively little is known about the circumstances under which these methods are appropriate. Especially the role of qualitative evidence in a mixed studies review is topic of ongoing discussion among researchers. In this dissertation, we aim to further develop ways to include both quantitative and qualitative evidence in a mixed studies review. Two examples are used: Chapter 2 and 3 are based on two datasets (quantitative and qualitative) dataset concerning collaborative learning in primary and secondary education, Chapter 4 and 5 are based on two datasets concerning smoking cessation during pregnancy. In Chapter 1, we describe the development of review methodology for quantitative and qualitative reviews, and we explore the concept of the mixed studies review. We distinguish between mixed studies with a configurative aim and mixed studies reviews with an aggregative aim. In addition, we distinguish between reviews conducted in a segregated and an integrated design. In Chapter 2, we present a qualitative evidence synthesis on collaborative learning. This chapter examines the experiences of students on this topic. We show the type of output that can be generated and that can be used as input for a mixed studies review in a segregated design and a configurative aim. In Chapter 3, we proceed with the findings from the qualitative evidence synthesis reported in Chapter 2 by presenting a mixed studies review method in a segregated design. Chapter 3 describes an existing method called the matrix-approach which compares the findings from the qualitative evidence in Chapter 2 to the results from primary quantitative studies. We have developed a methodology to derive concrete moderation effects from a theoretical model from the qualitative evidence synthesis which broadens the applicability of the matrix-approach. In Chapter 4, we present a method to quantitize findings from qualitative studies in preparation for a mixed studies review in an integrated design. We quantitize in such a way that we have an estimated correlation and variance per study and a weighted overall correlation and variance for all included qualitative studies. This information may be used as input for a quantitative review. In Chapter 5, we suggest a method to include the quantized information from Chapter 4 using a Bayesian meta-analysis in an integrated design. We discuss the similarities between evidence from quantitative and qualitative studies and show how these can be matched using a Bayesian meta-regression model. In Chapter 6, we conclude that the two types of designs (segregated and integrated) can serve both types of review aims (configurative and aggregative). A recurring theme throughout the thesis is the unachieved potential of qualitative evidence for systematic reviewing. We conclude with the recommendations that the assumed distinction between quantitative and qualitative evidence in reviewing should be abandoned and that the type of model that is analyzed should be input for the chosen review method instead of the type of data.