Assessing university policies for enhancing societal impact of academic research: A multicriteria mapping approach

Abstract

While there is a prolific debate on evaluating the societal impact of academic research, so far little attention has been paid to assessing the design and implementation of suitable organizational strategies. This article argues that evaluation methods are needed which are both formative and sensitive to diverging views on what defines and constitutes societal impact. We use a participatory deliberation method, the Multicriteria Mapping (MCM) approach, to examine how stakeholders appraise the use of university policy options for enhancing societal impact, and understand on what basis they judge the performance of these options. Focusing on a large Dutch research-based university, we conduct 22 interviews with academics, management, and support staff as well as strategic policy officers to examine how they rank and discuss the expected performance of university policy options identified in previous literature. Our results show that interviewees base their scores on criteria related to policy options’ expected organizational output and external outcomes, as well as their practical and, to a lesser degree, cultural, and financial feasibility. The resulting rankings also point at contrasts in the perceived potential of policy options, with interviewees assigning priority to providing researchers with recognition and rewards for impact-based activities. We conclude by discussing how MCM can be used as a formative evaluation method to assess and select policies and inform decision-making that fit a university’s particular situation. Besides drawing lessons for the context of our illustrative case, we also reflect on the relevance of the evaluation method and our findings for other universities.