Introducing Summative Progress Testing in Radiology Residency: Little Change in Residents’ Test Results After Transitioning from Formative Progress Testing

Abstract

Introduction: Educational effects of transitioning from formative to summative progress testing are unclear. Our purpose was to investigate whether such transitioning in radiology residency is associated with a change in progress test results. Methods: We investigated a national cohort of radiology residents (N > 300) who were semi-annually assessed through a mandatory progress test. Until 2014, this test was purely formative for all residents, but in 2014/2015, it was transitioned (as part of a national radiology residency program revision) to include a summative pass requirement for new residents. In 7 posttransitioning tests in 2015–2019, including summatively and formatively tested residents who followed the revised and pre-transitioning residency program, respectively, we assessed residents’ relative test scores and percentage of residents that reached pass standards. Results: Due to our educational setting, most posttransitioning tests had no residents in the summative condition in postgraduate year 4–5, nor residents in the formative condition in year 0.5–2. Across the 7 tests, relative test scores in postgraduate year 1–3 of the summative resident group and year 3.5–4.5 of the formative group differed significantly (p < 0.01 and p < 0.05, respectively, Kruskal-Wallis test). However, scores fluctuated without consistent time trends and without consistent differences between both resident groups. Percentage of residents reaching the pass standard did not differ significantly across tests or between groups. Discussion: Transitioning from formative to summative progress testing was associated with overall steady test results of the whole resident group in 4 post-transitioning years. We do not exclude that transitioning may have positive educational effects for resident subgroups.