Inside Out: Program Integrity and Effectiveness of the Cognitive-Behavioural Program EQUIP for Incarcerated Youth

Abstract

In correctional facilities intervention programs are used to reduce behavioral problems and recidivism. Intervention programs can be effective when they contain effective ingredients and are implemented with high levels of program integrity. Program integrity is the degree to which programs are implemented as designed and program integrity is acknowledged as an important factor influencing the effectiveness of interventions. Nevertheless, still many intervention studies do not provide information on the program integrity of the intervention. When no information on program integrity is provided it is difficult to explain positive, negative, or absent intervention effects. For example, can absent intervention effects be explained by poor program implementation or does the intervention not contain the effective ingredients? Although correctional treatment researchers have written extensively about the importance of program integrity, studies on the effectiveness of correctional treatment that include measures of integrity are almost non-existent. In this dissertation we aim to contribute to closing the program integrity gap by investigating the program integrity and effectiveness of EQUIP for incarcerated youth. The newly designed Measurement Instrument Program Integrity EQUIP is a multifaceted instrument that includes the program integrity elements ‘exposure’, ‘adherence’, ‘participant responsiveness’, and ‘quality of delivery’. The instrument showed good psychometric quality, in terms of construct validity, internal consistency, inter-observer agreement, and convergent validity. Further, we demonstrated that the EQUIP program had been implemented with low to moderate levels of program integrity in juvenile correction facilities in The Netherlands and that EQUIP was implemented with higher levels of integrity in the United States. In The Netherlands, the average composite program integrity score was 55%, ranging from 35% to 64%. With the low to moderate levels of program integrity the EQUIP program did not show the expected effects. Both the EQUIP and the control group remained stable on cognitive distortions and moral judgment and the groups did not differ on recidivism outcomes. However, the EQUIP group did remain stable in social skills and moral values, whereas the control group showed a small decrease in social skills and moral values. Within the low to moderate program integrity range, EQUIP was not more effective on cognitive distortions, social skills, moral development, and recidivism when implemented with higher – thus moderate instead of lower– levels of integrity. Next, we implemented a multi-actor multi-method “program integrity booster” in the participating correctional facilities in The Netherlands with the aim to improve program integrity and, subsequently, program effectiveness. The program integrity booster resulted in small improvements in program integrity, but these integrity improvements did not lead to improved effectiveness of EQUIP on cognitive distortions, social skills, moral development. Finally, in a novel meta-analysis we zoomed in on cognitive distortions. In a set of 53 studies we found a strong association between cognitive distortions and externalizing problem behavior. In addition, in 18 intervention studies we found that interventions can effectively reduce cognitive distortions. In a subset of 9 intervention studies, assessing both cognitive distortions and externalizing problem behavior, neither reductions in cognitive distortions nor reductions in externalizing problem behavior were established.