Validity and reliability of the (adjusted) Impact on Participation and Autonomy questionnaire for social-support populations

Abstract

Background Decentralisation of social support and budget cuts spurred interest in outcome-oriented payment systems in the Netherlands. Hence, measurement of relevant outcomes, such as self-reliance and participation, is needed. The Impact on Participation and Autonomy (IPA) questionnaire for rehabilitation clients was adapted for social support, called IPA-MO, and its validity and reliability were tested among social support clients in eight municipalities in 2014 and 2016. Methods The total research sample comprised of 4120 persons. Homogeneous subgroups were distinguished based on 1) disability and 2) age. Exploratory factor analysis (PCF) was used to identify domains for IPA-MO. Model fit was assessed with Confirmatory Factor Analysis (CFA) using structural equation modelling. Results PCF revealed that the IPA-MO model consists of all five original IPA domains: Autonomy indoors, Family role, Autonomy outdoors, Social life and relations, and Work and education. As a result of new items added, a new, sixth domain was developed: Financial autonomy. Due to high non-response on Work & education, construct validity was first tested for a five-domain IPA-MO model. The composition of the IPA-MO domains showed slight differences: the item on ‘mobility indoor’ shifted from Autonomy indoors (IPA) to Family Role in IPA-MO. The item on reciprocity shifted from Social Life and relations (IPA) to Autonomy outdoors (IPA-MO). Internal reliability was confirmed for all domains (Cronbach’s alpha >.80). CFA showed acceptable construct validity of the five-domain IPA-MO model for the social support population (CFI .936, TLI .925, SRMR .051), all age groups and most disability-based groups. Construct validity including Work & education was tested for 234 participants. Then, PCF revealed six domains and the model fit was acceptable (CFI .915, TLI .903, SRMR .067). Conclusions IPA-MO is a valid and reliable instrument to assess outcomes of social support. Further research on the domain Financial autonomy is needed. Social-support clients are numerous and dispersed and, in spite of the best intentions, hard to involve widely in policy processes. A valid outcome measure offers municipalities possibilities to gain insight in social costs and benefits of new policies. Outcome measurement also allows to contract bundled-services of providers, thereby changing the incentives for providers from increasing production to much needed innovation. Taking the perceptions of autonomy and participation of social-support clients as the acid test for the effectiveness of social support policies, may prove a serious game-changer in politics.