Improving survey fieldwork with paradata

Abstract

This thesis describes how paradata (like process data and interviewer observations) and other auxiliary information (like registry information) can be used to optimize survey design and survey management and to study survey errors and survey costs in person and household surveys. The following aspects are described: 1. Personalisation in advance letters. An experiment with personalised and non-personalised advance letters showed that groups of sample persons react differently to personalisation. This differential reaction may account for the mixed findings in literature. 2. Adaptive survey design. An experiment showed how using paradata and other auxiliary data can be used to adapt fieldwork to characteristics of the sample person. Differential treatments could consist of the mode used (web, paper or CATI), the timing and spacing of calls in CATI and the interviewers assigned to a certain sample person. The differential treatment resulted in more representative response, while maintaining response levels and costs. 3. Predicting contact and cooperation propensity. A large number of Statistics Netherlands Surveys with varying subjects and sampling designs was linked to a large number of auxiliary registry information. It was found that to some extent it can be predicted if sample persons will be contacted. If people would subsequently cooperate, however, cannot be predicted. Significant univariate relations, as described in literature, were replicated. However, multivariate analyses showed that hardly any variance in cooperation propensity could be explained with the available auxiliary variables. 4. Fieldwork behaviour and call scheduling. The implications of interviewer call scheduling, in terms of contact and cooperation rates was studied. Results showed that careful interviewer instruction as to timing and spacing of visits is important to optimize chance of contact in as little visits as possible. 5. Understanding factors leading to interviewers’ non-compliance with fieldwork rules. There may be several reasons why people do not comply to rules. For example, the rule may not be known, the likelihood of detection, or the severity of the sanction may be low, or the rule may not be compatible with personal circumstances. 14 possible influences on rule compliance were identified. These influences were used in a questionnaire to gauge their influence on interviewer behaviour. It was found that different influences played a role in different rules. Most influential appeared to be perceived legitimacy of the rule. 6. The interplay between interviewer behaviour, sample unit characteristics and nonresponse bias. In this chapter I study whether interviewers introduce bias by their behaviour, specifically by not following the field strategy they are supposed to follow. I studied whether interviewers deviate with certain kinds of sample units. Bias could be ascertained by comparing survey data with registry information. It was found that generally, interviewers work hardest for the most difficult cases. But some interviewers tend to shirk from difficulties, for example by not visiting during evenings, or not visiting the required number of times. These interviewers bias results, and they should be identified early in the fieldwork