Extending the within-persons experimental design: The multitrait-multierror (MTME) approach

Abstract

This chapter introduces a general framework that uses a within-persons experimental design but deals with two of the three problems with previous approaches: the assumption of only one error type and the assumption of zero test effects. The “multitrait-multierror” (MTME) framework does this by applying a simple idea: extend the within-persons design to vary several error sources at a time and randomize methodological variation such as question order. This design then enables researchers to concurrently estimate multiple sources of stochastic measurement errors from the Total Survey Error framework, allowing for the improvement of question design and removal of biasing effects from analyses. The chapter presents the MTME framework. It then gives practical advice on how to design and implement such experiments in surveys and on how to estimate the effects of the experimental treatments. The approach is illustrated using the Understanding Society Innovation Panel in the United Kingdom.