The Effect of Seriousness and Device Use on Data Quality

Abstract

Nonserious, inattentive, or careless respondents pose a threat to the validity of self-report research. The current study uses data from the Growth from Knowledge Online Panel in which respondents are representative of the Dutch population in education, gender, and age over 15 years (N = 5,077). By doing regression analyses, we investigated whether self-reported seriousness and motivation are predictive of data quality, as measured using multiple indicators (i.e., nonsubstantial values, speeding, internal data consistency, nondifferentiation, response effects). Device group and demographic characteristics (i.e., education, gender, age) were also included in these analyses to see whether they predict data quality. Moreover, it was examined whether self-reported seriousness differed by device group and demographic characteristics. The results show that self-reported seriousness and motivation significantly predict multiple data quality indicators. Data quality seems similar for different device users, although smartphone users showed less speeding. Demographic characteristics explain little of the variance in data quality. Of those, education seems to be the most consistent predictor of data quality, where lower educated respondents show lower data quality. Effect sizes for all analyses were in the small to medium range. The present study shows that self-reported seriousness can be used in online attitude survey research to detect careless respondents. Future research should clarify the nature of this relationship, for example, regarding longer surveys and different wordings of seriousness checks