A Delphi Study: Mapping Weighting Techniques in Non-probability Web Surveys

Abstract

Weighting techniques are applied in non-probability web surveys to compensate for biases from self-selection, undercoverage, and nonresponse; however, there is incomplete knowledge of what techniques are being used in practice and how they are being applied as well as what variables are being applied as auxiliary variables in the weighting adjustments. The objective of this study is to investigate these main ideas. We report findings from a Delphi study. In the study, 38 survey experts addressed weighting techniques and auxiliary variables in non-probability web surveys. Results indicated that the majority of experts who analyze non-probability web surveys generally correct for all of the biases at the same time and typically apply calibration and PSA weighting techniques. These experts were significantly more satisfied with the successfulness of their adjustments so survey practitioners should consider these techniques in their analyses of web surveys. Although the experts believe that a combination of sociodemographic and webographic variables are the most successful auxiliary variables to employ in the adjustments, they typically employ only sociodemographic variables due to practical reasons. This indicates that survey practicioners should add webographic variables to their surveys so that they are available for post-survey adjustments. In addition, there were no significant differences amongst the experts on key variables in terms of job sector and geographical location indicating that there is an even spread of opinions and practices amongst science, market research, and government as well as Europe and North America.