Web survey comparison study

In 2011, the Wellcome Trust provided funding for additional methodological work to develop a shorter version of the Natsal-3 questionnaire to be administered as a web survey. Three well-known market research organisations in the UK were commissioned to carry out four surveys among their existing web panel members, and their results were compared with Natsal-3 and, for a few socio-demographic variables, with external benchmarks. Two of the web surveys set quotas on 'basic' variables (sex, age, and partnership status), while the other two were 'modified' to set quotas on additional variables known to be related to some of the key behavioural measures.  These surveys were restricted to panel members aged 18-44 years. The main objectives of this work were:

a) to examine the extent of agreement between results from the surveys carried out among volunteer web panel members with each other and with those obtained from the Natsal-3 CAPI/CASI probability sample survey and

b) to determine whether modifying the quota controls would lead to web panel results more similar to those found on Natsal-3.

Another component of the work involved following-up a sub-set of participants in the main Natsal-3 interview survey. About one to two months after their CAPI/CASI interview, 527 Natsal-3 participants completed the web version of the questionnaire. This version of the questionnaire can be found here. The aim of this part of the project was to give an indication of the willingness of participants to complete the survey on the web and to provide direct comparison of responses to the same questions asked in CAPI/CASI and on the web.

A final part of the project involved one of the market research organisations obtaining samples of 500 participants of two minority groups (gay men and Black Africans) who are represented in small numbers even in large surveys like Natsal-3, and are difficult (and costly) to 'boost' using probability sampling methods.

Results from this work package will be published in a number of papers in 2014.