Google Consumer Surveys

Obviously, survey research is a big part of our work. Recently, various ways of conducting internet surveys have been discussed as a potential way of generating representative survey data at a fraction of the cost of face-to-face probability surveys. The issues in using internet samples with respect to sample selection, mode effects, respondent privacy and many others have been discussed at length in many forums over the last few years. One potential addition to this debate is the emergence of Google surveys (h/t Michael). This is a tool by Google that allows for the creation of surveys that can be hosted on various websites with a pay-per-complete model that makes it worthwhile for different websites to host them. It will be interesting to see whether this new tool finds academic applications. Clearly, a lot of academics are using various different methods of sourcing on-line respondents, including Amazon M-Turk and various access panels. Whether this new tool can offer the type of programming flexibility to design experiments and truly deliver a sample with known properties is a very interesting question (see paper below from google). In general, comparison of the results of important studies across these various methods of recruiting respondents is one of the most important practical issues for researchers that need this type of data to develop their research. For me, I am still a big fan of the approach whereby large representative samples are recruited from known sampling frames and incentivised to take part and then repeatedly surveyed (the US ALP panel being a brilliant example). But it would be very unwise to  ignore the technologies that are emerging particular as internet usage becomes universal in industrialised/post-industrial countries.



Comparing Google Consumer Surveys to Existing Probability and Non-Probability Based Internet Surveys
Paul McDonald, Matt Mohebbi, Brett Slatkin
Google Inc.



Abstract
This study compares the responses of a probability based Internet panel, a non-probability based Internet panel and Google Consumer Surveys against several media consumption and health benchmarks. The Consumer Surveys results were found to be more accurate than both the probability and non-probability based Internet panels in three separate measures: average absolute error (distance from the benchmark), largest absolute error, and percent of responses within 3.5 percentage points of the benchmark. These results suggest that despite differences in survey methodology, Consumer Surveys can be used in place of more traditional Internet based panels without sacrificing accuracy.