Thanks to technical advances in recent years, a new methodology for research has evolved – population-based survey experiments. It is a combination of surveys, typically done by phone or the internet, with experiments. The book provides a first introduction to this new method. A drawback of typical experiments held in labs with undergraduate students is the lack of external validity and a small sample size. Certain experiments can be held via the internet and telephone which enables scholars to obtain a large sample size at a modest cost. A drawback of a survey tool is that it cannot measure cause and effect in a rigorous way. That is why combining the two methods provides new opportunities.
Mutz describes various case studies across the social sciences where population-based survey experiments have been applied. Mutz was the principal investigator of TESS (Time-Sharing Experiments for the Social Science; http://www.tessexperiments.org/index.html) on which the examples are based. Most of these examples are focused on measuring opinions and preferences, and how these relate to certain population groups. For example, when assessing attitudes towards race one can ask each participant how many of three statements theoretically unrelated to attitudes on race upset the participant (e.g., the federal government increasing the tax on gasoline), and do the same for another group of participants who get four statements including one that is linked with racial dynamics such as “a black family moving in next door”. Since the sample groups have been generated randomly, a difference between the three and four statement treatments gives an estimate on attitudes towards race. Furthermore, one can get information on differences of effects for different demographic groups.
In implementing population-based survey experiments one needs to use large randomized samples. Since the participants will come from a broad background the tasks need to be simple, clear and brief, otherwise the participants will drop out of the experiment.
Some population-based survey experiments have been performed on collective action problems, such as dictator games, and on the framing effects on donations to charity. Most examples focus on measurement of various attributes of a population, such as preferences, attitudes, opinions, etc. It is up to the commons community to explore the potential of this method for commons dilemma experiments using a larger and more diverse pool of subjects than most lab experiments focus on.
This book provides a good introduction to the field with many examples, a discussion on strengths and weaknesses, and practical information regarding how to plan and implement the experiments as well as analyze the data.