Checkmate: Exploring Backward Induction among Chess Players

Levitt, Steven D., John A. List, and Sally E. Sadoff. 2011. "Checkmate: Exploring Backward Induction among Chess Players." American Economic Review, 101(2): 975–90.
DOI:10.1257/aer.101.2.975

Abstract
Although backward induction is a cornerstone of game theory, most laboratory experiments have found that agents are not able to successfully backward induct. We analyze the play of world-class chess players in the centipede game, which is ill-suited for testing backward induction, and in pure backward induction games—Race to 100 games. We find that chess players almost never play the backward induction equilibrium in the centipede game, but many properly backward induct in the Race to 100 games. We find no systematic within-subject relationship between choices in the centipede game and performance in pure backward induction games. (JEL C73)

Ungated Version

Gender Differences in Risk Behaviour: Does Nurture Matter?

Alison L. Booth, Patrick Nolen
Women and men may differ in their propensity to choose a risky outcome because of innate preferences or because pressure to conform to gender-stereotypes encourages girls and boys to modify their innate preferences. Single-sex environments are likely to modify students' risk-taking preferences in economically important ways. To test this, we designed a controlled experiment in which subjects were given an opportunity to choose a risky outcome - a real-stakes gamble with a higher expected monetary value than the alternative outcome with a certain payoff - and in which the sensitivity of observed risk choices to environmental factors could be explored. The results of our real-stakes gamble show that gender differences in preferences for risk-taking are indeed sensitive to whether the girl attends a single-sex or coed school. Girls from single-sex schools are as likely to choose the real-stakes gamble as boys from either co ed or single sex schools

http://d.repec.org/n?u=RePEc:esx:essedp:672&r=cbe

From Vox - John List on Field Experiments

John List being interviewed at the AEA meetings

http://www.voxeu.org/index.php?q=node/3522

Time-sharing Experiments for the Social Sciences

Time-sharing Experiments for the Social Sciences (TESS) is an NSF infrastructure project that offers researchers opportunities to test their experimental ideas on large, diverse, randomly-selected subject populations. Investigators submit proposals for experimental studies, and TESS fields selected proposals on a random sample of the United States population using the Internet.

Dan Hopkins and Gary King have successfully submitted an experiment: "Priming to Improve Survey Measurement through Anchoring Vignettes". Their experimental module employed vignettes placed before or after a self-assessment question to determine whether answering vignettes first improved respondents’ capacities to provide meaningful responses. They find that: 1) asking individuals to assess themselves immediately after hearing the vignettes produced responses that were more closely related to key covariates; and 2) inconsistent responses were more likely when individuals were asked to compare themselves directly to hypothetical individuals.

Liam mentioned the relevant paper on the blog before (here). Interestingly, TESS data will be made available to other investigators one year after they are first made available to the authors of successful proposals. The data from the King and Hopkins experiment (and accompanying documentation) is available for download (data in SPSS .sav format).

Geary researchers may also be interseted to know that Howard Schuman (University of Michigan) successfully submitted an experiment entitled: "Recall vs. Judgment: Open-Closed Question Differences in Studying Collective Memory".
Eric Oliver (University of Chicago) and Taeku Lee (UC Berkeley) successfully submitted an experiment entitled "Measuring Perceptions and Attitudes about Overweight and Obesity".

Deaton - Limits of Experiments

Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development
Follow the link through to Princeton Working Papers


Abstract
There is currently much debate about the effectiveness of foreign aid and about what kind of projects can engender economic development. There is skepticism about the ability of econometric analysis to resolve these issues, or of development agencies to learn from their own experience. In response, there is movement in development economics towards the use of randomized controlled trials (RCTs) to accumulate credible knowledge of what works, without over-reliance on questionable theory or statistical methods. When RCTs are not possible, this movement advocates quasi-randomization through instrumental variable (IV) techniques or natural experiments. I argue that many of these applications are unlikely to recover quantities that are useful for policy or understanding: two key issues are the misunderstanding of exogeneity, and the handling of heterogeneity. I illustrate from the literature on aid and growth. Actual randomization faces similar problems as quasi-randomization, notwithstanding rhetoric to the contrary. I argue that experiments have no special ability to produce more credible knowledge than other methods, and that actual experiments are frequently subject to practical problems that undermine any claims to statistical or epistemic superiority. I illustrate using prominent experiments in development. As with IV methods, RCT-based evaluation of projects is unlikely to lead to scientific progress in the understanding of economic development. I welcome recent trends in development experimentation away from the evaluation of projects and towards the evaluation of theoretical mechanisms.

Addendum: The Economic Logic blog is discussing this also.

Advertising Works - According to Yahoo! Research

We have discussed the possibility of evaluating adevertising campaigns on this blog before (here and here). So it is interesting to read that Yahoo! Research is measuring the effects of advertising on sales through a controlled experiment (more details available here). Researcher David Reiley has recently collaborated with Yahoo!’s Marketing Insights team in their ongoing efforts to help advertisers evaluate the effectiveness of their advertising campaigns.

Reiley joined Yahoo! Research because of his longstanding interest in field experiments and he points out a potential weakness of a study reported in a recent Harvard Business Review article, which measured large increases in sales due to online advertising. The study used large quantities of data from comScore, a key online information provider that logs the Internet browsing behaviour of two million users worldwide. By comparing the purchases of those who saw a given online ad with the purchases of those who do did not see it, the study concluded that there are large positive effects of online advertising.

However, "the population of people who sees a particular ad may be very different from the population who does not see the same ad,” says Reiley. Reiley's research made use of a database match between Yahoo! and a nationwide retailer by identifying users who registered the same email address with both companies. After finding over one million matched users, the researchers randomly assigned them to treatment and control groups for one of the retailer’s online advertising campaigns.

The project then tracked sales each week at the retailer, both online and in stores. In a paper co-authored with summer intern and MIT PhD student Randall Lewis, Reiley found that the online display advertising increased total revenues by approximately 5% for those users exposed to the ads, with 93% of the total effect happening in offline sales. They also observed online ads to have a large impact on sales even when the ads are not clicked: 78% of the increase in sales came from those who viewed, but did not click, the ads.

Could this have implications for the "pay-per-click" model in online advertising?

How to Design Smart Business Experiments

the Harvard Business Review carries a piece by Thomas Davenport that is recommended reading for companies looking to examine "what works"

http://hbr.harvardbusiness.org/2009/02/how-to-design-smart-business-experiments/ar/1

Field Experiments in Economics: The Past, The Present, and The Future

We already (here) have discussed Stefano della Vigna's recent NBER paper on field experiments in economics. In this NBER paper (here), Steven Levitt John A. List write on the topic, with a historical perspective.

Abstract:

This study presents an overview of modern field experiments and their usage in economics. Our discussion focuses on three distinct periods of field experimentation that have influenced the economics literature. The first might well be thought of as the dawn of "field" experimentation: the work of Neyman and Fisher, who laid the experimental foundation in the 1920s and 1930s by conceptualizing randomization as an instrument to achieve identification via experimentation with agricultural plots. The second, the large-scale social experiments conducted by government agencies in the mid-twentieth century, moved the exploration from plots of land to groups of individuals. More recently, the nature and range of field experiments has expanded, with a diverse set of controlled experiments being completed outside of the typical laboratory environment. With this growth, the number and types of questions that can be explored using field experiments has grown tremendously. After discussing these three distinct phases, we speculate on the future of field experimental methods, a future that we envision including a strong collaborative effort with outside parties, most importantly private entities.