Behavioural Economics Replication Project

I've highlighted the work of the Many Labs Replication Project on the blog before. Since then, Many Labs 2 and 3 have also launched, but the replication efforts have so far remained confined to psychology.

Until now. The Behavioural Economics Replication Project prediction market opens today and will run for 11 days (April 15-26). The studies being replicated are drawn from 16 articles from American Economic Review and 2 from Quarterly Journal of Economics, two of the premier journals in economics. The purpose of the site is to allow people to bet on whether they think the effects in the original articles will be replicated. I could not find details on the website about when the replications will actually be run.

While I am interested in prediction markets like many nerds, I wonder what the added value of it is here. I wouldn't be shocked to see one or two null effects out of 18 but I would be pretty surprised if there are studies in journals like QJE and AER which everyone thinks won't replicate. That's the beauty of prediction markets though, they force everyones cards onto the table.

More details below:

Who will do the replications?

The replications will be conducted by research teams from Caltech, the University of Innsbruck, the Stockholm School of Economics and the National University of Singapore.

What does it mean for a study to replicate?

Of course, prediction market design requires that event outcomes can be clearly said to have either occurred or not. Therefore, we define a result as being replicated if the statistical method used in the original paper yields a p-value <0 .05.="" a="" corresponding="" if="" is="" not="" p-value="" replicated="" result="" the="">0.05.

What are the prediction markets for?

In the prediction markets the traders can give their assessment of whether a result will be replicated or not. For each study, a separate prediction market is run to predict the outcome of the replication. In each market one asset is traded whose payoff depends on whether the result/hypothesis will be replicated or not.

Market Setup

You can trade on the outcome of each of the 18 replication studies in a web-based market interface. This webpage (www.sciencepredictionmarkets.com) provides information on the 18 studies (and the specific hypotheses therein) which are to be replicated.

Your initial endowment is 100 Tokens. You can trade in as many of the 18 markets as you want to. However, any residual Tokens not invested will have no value after markets have closed.

Once you have identified a market (study) that you would like to trade in, click on the corresponding button to access it.

Then you will see two possibilities to trade:

• Increase position by # Tokens:
  If you believe that the study will be replicated according to the criteria outlined above, you can invest here.
• Decrease position by # Tokens:
  If you believe that the study will not be replicated according to the criteria outlined above, you can invest here.