Predicting behavior in games: a competition

Sometimes an experimental design is meant in part to solve a market design problem. That's the case with the call for entries reproduced below. You are invited: Enter and win:)

The market design issue is twofold. Scientific publishing gives a lot of incentives for reporting positive results about interesting problems, but can have the effect of suppressing negative results (the "file-drawer effect") or sometimes promoting false positives. It is also hard for researchers to report how models perform on representative random samples of problems, both because this requires relatively big and costly experiments, and because randomly chosen problems (some of which are, by themselves, boring) may not be as rewarding to examine as are problems carefully selected to showcase the virtues of a particular model.


The competitions we're proposing are an attempt to ameliorate both incentive problems. The hosts of the competition will run the necessary experiments on random samples of games, and invite researchers to submit models to predict the observed behavior. (Researchers can test their model on a first random sample of games for which the experimental results are reported before the competition, and they are invited to predict results for a second random sample of games that will not be published until all the models are submitted.) So the competition is cheap to enter (the random sampling experiments are already taken care of), and the entries are submitted before their authors know how well they will perform.


Here's the call for a competition to predict behavior in market entry games. And here is a version just sent out by email:

Ido Erev, Eyal Ert, Al Roth and Games Editorial Office invite you to participate in the choice prediction competitions that will be conducted as part of the special issue of the journal Games (http://www.mdpi.com/journal/games/) on “Predicting Behavior in Games” (http://www.mdpi.com/si/games/predict-behavior/). Below is the call to participate in the first competition which focuses on market entry games.
The first “Games” competition: Predicting behavior in market entry games.
The first competition focuses on the prediction of behavior in repeated 4-person market entry games. The organizers first ran (in March 2010 at Harvard) an experimental study of (40) games that were randomly selected from a well-defined space of market entry games. The raw experimental results of this study, referred to as the “estimation experiment,” are presented in the competition’s website (http://sites.google.com/site/gpredcomp/).
In addition, the competition website includes the rules of the competition, and a link to a paper that summarizes the results of the estimation experiment and explores the value of several baseline models (http://www.mdpi.com/2073-4336/1/2/117/.)
The site explains that the goal of the participants in the competition is to predict the results of a second experiment. This study, referred to as the “competition experiment,” will be run by the organizers in May 2010 (but the results will be kept confidential until 2 September 2010). The competition experiment will use the same method as the estimation experiment, but will study different games (drawn from the same space of games) and different subjects.
To participate in the competition you will have to email us a computer program (in MATLAB, Visual Basic, or SAS) that reads the parameters of the games (the incentive structure) as input, and predicts the main results as output. The program should be an implementation of your favorite model. To develop and/or estimate your model you are encouraged to analyze the data of the estimation experiment, and to build on the baseline models that were posted in the competition website.
The submitted models will be ranked based on the mean squared deviation between the predictions and the results of the competition experiment. The prize for the winners will include an invitation to publish a paper that describes the winning model in Games, and an invitation to a special workshop.
The submission deadline for this competition is 1 September 2010. You are allowed to submit one model as a first author and to co-author up to three additional submissions.
Best regards,

Ido Erev, Eyal Ert and Al Roth
Guest Editors
Games Special Issue "Predicting Behavior in Games"
http://www.mdpi.com/si/games/predict-behavior/