It is well known that very few people play the “centipede game” in the backward inductive way suggested by Nash equilibria. In particular, extensive experimental work suggests that very few people play the Nash strategy of stopping on their first move. Why? Reasons might be that people have cognitive limits on their ability to discover backward inductive strategies, or have social preferences not captured by the assumption that u(x)=x where x is the money payout, or have a taste for fairness, etc. Palacios-Huerta and Volij (2009) suggested that, when the game is played with chess players, who presumably are able to find solutions to difficult backward inductive strategies, around 70% of chess masters paired with other chess players “stop” on the first move.
In a forthcoming AER, Levitt, List and Sadoff all but call out these results are implausible. They gather expert chess players and play centipede games, and unsurprisingly find that chess players, in fact, almost never stop at the first node of centipede. The following experiment is more useful, however: the chess players then play a game called “Race to 100” which essentially has no problems of social preferences, and who backward inductive strategy is (relatively) simple to compute. A number of the chess masters play Race to 100 with precisely the Nash backward inductive strategy, and there is no link between their ability to “solve” the Race to 100 game, and their willingness to play Continue in the centipede games. This suggests that the usual caution of games – that the researcher has, by writing down u(x)=x implicitly, misspecified the preferences of the players – can drive a number of non-Nash results without resort to questions of cognitive limits. This paper also suggests a healthy skepticism toward experimental results, and a better way to verify what has happened in experiments before data goes to top journals. I don’t mean to suggests that the Palacios-Huerta/Volij data is made up, but I do mean that the numbers reported in their paper are almost laughable: I would be shocked if you could get 70% of academic game theory professors to stop at the first node in centipede, let alone anyone else.
A Fine Theorem 
I have several comments about this interesting post.
First, what it does mean to call the results “implausible” (your word for Levitt et al’s impllication for P-H&V) or “laughable” (your word)? Results cannot be plausible or implausible unless you are asserting fraud– they are simply results. You might say that the general implication from these data to other settings is unlikely to generalize. Is that what you mean?
You also note that “This paper also suggests a healthy skepticism toward experimental results”. I don’t see why this is so (given that is already plenty of skepticism, much unhealthy in the sense of being uninformed or relying on disproven conjectures). It’s true that one experiment got a different result than a different experiment. But this is often true in all empirical work, and experimental results have the distinct virtue of being easily replicable (which many nonexperimental analyses are not).
You write that “This paper also suggests…[and] a better way to verify what has happened in experiments before data goes to top journals.” What is the better way you have in mind? Have all experiments replicated before publication? I would be all in favor of that, actually (or a milder “audit” model in which some experiments are randomly replicated), but journals have resisted.
Your accurate point that misassuming u(x)=x “can drive a number of non-Nash results without resort to questions of cognitive limits” is certainly correct, but has been well-known in the literature for almost thirty years, since at least Guth et al (1982) ultimatum games (since rejections by responders do not involve cognitive limits). Camerer et al (1993) (chapter in a Binmore-Tani edited book) was one of the first papers to do this by controlling social preferences and cognition (playing a selfish algorithm and instructing subjects about backward induction). Guth’s original finding inspired a very large (probably 500 paper) literature. The Levitt et al paper is a contribution on highly trained unrepresentative subjects, but is not especially innovative about the basic question of the combined roles of social preference and cognition.