Famously, the first auction we know of in the historical record is the “Babylonian Wives” auction reported by Herodotus (I know – you likely prefer Didius Julianus buying the entire Roman Empire in an auction!). Every year, Babylonian villages would bring out all the girls of marriageable age, ordering them from the prettiest to the ugliest. Suitors would first bid for all the beautiful women. With the money thus collected, then, men would be paid to marry the uglier women. How romantic.
But beyond being a great history of econ story, the Babylonian auction turns out to have some really interesting game theoretic properties. Consider an auction with two women and two bidders. Each bidder gets a known amount of utility from the beautiful and the ugly maiden: for now, assume each suitor gets H from the beauty and L from the other woman. Whatever bid wins the beautiful maiden is paid to the winner of the ugly maiden. If bids are identical, then each suitor is given the beautiful woman or the ugly woman with equal probability, and no money changes hands. There is a simple equilibrium in pure strategies: each agent bids (H-L)/2. But what of mixed strategy equilibria?
The normal way we find symmetric mixed strategy atomless equilibria is the following. First, identify a strategy distribution F such that payoffs are constant on the support of F, given that the other player also bids F. Second, verify that F is a well-defined, continuous cdf. Third, show that neither play can deviate by bidding off the support of F and increasing their payoff, given the opponent bids F. And such a method will, in fact, find the set of atomless mixed strategies without profitable deviations. But this does not guarantee that we have found equilibria when payoffs are potentially unbounded. Why? Fubini’s Theorem.
You may remember Fubini’s theorem from high school calculus: it tells you when you are allowed to change the order of integration in a double integral. Recall that expected utility calculations in a two-player game involve solving a double integral: integrate over the product space of my strategy’s probability distribution and over your strategy’s probability distribution. One way to know whether the expectation of a random variable X, such as utility given your and my strategies, is well-defined is to check whether max(X,0) and min(X,0) are both equal to infinity. If they both are, then the expectation does not exist.
If you solve the Babylonian Wife auction with complete information using the three step process above, you get that each player earns expected utility equal to L+m, where m can be arbitrarily large. That is, it appears each player can earn infinite payoff. But we can easily see the game is one with a constant sum, since one player earns H from the beautiful wife, one earns L from the ugly wife, and the transfers exactly cancel. The method correctly finds a mixed strategy with no profitable distribution. But the mixed strategy found is one such that calculating ex-ante expected utility from playing the mixed strategy involves solving a double integral whose solution is not well defined. That is, the three-step method needs a fourth step: check whether the joint distribution of putative mixed strategies gives well-defined expected utilities for each player. A similar problem exists in the Babylonian Wife auction with incomplete information, of which details can be found in the paper. Such a problem is likely rare: the war of attrition, for instance, also has potentially unbounded payoffs, but does not have problematic computation of expected utility.
http://www.ifo.de/portal/pls/portal/docs/1/1185352.PDF (2010 working paper. The 2012 final version in Games and Economic Behavior is substantially shorter.)