How should society aggregate preferences when people disagree on beliefs? The standard impossibility results are based on the von-Neumann-Morganstern world where all agents agree about the probability distribution of events. If disagreement exists over probabilities of evens (as this paper says, everyone wants peace but we disagree on how to achieve it), then aggregating beliefs in less straightforward. A standard axiom for social welfare functions has been Pareto: if all individuals prefer X, then society should as well. The authors argue that this is problematic in the context of disagreement over events. Consider a duel, where each agent loses 5 if he gets shot, gets 0 if he does not participate, and wins 1 if he shoots the other guy. Each believe with 85% probability that they will win the duel, and hence each prefers to take part in the duel. But surely society should not endorse the duel, since these beliefs over probabilities are contradictory. The authors show that a simpler condition – if everyone is indifferent over lotteries, then so is society – is satisfied iff social welfare is a convex combination of probabilities and of utility functions, and that if that is the case, earlier results from Harsanyi show exactly when a social welfare function can exist even with subjective probability.
“Utilitarian Aggregations of Beliefs and Tastes,” I. Gilboa, D. Samet & D. Schmeidler (2004)