“Minimal Model Explanations,” R.W. Batterman & C.C. Rice (2014)

I unfortunately was overseas and wasn’t able to attend the recent Stanford conference on Causality in the Social Sciences; a friend organized the event and was able to put together a really incredible set of speakers: Nancy Cartwright, Chuck Manski, Joshua Angrist, Garth Saloner and many others. Coincidentally, a recent issue of the journal Philosophy of Science had an interesting article quite relevant to economists interested in methodology: how is it that we learn anything about the world when we use a model that is based on false assumptions?

You might think of there being five classes which make up nearly every paper published in the best economics journals. First are pure theoretical exercises, or “tool building”, such as investigations of the properties of equilibria or the development of a new econometric technique. Second are abstract models which are meant to speak to an applied problem. Third are empirical papers whose primary quantities of interest are the parameters of an economic model (broadly, “structural papers”, although this isn’t quite the historic use of the term). Fourth are empirical papers whose primary quantities of interest are causal treatment effects (broadly, “reduced form papers”, although again this is not the historic meaning of that term). Fifth are descriptive work or historical summary. Lab and field experiments, and old-fashioned correlation analysis, all fit into that framework fairly naturally as well. It is the second and third classes which seem very strange to many non-economists. We write a model which is deliberately abstract and which is based on counterfactual assumptions about human or firm behavior, but nonetheless we feel that these types of models are “useful” or “explanatory” in some sense. Why?

Let’s say that in the actual world, conditions A imply outcome B via implication C (perhaps causal, perhaps as part of a simultaneous equilibrium, or whatever). The old Friedman 1953 idea is that a good model predicts B well across all questions with which we are concerned, and the unreality of the assumptions (or implicitly of the logical process C) are unimportant. Earlier literature in the philosophy of science has suggested that “minimal models” explain because A’, a subset of A, are sufficient to drive B via C; that is, the abstraction merely strips away any assumptions that are not what the philosopher Weisberg calls “explanatorily privileged causal factors.” Pincock, another philosopher, suggests that models track causes, yes, but also isolate factors and connect phenomena via mathematical similarity. That is, the model focuses on causes A’, subset of A, and on implications C’, subset of C, which are of special interest because they help us see how the particular situation we are analyzing is similar to ones we have analyzed before.

Batterman and Rice argue that these reasons are not why minimal models “work”. For instance, if we are to say that a model explains because it abstracts only to the relevant causal factors, the question is how we know what those factors are in advance of examining them. Consider Fisher’s sex ratio model: why do we so frequently see 1:1 sex ratios in nature? He argues that there is a fitness advantage for those whose offspring tend toward the less common sex, since they find it easier to procreate. In the model, parents choose sex of offspring, reproduction is asexual (does not involve matching), no genetic recombination occurs, there are no random changes to genes, etc: many of the assumptions are completely contrary to reality. Why, then, do we think the model explains? It explains because there is a story about why the omitted factors are irrelevant to the behavior being explained. That is, in the model assumptions D generate E via causal explanation C, and there is a story about why D->E via C and A->B via C operate in similar ways. Instead of simply assuming that certain factors are “explanatorily privileged”, we show that that model factors affect outcomes in similar ways to how more complicated real world objects operate.

Interesting, but I feel that this still isn’t what’s going on in economics. Itzhak Gilboa, the theorist, in a review of Mary Morgan’s delightful book The World in the Model, writes that “being an economic theorist, I have been conditioned to prefer elegance over accuracy, insight over detail.” I take that to mean that what economic theorists care about are explanatory factors or implications C’, subset of C. That is, the deduction is the theory. Think of Arrow’s possibility theorem. There is nothing “testable” about it; certainly the theory does not make any claim about real world outcomes. It merely shows the impossibility of preference aggregation satisfying certain axioms, full stop. How is this “useful”? Well, the usefulness of this type of abstract model depends entirely on the user. Some readers may find such insight trivial, or uninteresting, or whatever, whereas others may find such an exploration of theoretical space helps clarify their thinking about some real world phenomenon. The whole question of “Why do minimal models explain/work/predict” is less interesting to me than the question “Why do minimal models prove useful for a given reader“.

The closest philosophical position to this idea is some form of Peirce-style pragmatism – he actually uses a minimal model himself in exactly this way in his Note on the Economy of the Theory of Research! I also find it useful to think about the usefulness of abstract models via Economic Models as Analogies, an idea pushed by Gilboa and three other well-known theorists. Essentially, a model is a case fully examined. Examining a number of cases in the theoretical world, and thinking formally through those cases, can prove useful when critiquing new policy ideas or historical explanations about the world. The theory is not a rule – and how could it be given the abstractness of the model – but an element in your mental toolkit. In physics, for example, if your engineer proposes spending money building a machine that implies perpetual motion, you have models of the physical world in your toolkit which, while not being about exactly that machine, are useful when analyzing how such a machine would or would not work. Likewise, if Russian wants to think about how it should respond to a “sudden stop” in investment and a currency outflow, the logical consequences of any real world policy are so complex that it is useful to have thought through the equilibrium implications of policies within the context of toy models, even if such models are only qualitatively useful or only useful in certain cases. When students complain, “but the assumptions are so unrealistic” or “but the model can’t predict anything”, you ought respond that the model can predict perfectly within the context of the model, and it is your job as the student, as the reader, to consider how understanding the mechanisms in the model help you think more clearly about related problems in the real world.

Final version in Philosophy of Science, which is gated, I’m afraid; I couldn’t find an ungated draft. Of related interest in the philosophy journals recently is Kevin Davey’s Can Good Science Be Logically Inconsistent? in Synthese. Note that economists use logically inconsistent reasoning all the time, in that we use model with assumption A in context B, and model with assumption Not A in context C. If “accepting a model” means thinking of the model as “justified belief”, then Davey provides very good reasons to think that science cannot be logically inconsistent. If, however, “accepting a model” meaning “finding it useful as a case” or “finding the deduction in the model of inherent interest”, then of course logically inconsistent models can still prove useful. So here’s to inconsistent economics!

Advertisements

6 thoughts on ““Minimal Model Explanations,” R.W. Batterman & C.C. Rice (2014)

  1. Noah Smith says:

    I don’t think structural models seem strange to non-economists. They seem to be the standard for empirical work in most sciences. In fact, in this essay, Larry Summers criticizes structural modeling as suffering from a “scientific illusion”: http://www.econ.ucdavis.edu/faculty/kdsalyer/LECTURES/Ecn200e/summers_illusion.pdf

    • afinetheorem says:

      Fair point – I ought have said weird to other social scientists: there is no doubt that structural work looks strange, or even gets a hostile reaction, among most historians, sociologists, anthropologists, etc. We can safely include law in that group as well, at least in my experience.

      • Noah Smith says:

        Ah, true.

        I think what scientists might find weird about structural papers in econ is that they rarely focus on evaluating the models whose parameters they estimate. In sciences, you often see structural papers whose purpose is to investigate how well a model is specified (i.e. test the model’s fit). But in econ, structural papers usually seem to just assume that a model is correctly specified, and focus on the values of the parameter estimates. Do you also find this to be the case?

      • afinetheorem says:

        Agreed, but I am not as sanguine about the usefulness of such estimates as some people are. These models in economics are generally used to discuss counterfactuals. Formal test of the model’s fit on factual data is of very little use, it seems to me. Essentially, I read the numbers in these papers as “the qualitative model could imply very important economic effects or very minor ones depending on the parameters at play; let’s say as little as possible about the features not implied by economic theory while still doing enough to be able to estimate a model, and see whether with reasonable sounding numbers we learn anything interesting.” We then have another “case” which may be useful. To try to say more than that is hopeless. Note the same is true of empirical work: if you estimate a LATE from some policy experiment in Denmark in 1992, there are implicitly some pretty crazy assumptions you need to make if that empirical estimate is to be useful to a policy in a similar area in Tennessee in 2014. At least the structure makes clear what the implicit assumptions are!

    • mulp says:

      But scientists do not use models that contradict to advocate a policy like economists seem to do.

      Nor do scientists erase factors of nature from models to make the models justify a policy.

      For example, today we have what I call free lunch economics.

      In this model, the way to create more jobs is to lower wages without allowing for labor to incur any costs like transportation, food, housing. Thus a job for $1 per hour is a good job even if in two hour shifts and 50 miles from home, as long as there are not taxes on the wages, because taxes of 17 cents will destroy all incentives to work. Nothing about the $20 in cost to get to the job is ever allowed in any model.

      And higher employment at lower wages is good if it boosts profits which creates wealth that creates more consumption through the wealth effect. Ie. consumption fueled by debt. Higher minimum wages will both slash unemployment and profits and that will destroy wealth cutting consumption due to credit crunch, because workers are not consumers and consumers never workers.

      No discussion I have seen in decades on the minimum wage makes any case for higher low end wages leading to higher consumption, clearly based on not wanting to justify higher wages as the way to generate higher growth. The few people arguing for higher wages to end the government subsidies of big highly profitable corporations by getting their workers off welfare are dismissed as crackpots who do not understand economics, or call for ending welfare so they working poor will drive down prices of food, housing, cars, college, etc.

  2. Noah Smith says:

    But this brings us to the eternal question: How do you know if the model you’re using to examine counterfactuals is appropriate?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: