“Climbing Atop the Shoulders of Giants,” J. Furman & S. Stern (2011)

I have been asked a couple times what the difference is between innovation and invention. A critical distinction is that innovation involves not just the creation of new knowledge, but also its dissemination. And dissemination is what really matters when it comes to economic growth or pushing the frontier of science. Particularly given Ben Jones’s papers about science slowing because reaching the frontier takes longer for young researchers, you might wonder whether there are policies governments and R&D labs can enact in order to get their scientists up to speed more quickly.

There are many such policies, in principle. We can have editors rewrite scientific studies in order to make them easier to read (or set up weblogs devote to summaries of new research, of course). We can have replication done in a more formal way in order to make previous results easier to trust. We can establish open data policies to make it easier to follow up on earlier articles. Are such policies worthwhile?

Furman and Stern examine a set of institutions called biological resource centers (BRCs). These centers, of which there are many, archive and certify biomaterial used in research studies. They forward samples to researchers who want to follow up on earlier work. Famously, Kary Mullis, a private sector researcher, used a strange organism that lived in geysers at Yellowstone and had been archived a decade before at a BRC, to develop a technique allowing quick replication of genetic material. He won the Nobel for this work. When archived, no one suspected the organism would have any major practical use.

Assuming citations measure dissemination of knowledge in some sense, do studies using BRC-archived material get cited more than other work in similar journals on similar topics, and does the citation profile over time look different? The BRC articles get 220 percent more citations, though much of this is due to a selection effect: high quality articles are more likely to have their materials archived. Furman and Stern have a nice trick to control for selection, though. On three occasions in their sample, a private sector “special collection” was forwarded to a major public BRC – for example, one private sector lab shut down and forwarded many years of their internal archives. Papers using biomaterial from the special collection have an expected lifetime citation profile given the citation they received during, say, the nine years they were not in a BRC. The marginal impact of being added to a BRC is 50 to 125 percent. The impact is biggest for articles originally published in lower ranked journals (a certification effect), and BRC accession leads to a roughly 100 percent increase in unique labs and universities appearing in future citations. The idea in the latter effect is that, with private archiving, follow-up studies generally come from friends and associates of the original author, whereas BRC material is open to everybody.

Finally, a nice back of the envelope calculation: are the RBCs cost effective? We have estimates from other research about the average “cost” of a citation in biology studies, which is about 2400 dollars. Accession of materials to BRCs costs an average of 10000 dollars. Given the marginal impacts of BRC accession, funding of BRCs is 3 to 10 times more cost effective than funding new research. More broadly, it might do the NIH and NSF well to shift some money from new research to dissemination strategies. And indeed they are doing just that – I hope to show you some results of this policy by next spring.

ftp://ftp.zew.de/pub/zew-docs/veranst_upload/1232/525_BRC%20FS%20(Jun-06-2010).pdf (June 2010 working paper – final version in AER 2011. Ironically given the focus of this paper on quick dissemination of research, Furman and Stern took at least seven years to publish; there are 2004 working paper versions that have basically the same results as the final published version. Ellison showed in a paper a few years back that the slow pace of publishing in economics is due to referees asking for endless rounds of minor revisions. It’s outrageous that our field puts up with 5-10 year lags in publishing.)

Advertisement