Tom Pepinsky

Category: Research

  • Paul Romer and the Principles of Academic Engagement

    Paul Romer has written a couple of really biting commentaries on the contemporary state of economic theory (this and this). The enemy is “mathiness,” a term he employed—if not coined—in a recent paper in the American Economic Review Papers and Proceedings.

    Like mathematical theory, mathiness uses a mixture of words and symbols, but instead of making tight links, it leaves ample room for slippage between statements in natural versus formal language and between statements with theoretical as opposed to empirical content.

    There’s a lot more to this debate, and it will no doubt continue to unfold in the coming weeks. (And in the academic economics blogosphere, no less!) The debate among economic theorists has very little relevance for how to do contemporary political science research, but it will still be interesting to follow for anyone interested in the history of economic theory, the politics of macroeconomic policymaking, and so forth. I do agree that reading Romer’s interpretation of the point of Milton Friedman’s “The Methodology of Positive Economics” helps me to put it in a new light.

    What catches my eye, though, is how Romer describes the foundations of academic collaboration in a science of economics.

    a) We trust that what each person says is an honest account of what he or she thinks is true.
    b) We all recognize that reasonable people can differ and that no one has privileged access to the truth.
    c) We take seriously the claims of people who disagree with us.
    d) We are ready to admit that others might be right that each of us might be wrong.
    e) In our discussions, claims that are recognized by a clear plurality of members of the community by as being better supported by logic and evidence are the ones that are provisionally accepted as being true.
    f) In judging what constitutes a “clear plurality,” we put more weight on the views of people who have more status in the community and are recognized as having more expertise on the topic.
    g) We update the status of a members of our community on the basis of his or her contribution to progress a clearer understanding of what is true, not on the basis of “unwavering conviction” or “loyalty to the team.”
    h) We shun, or exclude from the community, someone who reveals the he or she is not committed to these working principles.

    I’m not sure I agree about the place of experts with status in the community: it puts a heavy burden on the experts to be self-critical in a way that I just don’t think is realistic. But in general this seems like a good set of principles. If we hold the commentary on experts and status aside, in fact, Romer’s principles can be simplified into just four.

    Modified Principles for Academic Engagement
    1. Mostly everyone is really smart
    2. Mostly everyone is really trying
    3. Mostly everyone is usually wrong
    4. All of these apply to you too, especially 3

  • An Academic Journal Draft

    Here is how the academic journal market currently works:

    Authors write manuscripts. Then, they submit to journals, which decide whether or not to publish their manuscripts based on editors’ tastes, referees’ evaluations, and a bunch of other stuff we don’t observe. Authors can submit each manuscript to only one journal at a time, which is responsible for the generally atrocious amount of time that it takes for a working paper to turn into an actual publication. The currency of journal publications for tenure and promotion is incredibly valuable. The present discounted value of a top publication for a new assistant professor is probably around a million dollars over his or her lifetime.

    As a result, authors compete with one another to get into the best journals. Journals compete with one another to accept the best manuscripts, but that’s basically the only way that they compete.

    I’ve often wondered why the academic journal submissions market works this way. This isn’t the only way to match manuscripts to journals. Looking at professional sports, in fact, gives us a very different model. Why don’t we have an academic journal draft?

    Here’s what I have in mind:

    Authors produce manuscripts and submit them to a centralized repository. Journals then make proposals to authors after having seen the manuscripts: “we promise to put this under peer review within 3 months, no restrictions on other journals reviewing it at the same time, 15,000 words to work with, and if accepted, we will make it available in on month, with 6 months open access and help publicizing it.” Or some other suite of proposals. It would depend on the journal and the manuscript, as well as the academic discipline. Basically, just like a professional sports draft, only instead of teams drafting players, we have journals drafting manuscripts. (The major difference is that the academic journal draft would not happen one journal at a time, once a year, but rather continuously.)

    In this model, authors and journals would both compete. Authors compete to write the most attractive manuscripts, but journals also compete to offer the best review and publication experience. Journals might also discriminate among authors, offering some manuscripts better terms.

    So why don’t we have a journal draft? It’s probably just a form a path dependence based on how the earliest journals worked, but I also suspect that peer review stands in the way of implementing the model that I just described. I cannot see how an editor could ensure vigorous peer review at the same time that s/he is trying to entice the author to choose his or her journal. And in fact, the process for submitting manuscripts to law reviews does look a lot like an academic journal draft, with one very significant exception—nearly all law reviews lack formal peer review.

    There are of course other, much more elaborate proposals for reforming academic publishing. Their goal is better science, which of course I support too (regardless of whether those specific prescriptions would work is a separate question). But from an author’s or researcher’s perspective, the reforms to publishing are fairly minor. It’s interesting to think how about how a draft system would upend the entire academic publishing model, even if it probably could never actually be implemented.