Tom Pepinsky

Month: September 2014

  • What If We’re Measuring Policy Relevance All Wrong?

    The standard debate about policy relevance in academic political science is now familiar. There are, broadly, three sides.

    1. We should be doing more to engage policymakers with the work that we do.
    2. We should do our work differently in order to better engage with policymakers.
    3. It is not our job to care if policymakers engage with our work (and as a result, a lack of policy relevance is not reason to change how we work).

    I tend to believe that it’s perfectly reasonable to care that our work is relevant to policy, which leaves the debate (for me) between (1) and (2). The terms of that debate, alas, look depressingly like every other disciplinary disagreement. Are we too quantitative, too theoretical, too abstract, too driven by disciplinary or subdisciplinary incentives to be policy relevant? Replace “be policy relevant” with “train graduate students” or “understand current events” and you’ve just described 95% of our disciplinary debates. The profile of political scientist/former Ambassador to Russia Michael McFaul in the New Yorker is a perfect example.

    There does, though, seem to be something of an unstated agreement of how to measure policy relevance: breadth of influence, usually of big ideas. Impressionistic accounts abound about how few Congressional staffers read the APSR. Most systematic data collection efforts look to see if some major theoretical perspectives have been taken up by lots of policymakers (the TRIP Policymaker Survey is one example). The basic presumption is that research is relevant if lots of policymakers know about a Big Theory or a Grand Paradigm. Another version of this, one freed from theories and paradigms, is to see if there is One Important Scholar known by lots of these policymakers.

    But what if this is exactly wrong? What if it’s the murky, quirky, small-bore, nuanced analytical work—that does not have Broad Disciplinary Implications, and is not designed to Shape Basic Thinking about Our World—that policymakers want? What if this is the stuff that actually influences how they make decisions?

    If that’s the case, you wouldn’t expect to find evidence of policy relevance by searching for awareness of a small number of broad ideas across a large number policymakers. Instead, you might need to find analyst-level policymakers, those doing the basic research that provides the background for major policy choices. These analysis will have narrow ambits, and will focus tightly on specific problems and issue areas. They need not care about whether civilizations are clashing generally, they need to learn if we have evidence that (say) Putin can be dissuaded from invading Ukraine. Lots of middle range theories may matter here, as can basic analysis of current events or survey data using the latest techniques.

    It could be the case, in other words, that political science research already is very relevant, we just aren’t measuring it. Indeed, we probably cannot measure this type of policy relevance, not systematically, because we cannot identify with any precision the population being influenced, or the concrete features of that influence. If my own (non-systematic) observations from my own (probably not unique) engagement with the policy world are any indication, academic political science already is very influential, just not in a Big Thinker/Big Idea kind of way, and only for the select few individuals with focused, narrow policy briefs. But this is actual influence, not just awareness of academic debates.

    I should note, incidentally, that the TRIP Policymaker Survey already shows that policymakers claim to value area studies. I suspect that the marriage of good area studies with good social science is at the heart of some of the most influential and policy-relevant “small-bore” analytical work. (An example.)

    One interesting observation follows. Some readers will remember that I argued recently that we “should not want” to produce superficial (what Dan Murphy called “sweeping, likely wrong“) research that can be taken up on short soundbites. Part of the reason why I think this is because I have a normative belief that policymakers should prioritize valid data, detailed history, and careful causal inference over big paradigms or grand theories. In other words, a world in which just one or two policymakers know exactly why there are internal splits within the People’s Bank of China and how to interpret them may actually be preferable to one in which 85% of policymakers can confirm their awareness of a Realist Theory of the Rise of China. And I bet that is actually how policymakers value things too.

  • All Islam is Vernacular Islam

    I happened to come across this announcement for a Mellon Visiting Postdoctoral Fellowship at Vanderbilt, carrying the title “When the Fringe Dwarfs the Center: Vernacular Islam beyond the Arab World.” It looks like a fantastic opportunity, but it highlights a continuing problem in both academic and popular discussions of Islam.

    That problem is the perceived asymmetry between “central” and “fringe” Islam. The Mellon program is to be commended for encouraging us to remember that about 80% of the world’s Muslims are not Arabs. But even in making this point, it commits the standard mistake of opposing the Great Tradition of Arab Islam with the Little Traditions found in Istanbul, Tashkent, Lahore, Bamako, Dakar, Mogadishu, Dhaka, and Jakarta (to say nothing of Manchester, Marseilles, or Minneapolis). The term “vernacular” probably stems from a linguistic perspective, referring to those who speak a “vernacular” language, i.e., something other than Arabic. But as written in the program description, it reflects more than just that, implying a kind of purity or refinement of one particular form of Islam. It also implies a position for that form of Islam: at the center or core, versus the edges or fringes or peripheries.

    One way to clarify my objection is to ask what is the opposite of vernacular Islam? What is it called? Who practices it? What does it entail? If you find yourself uncomfortable answering that question—and nearly everyone who has thought hard about Islam will—then you should also be uncomfortable with the very idea of a fringe.

    So, say it together with me: All Islam is vernacular Islam. All of it. Wahhabis and Salafis in the Arab Middle East are just as much products of particular historical moments and sociopolitical contexts as are other Sunni Muslims who don’t happen to speak a form of Arabic as their native language (to say nothing of Ibadis, Ismailis, Alevis, etc.). The same imperative to gaze outward from the Arab Middle East must be complemented with a hard gaze inward among scholars of Islam in the Arab Middle East. Perhaps that is what the program leaders mean when they write

    as most Muslims live in the “fringes,” we need to problematize the notions of center and periphery, the relationship of the symbolic core to its ever-expanding outlying majority, and the latter’s creative adaptations of Islam.

    But even this implies that it is the fringes where we look for the “creative adaptations of Islam,” not the center. So long as scholars continue to write as if the Islam of the Arab Middle East is somehow unproblematic, they will continue to perpetuate the myth of a non-vernacular Islam.