Over lunch today, some economist colleagues and I discussed the recent turn towards randomized clinical trials (RCTs) in development economics. For those who don’t know, this is the move to implement large scale medical-style trials of economic policies in poor countries. Things like, if we give microfinance to people, does it work? Study this by randomly dividing the population into a group that receives it, and one that does not. Chart the difference in the two groups. (With the proper assumptions, of course) you therefore have a great way to tell if microfinance works, and if so, how much.
These RCTs are a big deal, and they are really expensive. They are probably a fad–although one that I’ve kinda taken part in myself–and I do not have any penetrating analysis beyond what much smarter people have already said about it. Our main conclusion over lunch is that above and beyond what existing critiques have said, RCTs aren’t likely to tell us much about what we care about for making poor people better off. These are two economists who care a lot about specific poor countries in Southeast Asia–take, for example, Burma or Cambodia–but who draw very few actionable conclusion from RCTs about how one might improving these people’s lives.
The main conclusions of magisterial studies like this are that you should probably let women participate in decisions and you should vaccinate children against malaria and you should let credit flow to productive growth sectors. That’s hardly radical, but it’s nice to be absolutely positive that Good Thing do Work, and that not always in way that we think that they do. It would useful to know if they did not, and it’s good to be sure that we’re not just cherry-picking the evidence to confirm our biases. (I am sure there are people who might disagree that governments should do Good Things, and it’s definitely important that the evidence on the effectiveness of what many of us used to think were Good Things is pretty weak.) But my lunch partners and I–again, these two are old school development economists with deep country expertise, something rare in the U.S. these days–concluded that the main problems facing poor countries are in implementation.
Here’s what I mean. RCTs are great for confirming that Good Things that we think Work actually do. But the main problem facing most poor countries is not that we don’t know that, but how to get the policies in place.
I can see six possible reasons why obviously pro-poor policies (and I don’t mean income redistribution, I mean things like making sure children aren’t drinking filthy water) aren’t provided by politicians.
1. politicians don’t know if it’s a good idea to provide them
2. politicians don’t know how to provide them
3. politicians have budget constraints and so they can’t afford to provide them
4. politicians have political incentives that make them direct their energies elsewhere
5. the bureaucracy responsible for making these things happen is by necessity so confusing and complicated that it’s just really easy for well-meaning politicians and bureaucrats to fail and it’s no one person’s fault
6. politicians are evil and don’t care about the poor (this is not a joke, I think it aptly describes people like Than Shwe)
RCTs solve problem (1). But I am convinced that (2-6) are the actual barriers to allowing Good Things to Work, and these are precisely the problems whose solutions are unlikely to be found through RCTs. It’s also probably the case that when you try to solve one of these problems you create another one (like, removing budget constraints [massive aid] creates the wrong political incentives [massive corruption]).
So how do we get Good Things to Work? No idea. I am increasingly convinced that the solutions are not to be found in institutional engineering or technocratic competence, but I don’t know where to go from there. I would tell Burma’s government to do Good Things and that they will definitely Work, but there’s no way to get them to listen, and there’s no secret why not.
Tom Pepinsky 