There have been some useful commentaries on Indonesia’s electoral results, and some interesting speculation as to why polling results appear to some to be misleading. The general issues are two: PDIP did worse than many expected, and Islamic parties as a whole did better. Tom Power, for example, argues that the polls were “wrong” about Islamic parties. Marcus Mietzner notes that some polling experiments generated misleading conclusions about support for PDIP when prompting people to remember that Jokowi would be the PDIP presidential candidate (the implication is that these experiments were unrealistic, which is true).
Yet we should not expect that any one poll is exactly right. How much of a difference between a poll’s average and the actual vote percentage does there need to be for the poll to have been “misleading,” for a party to have done “better than expected” given “the polls”? Some simple statistical reasoning is clarifying here.
Polls are polls, they are not censuses. That means that they have margins of error, and that any predicted percentage supporting any party or candidate has a margin of error too. These are rarely reported—the margin of error for the entire poll usually is, but that’s not the same.
Fortunately, we can calculate these standard errors if we have the sample size and a desired level of statistical significance. I have done just this, using the sample size and results for the most recent Indikator poll and the most recent Saiful Mujani Research and Consulting poll. (Full disclosure: I know the principal of Indikator, Burhanuddin Muhtadi, and Saiful Mujani is a co-author of mine. See here.) We can compare the expected vote return and its 95% confidence interval to the results of the Kompas Quick Count as of April 10. Here is what we get.
The results help to clarify to some of the debates. Saiful Mujani did better than Indikator: the results for PDIP are within his poll’s 95% CI, something we certainly haven’t heard much commentary about. Moreover, among the Islamic parties, only PAN and PKS really did better than the polls had expected. It is tempting to focus on the results for PPP and PKB, but although these results higher than the point estimate from Saiful Mujani, they are still within that poll’s margin of error. They are only outside of the margin of error of the Indikator poll, which is interesting, but suggests that that poll just did not fare particularly well.
The point of this exercise is simply that polling results are designed to present “toplines.” We care about those. But to make inferences from them, they alone just won’t do.
Another interesting result for students of Islam in Indonesian politics is the total fraction of the vote going to Islamic and Islamist parties compared to previous years. (I consider Islamist parties to be parties whose platforms either now or at one time in the recent past called for implementing sharia law in Indonesia; those are the dark green. The light green are Islamic parties, rooted in Muslim social organizations but making no such call for sharia.)
It is indeed true that Islamic and Islamist parties are not disappearing. Anyone who has argued that is wrong. But on the whole, these results aren’t particularly stark in any direction. If anything, it’s the Islamic parties that have done a bit better on the whole than the past. But nothing transformative, at least not based on legislative vote shares. Greg Fealy covers some of the implications, but it’s not clear to me that there’s anything about the overall vote for Islamic and Islamist parties that needs to be explained.
Pingback: Final 2014 Polls and Predictions | Tom Pepinsky