Null Results Monday: A Partisan Endorsement Experiment on COVID-19 in the United States

Endorsement experiments are a common tool in contemporary political behavior research. One particular strength that they have is in testing the effects of “endorsements” on behaviors and attitudes as a way to see investigate how politics shapes mass attitudes. So for example, imagine reading the following sentence.

Many people put ketchup on steak in order to improve the taste.

and compare it to a slightly different version:

Many people, including President Trump, put ketchup on steak in order to improve the taste.

If you randomly assign people to read one of those two prompts, and then ask them questions such as “do you support putting ketchup on steak?” you can use this to tease out the effect of Trump’s “endorsement” by comparing across the ketchup-on-steak-by-Trump treatment and the ketchup-on-steak control. I have long thought (see here [PDF], especially pp. 433-435) that a key frontier for research in comparative political behavior is to translate that unbiased causal quantity (the effect of encountering the Trump treatment) into a substantively meaningful piece of political information, but that is a topic for another day.

As part of an ongoing project with Shana Gadarian and Sara Goodman on the politics of COVID-19 in the United States, we recently conducted a multi-arm endorsement experiment to test the effects of partisan and political endorsements on mass attitudes. So, for example, do people support different policies if you provide them with information that the CDC has warned about COVID-19 versus with information that the CDC that has earned bipartisan political support has warned about COVID-19? Does it matter if we also add that President Trump has downplayed the threat? Does this affect their trust in politicians?

What did we find?

That’s right, nothing. Broadly speaking, no statistically significant effect on any policy preferences or trust in governmental institutions or politicians.

You can read the full paper here. To channel MathNet*, the paper is not a fib, but it’s short.

The paper raises some important questions about endorsement experiments as well as about the politics of COVID-19. We do not conclude that there is no partisan politics of COVID-19. But we do suggest that priming experiments face serious obstacles when implemented at the same time as a national crisis is unfolding, an especially in a cacophonous media environment. And although we do not speculate much beyond that, it is interesting to ask what value endorsement experiments would have if we were to conclude they only produce statistically significant treatment effects about issues that are not immediately politically salient.


* Also, some good news: If you’re stuck at home with children and need some tools to help “teach them” math, all of the old episodes of Mathnet are on YouTube.