There are some questions that you just can’t expect people to answer honestly: Are you smoking pot? Do you know how fast you were driving? Do you support voter identification laws, even if they suppress minority votes? Parents and the state police might be out of luck, but a recent study suggests there’s a good way to get around voters who don’t want to risk sounding racist: flip a coin.
Over the years, political scientists have developed a number of ways to navigate the fact that people likely hold opinions they’re not keen on sharing—or, in some cases, that they might even be afraid to share for fear of reprisals. In each case, the basic idea is the same: Give survey takers an out, a way to express support for a controversial idea without explicitly saying so.
Sixty-five percent of their sample had voted “no” on the personhood bill, yet, when asked directly, only 40 percent acknowledged voting against the initiative.
One of the ways to give people this out is through randomized response. Rather than have people answer directly whether they support voter ID laws, for example, first have them flip a coin in private, then answer “yes” if they support voter ID laws or if the coin flip came up heads. Since it’s a fair coin toss, researchers can, in theory, use the responses to estimate how many people support a controversial position, all without anyone ever directly stating their views. The trouble is, nobody is sure how accurate randomized response and other such methods really are. No one’s tested it out in the real world, and there are reasons to be wary.
Fortunately, Princeton University political scientists Bryn Rosenfeld, Kosuke Imai (who came up with randomized response), and Jacob Shapiro had an idea involving a 2011 Mississippi General Election ballot measure that would grant “personhood” to embryos at the moment of conception—a proposal that failed despite polls suggesting it was a shoo-in.
In the aftermath, Rosenfeld, Imai, and Shapiro asked 2,655 voters how they’d voted on the personhood question. They asked each one directly, but also used randomized response and two other methods, known as list and endorsement experiments, to investigate how survey participants had voted. The key here: The researchers knew which county each person had voted in, so they could compare how people from a given county said they voted with how people in that county actually voted.
Based on county-level records, the team estimated that 65 percent of their sample had voted “no” on the personhood bill, yet, when asked directly, only 40 percent of the people surveyed acknowledged voting against the initiative—a bias of about 25 percent. In contrast, randomized response had no discernible bias at all.
“While direct questioning leads to significant underestimation of sensitive votes against the referendum, indirect survey techniques yield estimates much closer to the actual vote count,” the authors write in the American Journal of Political Science.
Quick Studies is an award-winning series that sheds light on new research and discoveries that change the way we look at the world.
