How Does Language Impact Public Opinion Polls?

Our answers to survey questions have more to do with the wording than our actual ideologies.

Democracy depends on an understanding of the people’s will, but there’s one big problem: Oftentimes the people don’t know what they want. Take the Indiana “Religious Freedom” law that allows businesses to refuse service to same-sex couples. The law is sparking headlines and nation-wide protests, but, it’s damn near impossible to figure out what the American people think about the issue, because depending on the wording, one can easily get a majority of citizens to both support and oppose discrimination.

Based on the specific language, 67 percent of Americans “Oppose a law allowing refusal of service,” while 60 percent of Americans oppose a law “forbidding refusal of service,” according to two simultaneous national opinion polls I conducted this week.

(Poll: Gregory Ferenstein)

(Poll: Gregory Ferenstein)

The problem with asking people about the Indiana discrimination law is that they inherently fall victim to a very common error in survey methods: the forbid-allow asymmetry. Americans tend to loathe the government telling people what to do. For instance, many more Americans support both interracial marriage and gay marriage when asked if couples should be forbidden from getting married, rather than being asked if couples should be allowed to get married.

This is why over at Vox, Matt Yglesias reported on two respected polling organizations finding radically different outcomes on Religious Freedom-like laws, even though they were conducted the same year. Wording matters.

It’s damn near impossible to figure out what the American people think about the issue, because depending on the wording, one can easily get a majority of citizens to both support and oppose discrimination.

Aside from the importance of language, there are two other concepts at play here:

1). Americans are more concerned with government interference than the topic at hand. This is what I found with my discrimination polling experiment. The American people were opposed to any law, even though creating one would have opposite effects.

Interestingly enough, women were less immune to this effect. Women were 16 percent more likely to oppose allowing discrimination than men, but only three percent more likely to oppose forbidding it. In other words, women were more consistently against discrimination, regardless of wording.

2). Americans haven’t thought about the issue very much. For instance, when I conducted a poll about whether Americans wanted airlines to ban in-flight telephone calls, I got the same result, no matter how I worded the question. On this issue, Americans knew their preferences: no in-flight calls.

And without caring much about an issue, it’s easy to be swayed by language.

Related Posts