2012: The Death of Landline Election Polling?

By Emily Hoerner
IowaWatch.org

The landline phone is in decline in today’s social-media driven world, and going with it may be traditional political polling techniques. Experts including political scientists, social scientists and pollsters say a need for change in those techniques is both necessary, and on the rise.

Scott Rasmussen, founder and president of Rasmussen Reports, said his polling firm already is compensating for new technology.

“The election we just had is the last election of the telephone polling era,” he said.

Rasmussen said telephone polling worked for decades because speaking on the phone was the prime mode for communication. However, social media, texting and email are vital forms of communication today, and often used more frequently than old-fashion phone calls, he said.

“Calling you on your phone doesn’t solve the problem of reaching people. That’s not the way people normally communicate,” Rasmussen said. “We need to find ways to reach people in social media.”

Rasmussen’s poll received much flack this year, after being off on average by 5.8 points, a much higher number than most other polls this year, according to The New York Times’ Nate Silver.

Emily Schilling, a graduate assistant at the University of Iowa who works with the Hawkeye Poll — a public opinion survey that gathers election data in Iowa and other Midwestern states — said her survey’s inability to contact young people caused large errors that led to not publishing results from Wisconsin and Ohio.

“Our big issue was the landlines, and not getting a representative sample of 18- to 24-year-olds,” she said.

The Hawkeye Poll, operating out of the state that kicks off the presidential election with its precinct caucuses, used only landline phones. Schilling said having a list of cell numbers would increase the accuracy at least a little bit.

A large number of polls were accurate in the 2012 election cycle, even with the use of telephone polling.

Doug Usher, managing partner at Purple Insights — the research division of Purple Strategies in Washington, D.C. — said many of the problems pollsters ran into were because of ideological bias.

“That’s not an indictment of calling, that’s an indictment of the people who are running the polls,” he said.

But decline in landline use still causes problems.

DEALING WITH THE DATA

Simon Jackman, a political scientist at Stanford University, said low telephone survey response rates require pollsters to weight data to “rehabilitate” missing responses by unreachable parts of the electorate.

“I’d agree that we may be getting to a point in 2016 where the statistical tricks, it just becomes too hard a problem,” he said.

Experts IowaWatch interviewed agreed that the answer lies somewhere between moving away from the landline to a totally new form of polling. “It’s sort of great that the next presidential election is four years away, we’ve got time to figure this out,” Jackman said.

But not all polling experts are on the same page.

Usher said his firm didn’t need to apply a lot of weighting to its data.

Purple Strategies was named the fifth most accurate national poll in 2012 by a study conducted at Fordham University.

Usher said Purple Strategies used a mixture of techniques in data gathering, like Web based polling for people only using cell phones.  However, he said, he isn’t ready to hang up on telephone polling just yet.

“I have a bit of a mixed mind on this,” Usher said. “I’ve been doing this since 1999, and every year people have been telling me it’s the last year.”

Usher said failing to account for the decline in landline use and the proliferation of cell phones can create mistakes in the data. Things like interactive voice recordings are good for gathering data gathering, as long as they are combined with other methods to make up for its shortfalls, he said.

Cell phones especially add problems to the phone surveying process.

FOR THE CHATTY CATHY’S

Cell phone numbers change frequently, and using a computer to auto-dial them is illegal, Jackman said.

“You’ve got to have a person calling those numbers manually. That’s just really expensive,” he said.

Much landline phone polling is done with random digit dialing and interactive voice recordings.

Phone number lists for cell phones are more expensive than those for landlines. Schilling noted that many people have cell phones with area codes that differ from where they live, and asking them where they are might be an invasion of privacy.

With the troubles of obtaining gathering poll data from cell phones, the next step may be a combination of techniques.

“I think we’re headed towards multiple modes,” said Michael Schober, dean of the New School for Social Research in New York City. “We’re going to have to really allow options that are most convenient for when they (respondents) are willing — and how they are willing — to respond.”

Some pollsters and political scientists are looking for the answer to these polling problems on the Internet.

FOR THE WEB SURFERS

“There are a lot of different flavors,” said Fred Conrad, a professor and survey methodologist at the University of Michigan.

He said online surveys probably are the next step, but big differences exist in their sampling techniques. The difficulty is that pollsters can’t randomly generate email addresses like they can with phone numbers, he said.

“There is a concern that Web surveys are not representative, so the results won’t be predictive,” Conrad said.

Some 2012 polls used volunteer panels. Conrad said these respondents choose to participate, instead of being randomly selected.

Jackman said the difficulty with Internet political polling is reaching a wide array of people. “So much is bound up in trying to get this right,” he said. “Politics is sort of interesting because you’re not after 30-year-old women who’ve gone on vacation.”

The research doesn’t end there.

FOR THE ‘C U L8R’ CROWD

Even though texting has been around for 20 years, few pollsters have yet to try using it as a survey method. “It is surprising that texting is such uncharted territory, given how widely used it is for communication,” Conrad, who has researched text communications, said.

He said texting has many advantages, one being that it is associated with telephone numbers. And there are certain subgroups of the population, like 20-year-olds, who likely text more than they speak on the phone, Conrad said.

Texting gives people the feeling that a survey is anonymous, even though it isn’t, because there are fewer cues reminding the respondent that a real person is on the other end of the survey, he said. “People do really seem to provide more truthful, embarrassing information,” Conrad said.

Which is, in effect, a bias in the data.

NO ESCAPING THE BIAS

Different types of interviewing processes create different types of response biases.

Jackman said having an interviewer in the room tends to produce a more sanitized version of the person being questioned. He calls it the Readers Digest version of the person: “’Of course I’m registered to vote, of course I’m voting in November. And we have a technical term for that: social desirability bias.”

Jackman said respondents who are not with an interviewer are more likely to be honest about socially uncomfortable things, like racism.

That is not true for everyone, though. Take away the interviewer for some, he said, and “it just puts people off. A lot of people just hang up.”

Julianna Pacheco, a political science professor at the University of Iowa who studies public opinion, said, “Even with landlines, there is always going to be bias.” She said it always is more difficult to reach certain groups, like college students, who probably don’t have landlines, and elderly voters in nursing homes.

Online polls provide biases as well, because not everyone has available access to the Web. And computers still are more concentrated among the educated and affluent, Pacheco said.

Jackman said, “There is no free lunch here. If there was a magic bullet technology out there, we would be doing it.”

SWEETENING THE DEAL

Another suggested solution to biases in the responses is offering incentives, which some pollsters, like Usher, used this election cycle.

Purple Strategies pays sample vendors for lists of people who, in some cases, are given things like rewards program points and frequent flier miles in exchange for answering poll questions.

But, giving rewards for survey data could bring about biases as well. Schober said a tricky question is whether respondents being rewarded give true data, or if they give answers they think the person conducting the survey wants to hear.

Incentives for survey results also begin a whole new debate about what should be expected of the public, Schober said.

“We’ve been assuming that it’s part of people’s civic duty,” he said, about participating in polls. “That’s just part of being a good citizen, if you’re sampled it is important for the country…thus far, people have been willing to answer questions.”

Schober said he doesn’t think academics and pollsters agree on what types of surveys should merit incentives, or what they should be.

Amidst all of these problems, none of the polling experts IowaWatch spoke with believe polling will become forgotten. Instead, Rasmussen said pollsters will press on, testing innovative ways to deal with these technological changes.

“This is the way you figure out things in the polling world,” he said. “You test it, then compare it with reality.”

This project was produced by Iowa Center for Public Affairs Journalism-IowaWatch.org, a non-profit, online news Website dedicated to collaborating with Iowa news organizations to produce explanatory and investigative work.