Three reasons why polls are increasingly wrong

June 21st, 2015 at 12:58 pm

I found this to be an important bit of analysis from this AMs NYT. It’s a piece on three reasons why polling is increasingly failing to accurately predict electoral outcomes: the growth of cell phones, lower response rates, and greater difficulty predicting who are the likely voters (i.e., getting turnout wrong).

For reasons explained in the piece (related to an FCC ruling), it’s a lot more expensive to poll by cellphone and polling firms have been trying to make up the loss by cutting corners in sampling and interviewing.

For reasons that may also have to do with technology, response rates have fallen to the point where pollsters cannot count on ending up with a random sample. That in turn makes it hard for them to guess the probability that different types of people are in the sample, which implies greater guesswork to weight the sample up to be nationally representative.

Finally, you can ask people if they plan to vote, but that answer has long been known to have an upward bias. Apparently, it’s getting worse. I’ve thought for awhile that one reason polls seemed to be getting less reliable was due to the problem of accurately modeling turnout, and it turns out that’s true.

I was taught back in econ grad school that economists should work off of what people do, not what they say. (Unfortunately, there are too many economists who believe their models more than the evidence, but that’s a different problem.) So I’ve always taken poll results with a grain of salt. One doesn’t want to push that too far–when a Nate-Silver-style meta-analysis of many polls points in the same direction, I’d still give that some weight.

But for the most part, we should all add a few more grains of salt to the equation until the pollsters can straighten some of this out.

Print Friendly, PDF & Email

2 comments in reply to "Three reasons why polls are increasingly wrong"

  1. Kevin Rica says:

    Don’t forget that the secret ballot is part of the democratic process. We have the right to a secret ballot and polls are a violation of this principle.

    The secrecy issue is not a problem for the conventional and the conformist. In the old Soviet Union, it was considered patriotic to fill out one’s ballot under the nose of the policeman.

    But in this country, people who are aware that their votes do not conform to the opinions of their “betters” may not answer the polls honestly.


  2. Phil Perspective says:

    For reasons that may also have to do with technology, response rates have fallen to the point where pollsters cannot count on ending up with a random sample.

    Caller ID on the phone? How many pick up the phone, cell or landline, if the caller ID shows a number they don’t know?


Leave a Reply

Your email address will not be published.