Jay Reding.com

Polling 101

Michael Barone has an excellent piece on the way that polling works (or doesn’t) in the 21st Century. It’s looking increasingly like the old techniques of political polling aren’t coping well with technological change. Voters are less likely to have landline phones, and less likely to have time to bother with answering a pollster’s questions. Barone casts some doubt on the results of many of the polls that have come out for this election cycle:

If you could go back in history and conduct polls, I don’t think you’d find any, and certainly not many, two-year periods when the balance in party identification shifted from even to having one party 12 percent ahead of the other.

At this stage of the campaign, pollsters try to screen their respondents and report only those who answer a series of questions in ways that suggest they are actually going to vote. Many polls find that a higher proportion of Democrats than Republicans pass the screen. Others find similar proportions do. But pollsters of both parties will admit that polls do a poor job at projecting turnout.

I don’t buy for a second that the Democrats have shifted partisan ID by twelve points in two years. If that were true, the Democrats would be ahead in nearly every race by huge margins. However, the balance of power in Congress remains close — even if the Democrats win, it seems unlikely to be by more than a handful of seats.

It could be that the problems with polling is due to methodologies not keeping up with the times. It could be due to unconscious bias. It could be unconscious bias. Whatever it may be, the polls seem to be diverging more and more from the actual face of the electorate. When we’re talking about a potential 12 point bias towards the Democrats in an election that is already looking to be quite tight, it’s entirely possible that Election Night could be quite surprising…

6 responses to “Polling 101”

  1. Seth says:

    As usual, you are listening to people who don’t get it. For the purposes of a poll or voting behavior, it doesn’t matter one bit whether people are Democrats or Republicans. It matters whether they say they are Democrats or Republicans.

    With the party affiliation, it doesn’t matter whether people are Democrats or Republicans. It matters how they identify themselves in their heads–and that certainly has an effect on voting patterns.

    Given the climate this year, given that people want Democrats over Republicans by 15-20 points in generic Congressional ballots, and given that most people think the Democrats are going to be big winners this year (it’s not uncommon for 60-70% of people to say they voted for the winner of a race that was 51-49%), a 12 point swing in how people identify themselves doesn’t seem out of the range of possible at all.

    Of course, then we actually read that article and look to where it says 5-12% advantage for Democrats. We don’t know whether that 12% was an outlier and most are only looking at a 6-7 point advantage for Democrats or what.

    Sometimes I think you just post things so I can tell you what’s actually right.

  2. Splashman says:

    And we should trust that “Seth” is not pulling opinions out of his fanny because . . . ?

    Given recent history (2000, 2002, 2004), there is reason to expect the numbers to be somewhat optimistic on the D side. Exactly *how* optimistic is unknown, of course, but vague assumptions about the “climate” (tendered by the MSM and parroted ad nauseam by desperately hopeful libs) always point the same direction. Funny how that works.

  3. Mark says:

    Barone and Reding are right on this much. Most mainstream polls are oversampling Democrats to derive their “generic ballot advantage”. Considering that in the GOP landslide of 1994, Republicans had a mere nine-point generic advantage, there is zero chance the Dems will pull off a generic advantage of 13 points or more like most media polls are suggesting.

  4. Mark says:

    So Jay, when are you gonna stick your neck out and make some predictions, showing those pollsters and paid experts how it’s really done?

  5. Dave says:

    All polling whether for product research to political polling is becoming less reliable as fewer people will respond to polls and become disconnected from landlines since most polling is done via the telephone. Currently the biggest segment not being polling is the younger ages groups since they are the most committed to using the new technologies. This is having a big effect on polls and thier results.

    Additionally many who do repond to polls will give responses that are completely divergent from thier true opinions. Think about all the telemarketing games some people play(ed). Or the response to the ability to block telemarketers. Generally most of us just do not want to be probbed about out thoughts and feel no social pressure to do so or to be honest as was often the case in the past.

    Dave

  6. Jay Reding says:

    Mark: I do have some predictions I’ve been working on – which I won’t have occasion to post until at least the end of the week. Damnable legal memo assignments…