Jay Reding.com

Why You Shouldn’t Trust Polls

CNN has a poll that shows Mike Huckabee surging in the GOP race. Now, this may be true, but the poll itself is worthless as an indicator.

Here’s the key part:

The poll, conducted December 6-9, involved nationwide telephone interviews with 377 registered Republicans voters or independent voters who lean Republican.

Let’s count the number of problems with this poll:

  1. The poll has only 377 respondents for a supposedly national poll. That’s a very low number of respondents and not a large enough sample to be statistically significant.
  2. The poll includes independents who “lean Republican.” Exactly what does that term mean. Are they strongly Republican? Moderately Republican? Sorta-kinda Republicanish?
  3. For that matter, how many voters who “lean Republican” would actually vote in a Republican primary? If you’re sampling a group of people who aren’t likely to vote in a GOP primary then your sample isn’t representative of the real shape of that contest.
  4. The poll was conducted over a weekend. How many people care to spend time answering a pollsters questions on a weekend? Especially if you’re polling Republicans, who tend more often than not to be churchgoers, weekend polling tends to depress results. (Which may explain why the Democratic sample was 467 voters compared to the 377 Republicans sampled.)
  5. The margin of error was 5 points, which is very significant when you’re dealing with so many candidates separated by such small margins.

Here’s the problem: polling at this stage in the race is worthless. You have to find the voters who A) actually care about the race at this point in time and B) actually have a reasonable chance of voting in a primary. That’s a crapshoot, which is why these polls have less than 500 respondents in their samples. That’s not enough to make a statistically meaningful measure, and when you factor in all the other problems you get a result that doesn’t say a whole lot.

It seems likely that Huckabee is doing very well. It could also be that the polls are worthless. As a case in point, look back to a snapshot of the Democratic race at this point in 2003.It’s interesting how much the 2007 Republican race resembled the 2003 Democratic race. None of the candidates had found their niche. There was talk of a brokered convention. The field was entirely up on the air. The primary schedule had been compressed which gave less time for a slow build as in past races. John Kerry was at 4%. He was polling behind Al Sharpton. Howard Dean was unstoppable. He had the organization, he had a group of supporters that were fired up, and he was cruising towards the nomination. By the end of Iowa, Howard Dean was toast and it was John Kerry cruising towards victory.

Could history repeat itself? Will the 2008 GOP race follow the dynamics of the 2004 Democratic race? Who is going to take the role of John Kerry? Will it be Fred Thompson, John McCain, or will Duncan Hunter suddenly rise through the ranks to take the nomination out of nowhere? (OK, so that last one isn’t going to happen.)

I have no clue, and neither do the pollsters. As much as we’d all love to know how things play out, the race is changing too fast for the polls to have much meaning. The polls give the campaigns some ammo, but if you really want to know what’s going on they’re about as good as reading tea leaves.

One response to “Why You Shouldn’t Trust Polls”

  1. Mark says:

    “Will it be Fred Thompson”

    Awwww, Jay still thinks Fred Thompson could win the GOP nomination. How cute!

    Primary and caucus polling is notoriously difficult, and voters are often fickle right till the very end, but just like election polling, the aggregate of all combined polls is pretty accurate. And while the numbers were off due to last-minute momentum, Iowa caucus polls in 2004 showed both Kerry and Edwards surging to the finish line while Dean and Gephardt were slipping. Similarly, McCain’s burst of momentum in New Hampshire showed up before the 2000 primary day just as Bush’s comeback was evident in South Carolina primary polls back in 2000. The exact numbers are seldom accurate, but chances are if CNN (or any pollster) is showing a surge for Mike Huckabee, you choose to “not trust the poll” at your own peril.

    Mike Huckabee represents a day of reckoning for Republican Party robber barons who have accepted without hesitation their marriage of convenience with populist, low-income religious radicals at the expense of the political mainstream. He is the end result of Karl Rove’s divide-and-conquer electoral strategy, espousing from his bully pulpit a hybrid of the politics of Jesse Jackson and Jesse Helms that represents total empowerment of the big-government evangelical conservatives at the expense of those filling the party’s campaign coffers who are now horrified at the monster they’ve created. That’s not to say Huckabee won’t win. On the contrary, I believe he and McCain would be nearly unbeatable against any of the top Democratic challengers. But it is to say that the Republican Party would metamorphisize into something unrecognizable to the Goldwater/libertarian wing of the party that gave rise to Reaganism if he does win. Again, thank Turdblossom.