Why You Shouldn’t Trust Polls

CNN has a poll that shows Mike Huckabee surging in the GOP race. Now, this may be true, but the poll itself is worthless as an indicator.

Here’s the key part:

The poll, conducted December 6-9, involved nationwide telephone interviews with 377 registered Republicans voters or independent voters who lean Republican.

Let’s count the number of problems with this poll:

1. The poll has only 377 respondents for a supposedly national poll. That’s a very low number of respondents and not a large enough sample to be statistically significant.
2. The poll includes independents who “lean Republican.” Exactly what does that term mean. Are they strongly Republican? Moderately Republican? Sorta-kinda Republicanish?
3. For that matter, how many voters who “lean Republican” would actually vote in a Republican primary? If you’re sampling a group of people who aren’t likely to vote in a GOP primary then your sample isn’t representative of the real shape of that contest.
4. The poll was conducted over a weekend. How many people care to spend time answering a pollsters questions on a weekend? Especially if you’re polling Republicans, who tend more often than not to be churchgoers, weekend polling tends to depress results. (Which may explain why the Democratic sample was 467 voters compared to the 377 Republicans sampled.)
5. The margin of error was 5 points, which is very significant when you’re dealing with so many candidates separated by such small margins.

Here’s the problem: polling at this stage in the race is worthless. You have to find the voters who A) actually care about the race at this point in time and B) actually have a reasonable chance of voting in a primary. That’s a crapshoot, which is why these polls have less than 500 respondents in their samples. That’s not enough to make a statistically meaningful measure, and when you factor in all the other problems you get a result that doesn’t say a whole lot.

It seems likely that Huckabee is doing very well. It could also be that the polls are worthless. As a case in point, look back to a snapshot of the Democratic race at this point in 2003.It’s interesting how much the 2007 Republican race resembled the 2003 Democratic race. None of the candidates had found their niche. There was talk of a brokered convention. The field was entirely up on the air. The primary schedule had been compressed which gave less time for a slow build as in past races. John Kerry was at 4%. He was polling behind Al Sharpton. Howard Dean was unstoppable. He had the organization, he had a group of supporters that were fired up, and he was cruising towards the nomination. By the end of Iowa, Howard Dean was toast and it was John Kerry cruising towards victory.

Could history repeat itself? Will the 2008 GOP race follow the dynamics of the 2004 Democratic race? Who is going to take the role of John Kerry? Will it be Fred Thompson, John McCain, or will Duncan Hunter suddenly rise through the ranks to take the nomination out of nowhere? (OK, so that last one isn’t going to happen.)

I have no clue, and neither do the pollsters. As much as we’d all love to know how things play out, the race is changing too fast for the polls to have much meaning. The polls give the campaigns some ammo, but if you really want to know what’s going on they’re about as good as reading tea leaves.