CNN Polls: How to Know a Bad Poll When You See It
Earlier this week we looked at a CNN poll that didn’t make any sense. It didn’t make sense because no matter how much critics may decry the shortfallings of opinion polls, trends are undeniable and CNN’s data simply didn’t fit.
As everyone knows, not all polls are created equal. As someone who studied political science and statistics for the better part of a decade, it’s clear that you can get misleading data if your survey methodology is off at all. And CNN’s was definitely off. So let’s take a look at what makes a bad poll a bad poll and a “good” poll a better poll.
First, the numbers. CNN has Rand Paul in first with 15 percent. Paul has not finished first in any poll since Mike Huckabee began to be included in polls and has been polling at 11-14 percent since. It’s not out of the question that Paul is at 15 percent but certainly Huckabee is in better standing than the 10 percent CNN shows him at. We know this because he has been polling at 16-18 percent since January and has led every poll.
CNN has Rick Perry in third place with 11 percent. That’s ridiculous because we have well over a year of polling data that doesn’t have Perry higher than 6 percent at best, 2-5 percent recently, and most companies have stopped even including him in the polls. The same goes for Chris Christie’s 8 percent. We have seen Christie at a very consistent 12-14 percent since the whole Bridgegate debacle, we know where he stands right now.
So how did CNN get it so wrong? Bad methodology.
For one, the poll only sampled 367 people. Considering this is a national poll, that’s a tiny number. PPP regularly samples at least 500-600 people while Quinnipiac has only been releasing polls with 800+ respondents.
But that’s not the only place where they went wrong. Since Republican primary voters aren’t only registered Republicans, polls include a number of “Republican leaning independents.” How many? Good question.
McClatchy, for example, samples 64 percent registered Republicans and 36 percent Republican leaning independents. CNN, on the other hand, sampled 50 percent Republicans, 50 percent independents. That’s an incredibly high number of independent voters, far more than the percentage of Republican primary voters that are not registered Republicans. In some states, you can’t even vote in the Republican primary if you are not a registered Republican. And apparently these independents really like Rick Perry.
(Image courtesy of Gage Skidmore)