Kudos to Gallup for disclosing the process and perils of its likely voter model, but as Alan Abramowitz has noted at Pollster.com, something about the new USA Today/Gallup poll showing John McCain 4 points ahead among likely voters -- but 3 points behind among registered voters -- doesn't quite sit right:How do you get from a 47-44 Obama lead among RVs to a 49-45 McCain lead among LVs?
Whatever one thinks about likely voter models in general, the mathematics of this particular implementation defy credulity. Although, we should probably wait for USA Today to release its crosstabs so we can make sure there wasn't a typographical error of some kind in the write-up.
A few quick calculations shows how. You have 900 RVs and 791 LVs, so that means that among your 109 UVs (that's unlikely voters according to Gallup) Obama leads McCain by a whopping 61% to 7%.
Putting it another way, according to Gallup 16% of registered Obama supporters are unlikely to vote compared with only 2% of registered McCain supporters.
Also, this is a good time to mention Robert Erikson's critique of the extra volatility introduced by Gallup's likely voter model in past election cycles.
Monday, July 28, 2008
About that McCain +4
-- Nate Silver at 6:45 PM
Labels: gallup, likely voters, national polls
124 comments
Is it likely that John McCain is in fact 4 points ahead of Barack Obama? No. Would Nate being giving the poll this kind of scrutiny were the results reversed? Absolutely not. You can nearly always count on Nate to give extra scrutiny to any poll that shows Obama performing worse than expected. His bias is clearly that Obama shouldn't lose this election.
Any guess as to why RCP is using the "likely voter" figure rather than the "registered voter" one in their rolling avg?
I do statistical work for a living (I'm an econometrician), and honestly, what this looks like to me was an attempt to come up with an "objective" poll result most favorable to McCain.
I don't know if that was the intention here, but even if it was, such analysis has its place, I think, although not as a topline polling result. Essentially, doing something like that would be an attempt to quantify a "story" about how McCain can win the election. In this case, from this, that story basically boils down to, "McCain can win if enough Obama supporters stay home on election day", which, to me, isn't terribly interesting or insightful (or likely).
Those exact numbers may be incredulous but the general idea is right. Look at much of the democratic base: blacks, Hispanics, youths, poor people. All of those voter groups underperform at the ballot box.
Another question: why would you ever run a poll a not screen for likely voters? If you're going to poll people who do not vote, why not just ask Canadians or 10 year olds?
Looking beyond the (ridiculous) likely voter result, the fact that Gallup has one poll out that has Obama up by 8 among registered voters, and another poll out that was taken during the exact same time period that shows Obama up by 3 among registered voters should call the legitimacy of both polls into question, no?
Speaking as an Obama supporter, I think it's good to have polls like this sometimes. Otherwise a bored complacency could take over and there is no certainly no space for that. The polls are fun to follow, but they will be close for the next 3 months, and the election will be decided by enthusiasm and turnout.
Or maybe he's giving it scrutiny because of the peculiar nature of it as he writes about-and that McCain hasn't lead in any national poll in a very long time- so a lead of 4 stands out.
He has also been skeptical of +9 polls of late- and of the +12 and +15 national polls by Newseek when they came out.
Don't complain because McCain is losing.
Are you going to update the numbers for today based on the new national polls even though there are no new state polls (like you did over the weekend for Obama's bounce)?
Tim -- I think if you were a bit more neutral you might realize that Nate gives extra scrutiny to any poll that shows results far off from what other polls are showing. This poll is WAY off from other recent polls, even the Gallup daily tracking poll. It could just be small sample size, but that weird a result tends to suggest something not done quite right.
You go to great lengths to downplay any bad news the Obama receives.
Looking forward to visiting this website on Nov 5th.
F U GALLUP!
You are the same guys whose tracking polls goes up and down DEPENDING ON THE DAY OF THE WEEK!!!!
Paul O'Neill on Obama's team of advisors, hmmmmm.
Any poll that is not good news for Obama: IT MUST BE RIGGED! THE INTERNALS ARE WRONG! IT CAN'T BE RIGHT! SOMETHING IS WRONG!
Any poll that favors Obama: Factual polling, its 100% correct, internals are accurate. Great poll, great news.
Anon 7:08, please explain why Obama is way up in three of four Gallup polls.
I don't believe the McCain +4 result for a second.
But I do hope this serves as a wake-up call for the Obama campaign. They've been very lax in going after McCain lately. And with the media overwhelmingly biased in McCain's favor, they can't afford that kind of complacency. Hopefully this will push them to get back on offense, fast.
Even more to the point, Tim, Nate won't ignore the poll in his calculations, unlike, say, Real Clear Politics, which seems just to have ignored the Research 2000 poll, released yesterday, showing Obama up by 12.
It's certainly worth drawing attention to anomalous results, and, at this point, McCain +4 is pretty anomalous. But it'll get factored in, along with everything else.
The Obama up 3 with a MOE +/_ 4 and the Obama up 8 with a MOE +/_ 2 overlap at Obama up 6 or 7. Which is where Obama support likely actually is for all the noise.
Any outlier result deserves scrutiny. Either it is right and all the others are wrong or there is suptin funky goin on. Understanding why they give different results helps us interpret the numbers more intelligently.
I'm still trying to figure out why Nate is weighing the PPP poll in Ohio stronger than the Rasmussen. One is affiliated with a party, and one is independent. Take a guess which one is (D).
Neat site. Cool math.
Not very scientific.
"Would Nate being giving the poll this kind of scrutiny were the results reversed? Absolutely not. "
No shit, because the reverse is far more likely. Just because Nate is debating the polls specifics doesn't mean he's not right. Gallup put out two polls based on results from the same 3 days: one is one of Obama's best numbers ever, and one is Obama's worst number ever. Both of them can't be correct. Based on the preponderance of evidence (ie, every other recent poll), which do you think is the far more likely result?
JT - The PPP Ohio poll has a bigger sample size than the Rasmussen one, so it gets weighted more. Pretty simple.
JT, it's because the PPP poll has twice the sample size as the Rasmussen poll.
Bigger sample size = more accurate results.
"Bigger sample size = more accurate results."
Not necessarily.
Don--as far as I can tell, RCP always takes LV over RV when both are available. Likewise, it always takes "with leaners" over without when both are available.
I'd just finished posting this on the other thread when Nate posted this article.
Here's some more on the Prof. Alan Abramowitz v. Gallup debate about the recent Gallup flip-flop -- their daily tracker poll of RV with a sample size of over 3000 gave Obama a +8 lead, while their poll with USA Today released the same day with Likely Voters gave McCain a -4 lead. Now there's some feedback from Gallup:
"And Abramowitz notes that this isn't the first