Particularly before the Newsweek poll came out last night, which showed an oversized, 15-point post-primary bounce for Barack Obama, there had been an apparent discrepancy in the size of the bounce that Obama had received in state and national polling.
Firstly, let's update our chart of Obama's numbers in state polling. What follows is a comparison of the Obama less McCain margin in all states that have been polled since the primaries concluded that were also surveyed by the same agency at some point in May:State / Agency May June Change
Across 29 state polls, Obama's bounce is 4.1 points -- down slightly from our estimate earlier in the week. The bounce appears to be roughly normally distributed; if you drew a histogram of the bounce in individual states, it would resemble a bell curve.
AR Rasmussen -24 -9 +15
KY SurveyUSA -24 -12 +12
OH Quinnipiac -4 +6 +10
KS Rasmussen -19 -10 +9
ME Rasmsseun +13 +22 +9
GA Insider Adv -10 -1 +9
FL Quinnipiac -4 +4 +8
NY Siena +11 +18 +7
WA Rasmussen +11 +18 +7
NH Rasmussen +5 +11 +6
PA Quinnipaic +6 +12 +6
WI Rasmussen -4 +2 +6
AK Rasmussen -9 -4 +5
IA Rasmussen +2 +7 +5
CA SurveyUSA +8 +12 +4
VA Rasmussen -3 +1 +4
MI Rasmussen -1 +3 +4
WA SurveyUSA +14.0* +17 +3
NV Rasmussen -6 -3 +3
WI SurveyUSA +6 +9 +3
FL Rasmussen -10 -8 +2
NC Rasmussen -3 -2 +1
NC Civitas -5 -4 +1
OH Rasmussen -1 -1 0
MN Rasmussen +15 +13 -2
MN SurveyUSA +5 +1 -4
CO Rasmussen +6 +2 -4
IA SurveyUSA +9 +4 -5
OR Rasmussen +14 +8 -6
==========================================
AVERAGE -0.1 +4.0 +4.1
* Average of all May surveys.
What about Obama's bounce in national polling?State / Agency May June Change
In national polling, Obama's average bounce has been 2.7 points. That isn't all that far away from our state polling bounce to begin with, but there are a couple of additional things to notice that make discrepancy even smaller.
US Newsweek 0 +15 +15
US Harris +4 +11 +7
US Economist -3.2* +3 +6.2
US Rasmussen -1.1* +5 +6.1
US USA Today +2.0* +6 +4
US AP-Ipsos +4 +7 +3
US Cook/RT +1 +4 +3
US Gallup +0.1* +2 +1.9
US ABC/WaPo +7 +6 -1
US Diageo/Hotline +4 +2 -2
US Zogby +8 +5 -3
US IBD/Tipp +11 +3 -8
==========================================
AVERAGE +3.1 +5.8 +2.7
* Average of all May surveys.
Firstly, the national polls that showed Obama regressing this month all had him at a pretty high number before. ABC/WaPo had him 7 points ahead of John McCain last month, Zogby had him 8 points ahead, and IBD/Tipp had him 11 points ahead. The most recent version of each of these polls all showed Obama regressing. But really all Obama may have been doing is regressing toward the mean. Does anybody really believe that Obama was ahead by 7, 8 or even 11 points last month? That's certainly not the impression one was getting from the state-by-state polling results we were seeing in May. If Obama wins this election by 11 points, you will see things happen like him winning Pennsylvania by 16 points, or winning states like Texas, Arizona and Kansas, or winning Florida by high single digits. Those aren't the sorts of results we are seeing now, and they certainly weren't the sorts of results we were seeing in May.
So let's instead focus only on those pollsters that release national polls on a weekly-or-better basis. This means Rasmussen and Gallup, which release numbers daily, and Economist/YouGov, which releases numbers weekly. These polls are going to be far less subject to problems with small sample sizes than surveys that are conducted just once a month.State / Agency May June Change
These three polls show an average bounce of 4.7 points, barely different from our finding in the state-by-state results. If you exclude the Economist's poll and focus only on the two daily trackers, Obama's present bounce is 4.0 points, which is even closer to the target established by the state polls.
US Economist -3.2* +3 +6.7
US Rasmussen -1.1* +5 +6.1
US Gallup +0.1* +2 +1.9
===========================================
AVERAGE -1.4 +3.3 +4.7
* Average of all May surveys.
To the extent that one is going to use national polling results to divine trends, one ought to give a relatively large weight to the two national tracking polls. Gallup and Rasmussen are each surveying about 25,000 voters per month for their national trackers, as compared to a once-a-month survey like ABC/WaPo, which might poll one-twentieth that many people. Although there are some diminishing returns associated with large sample sizes, it is also not a case where all polls should be treated equally.
Saturday, June 21, 2008
Obama's bounce in state vs national polls
-- Nate Silver at 2:55 PM
Labels: bounces, national polls
25 comments
Would it make sense to weight the state polls by their relative population rather than calculating a straight mean value?
@15:37. You certainly could, but it doesn't appear to make much difference. If you weight the states based on their 2004 turnout, the average bounce is 4.2 points rather than 4.1
Nate: is there a certain time of day when you update your numbers? Or do you update on an as needed basis?
Nate,
A question regarding your use of national general election polling data - what are the pollster ratings for the national polls? Are these in use in the current model?
Also, are you going to use polls of states that have natural disaster strike during the polling?
Anon: I think that's been answered before here. The polls don't come out at any fixed time of the day, but generally after noon if the pollster wants to get evening news. Nate generally updates in early to mid afternoon. But if a significant number of additional polls come out later, or if there is some advanced indication of that, he waits or, on occasion, issues a second daily updated -- an update on the daily update.
It doesn't make sense to tie himself to a fixed time of the day. He has an interest in updating as soon as reasonably possible but to wait at least until a large majority, if not all, of the day's expected polls are published.
But he doesn't know in advance when many of the pollsters are going to release stuff.
@15:48. No, there's no particular time. It just depends on when the polls come out, and when I'm around my computer.
@15.58. I don't directly use the national polls in calculating the averages. Instead, I use them as *part* of the dataset that we use to calculate the trend/timeline adjustment. All polls are treated equally for purposes of calculating the timeline adjustment, although since there are a lot more data points available for the national tracking polls, they tend to have a lot of influence over the model.
If you're looking at the change from one month to the next do the 24000 people in the middle really matter? All that really matters is the sample size of the first and last polls.
Gallup's poll is less likely to be way off because over three days for their sample they survey a large number of people, over 2500. On the other hand the Newsweek poll is just over 1000 people with a MMOE of 4%.
The STDErr of a difference is sqrt((p(1-p)+q(1-q)+2pq)/n). For a poll of 1000 respondents that puts the MMOE of the Difference at 6.2%. Increasing to 2500 respondents reduces that to 3.9% and they approach 0 as the values approach 1 or 0 from .5. In any case, over 1/3 of the 15 points can be explained by potential error. Given the other results and what we know I doubt the 15 points is 6% low (but oh how I dream).
Now, here's a question. Given the voting population of each state and the state polling data. Does the state polling data match up with the national numbers?
Glad you added more polls. But is it a bounce if Obama didn't gain in his % of supporters, and McCain lost support to undecided?
Can you show the increase/decrease in Obama %, increase/decrease in McCain %, and increase/decrease in undecided % for these polls too?
Much of the bounce is McCain losing support to undecided as much as Obama gaining, and this is loss when you just look at the margins and ignore undecideds.
As usual, Nate, your data-driven analysis blows everyone else's guesses clear out of the water. Bravo!
More implicit bias covered up with 'objectivity' in this post: You artificially divide May and June Rasmussen, Economist, and Gallup polls together.
But Clinton didn't pull out of the race until the weekend of June 7. The dividing line should be before June 7 and after June 7. Rasmussen and Gallup have recently shown a tightening of the race (Obama +2).
If you take averages, why not do pre/post June 8 or June 9?
I don't know about the Economist, but Rasmussen and Gallup have generally had Obama around 45-47%, stuck there, while McCain has declined to the lower 40s since June 7. So is there a bounce? Or is there a McCain decline toward undecided?
Be more objective than you are with your analysis.
Previous commenter said "Nate: is there a certain time of day when you update your numbers? Or do you update on an as needed basis?"
He usually posts and updates when the polling data is best for Obama. That way, the most objective analysis looks good for his preferred candidate. Case in point: he had the old 'state bounce' article up, but ignored polls like Rasmussen Ohio showing no bounce, until he finally did this new post, but only once other state data came out looking good for Obama.
Timing is everything.
Bull crap.
All polls show up when they show up. If they arrive two late in one day the show up in the next.
... of course, Anon@4.41, that only works as long as there keeps being good news for Obama every day. Which just happens to have been the case for every day the last couple of weeks.
Following up on the last comment suggesting not to look at the margin, in case McCain has lost to undecided but Obama has not gained, there might be something to that.
Just look, for example, at Gallup tracking data, which I got from this site:
http://www.gallup.com/poll/108208/
Gallup-Daily-Obama-46-McCain-44.aspx
There is no bounce toward Obama since Clinton dropped out of the race and conceded. She did this on June 7.
If we look at Obama support up until and including June 7, his average Gallup tracking support % is 45.70%. If we look at Obama support after June 7, his average Gallup tracking support % is 46.27%.
This is an increase of less than 1% in support for Obama.
This isn't the margin of support. Any bounce found in this poll is due entirely to McCain losing support to undecided, not due to Obama gaining.
No Obama bounce in Gallup. There is an undecided bounce away from McCain though.
Why not update and report on this too, even if it doesn't look as good for Obama as your analysis based on margins?