Zogby Interactive has polling detail out in 34 states.
Zogby's regular polls rate as middle-of-the-pack and get beat up upon a little more than they probably should. His interactive polls, however, are not so good. We give them most skeptical treatment possible by not regressing Zogby Interactive's results to the mean when computing their pollster rating. But they will be included, weighted approximately one-quarter as much as regular polls. Some of his states have huge sample sizes, but as you can see from the chart above, our ratings have figured out that big sample sizes can't cover for questionable methodology.
For what it's worth, I'm actually more sympathetic than you might think to the idea of Internet-based polling. Something like 20 percent of Americans are cellphone-only now, and many others use all sorts of call screening technology that pretty much ensures that they will never be contacted by a pollster. These problems are going to get worse before they get better, and Internet-based polling may eventually need to be part of the solution.
But I don't know. There are sure are some strange results in here, such as Bob Barr's numbers being fairly high in many states, as well as Obama's performance over parts of the South, where Internet usage rates tend to be lower, thereby making a quasi-representative sample more difficult to obtain.
7.09.2008
Zogby Interactive Data Dump
@ 8:55 AM
Subscribe to:
Post Comments (Atom)
116 comments
It would seem to me that these polls are useless standing alone, but I think they are useful in the averages to offset the lack of cell phone users (the youth vote). With the weighting you have used (which looks about right to me), I would say that adding these polls will improve the overall numbers. There's no doubt to me that youth turnout will be higher this year and this will help reflect that a little better.
Nate,
Any idea when these will get worked into the charts, maps, etc? I know it is a fairly large dataset, but it is also the first update that we get in the last four days. For us junkies, that's a long time.
Since they have small weighting they shouldn't show much more than a nudge, but still all data is intriguing.
You once referred to NV as having one of the lowest rates of internet usage per capita, it would seem useful with Zogby Interactive data to take a look at internet usage per capita in each of the states that they have released, as it might give us a further indicator of accuracy or bias on this set.
As always, great work.
I have had no respect for Zogby's interactive polls. Their results in the last cycle were particularly awful. However, looking through these numbers, most of them seem fairly close to the mark, and much better than what Zogby was putting out for the 2006 races. Maybe he has perfected the art a bit better.
To be totally honest, I happen to think that Zogby's regular polls are pretty crummy.
Oh Nate, don't factor these in! Zogby is totally and utterly useless.
Bleuughh. What a mess. Would've been nice to look at if this was a bunch of SUSA/Rasmussen polls, but as it stands, I look at this and just see a steaming pile. You'd do good not to give any weight to these results.
Mark, I don't know if I agree that these are on the mark at all. I wouldn't question Nate's decision to include them with some small weight, I guess.
But Obama leading NC by 9? Obama winning Arizona and South Carolina? McCain only winning Texas by 3? These might be the eventual results after a lot of McCain gaffes or some other disaster in his campaign, but they certainly don't look realistic in the campaign as it is now.
There are just so many problems with these polls. Honestly, I'd prefer you didn't use them at all - they just cloud things.
On Realclearpolitics, that just averages the most recent polls, McCain is about to take a big jump, because pretty soon that LA Times Bloomberg travesty will fall off.
How does the time since the outliers like LA Times and Newsweek, affect your national numbers Nate? Its obvious on their site, add them up and divide by the number of polls.
Your sight obviously uses polls and some sort of regression, because very few polls have Obama at or above the 50% Pop vote mark, and yet that is where you have him.
I wish I understood the national better, especially from your old chart that shows a 5%+ win in the popular vote, yields all this electoral college trivia useless, i.e creates a landslide type victory.
Obama winning Arizona? Obama winning South Carolina? And only up by FIVE in Oklahoma and Kentucky? I roll to disbelieve.
Hard to believe Obama's beating McCain in Arizona.
The problem with Zogby Interactive isn't that it's internet-based. The internet is just a communication medium, after all.
The problem with Zogby Interactive is self-selection. They only poll the people that've signed up and even then there's no requirement to participate.
It's totally useless and I'm surprised you'd even consider using it.
anon@8:24,
Those results are pretty screwy, I agree, though one of the two polls done of SC had McCain up by only 3, so Obama up by one isn't totally out of line.
A lot of the other states look pretty reasonable (CA, FL, NY, MA, CT, IL, CO, IA, MN...).
There are some real inconsistencies, of course. If Obama were really leading in AZ, and ahead by 16(!) in NM, he'd be ahead by more than 2 points in CO.
I'll be interested to see what effect these numbers have on the overall results of the model!
On Realclearpolitics, that just averages the most recent polls, McCain is about to take a big jump, because pretty soon that LA Times Bloomberg travesty will fall off.
I've been watching this too and I think you are mistaken. The current average is 5.5. If you just remove the 12.0 poll, you get 21/5, which is 5.2. Not a big jump.
Furthermore, look at the graph. The spread has been shrinking as those two junk polls fall off, but Obama's trend is still upward. Obama is in the lead and putting more distance behind him.
Is there any evidence that Cellphone People (of any demographic) poll differently from Landline People? Or that Internet People are more representative of the population as a whole?
Also I would assume that online polls like this adjust for age and such, but even so the sample sizes for some segments of society are going to be small and unrepresentative.
I agree with most posters. These figures are trash.
Tnankfully there are some real polls out later today. Rasmussen will release polling for Missouri and New Jersey. Their previous polls showed Obama leading by 1 points and 9 points respectively and I'll be looking for any significant movement from those figures.
If anyone has any intelligence (!) as to what and when Survey USA and Quinnipiac might release this week, it would be good to know.
Picking and choosing which poll results are realistic--based on your expectations--isn't a very scientific methodology. If we already know that Obama will lose South Carolina by 15, then what's the point of the polling? (Rhetorical question please don't answer.)
Yeah, taken individually, most of these results seem to be in line with previous data (except maybe in the Southwest; who, exactly, projects Arizona for Obama?) But honestly, Obama by 14 in Michigan and a virtual tie in Indiana and Colorado? I don't buy it.
I'd also like to point out that, in my experience, both big- and small-l libertarians (I'm the latter) of all ages tend to have a greater online presence than other demographic factors would suggest. Which helps explain the "Barr factor."
@Haldfan,
He bases how much weight to give a poll on how well said poll did in past elections, it's not arbitrary.
Nate,
Zogby claims rigorous methods to weight responses, but the results in some states are so far removed from conventional wisdom.
Conventional wisdom may be wrong, but I would offer very long odds against McCain losing AZ. In fact, I would expect, no matter what, McCain will win AZ by at least 8%.
Perhaps your weighting for the Zogby Interactive polls should factor in divergence from the regression model? While using the regression model for this purpose is not exactly a rigorous method, it just feels like a better way to weight this type of poll.
Nate, you choose to discount the Zogby polls based on the idea that they are internet based?
I'm not sure if you are aware of the polling company YouGov, they’re based in the UK and have consistently beaten out Gallup, ICM and Mori in predicting the results of UK elections. For example, YouGov proved the most accurate in the two previous UK general elections as well as the London Mayoral race.
In fact UK bookies (who have the most to lose if their predictions are wrong) currently consider YouGov as the main basis for odds setting.
Nate,
Add me to the chorus of those who object to inclusion of Zogby's "interactive" polls AT ALL!
There is simply no way to compensate for the self-selection bias in the sample. And Zogby's snarky efforts to divert attention from it by inclusion of huge numbers of respondents is no different from the infamous Literary Digest "poll" of 1936. His utterly illegitimate inclusion of a "margin of error" only adds to the dishonesty of his results.
Zogby has a long history of tailoring his polls to support the agendas of his clients, especially in terms of question wording. That, alone, should raise suspicions about his firm's methods. Considering that his claims of "rigorous weighting" to compensate for sample bias are not detailed, there is a reasonable suspicion that they are nothing more than an effort to make the results of his internet polls "look reasonable."
Zogby's telephone polls may deserve to be included. His continuing efforts to promote his "internet" polls should be rejected out of hand. It is a marketing effort, not a serious effort to measure public opinion.
I read a study somewhere a few years back that said that fully two-thirds of internet users have Democratic views, and of the one-third that don't, a significant chunk is of the libertarian bent. If true, I would think this would be more of a hindrance to getting a good online sample than regional concerns.
As it is, these numbers are way too friendly to Obama as a whole. I have a hard time buying he's within 5 in Oklahoma, he leads by 1 in South Carolina, within 5 in Kentucky, etc.
Personally, I would ignore them completely, but I'm glad you're not giving them full weight at any rate.
Looks like junk. I am on the panel and was never polled in my state.
Hardly worth commenting on, others have said it. I can't resist noting Obama by 9% in NC and 14% in NM?
Using these polls at all that so obviously show a pro-Obama bias only convinces me that this site exists only for masturbatory pleasure for the Obamaphiles out there.
I came here for some hard facts, but am becoming disillusioned by the statistical bias that is being built into the reporting/modeling.
Having said that, interesting poll on Rasmussen today: Voters see Iraq as biggest difference between the candidates.
I think the internal numbers are bad for Obama. He is seen as the candidate most likely to raise taxes, while McCain is the one seen as most likely to control spending. Also by 20 points the voters see McCain as having the view that America is a fair and decent society. Only 45% think Obama shares that view. If the people really have that assessment of Obama he has got some work to do before he passes a crucial credibility test for a Presidential candidate: Love of Country.
McCain is of course seen as the candidate who will finish the job in Iraq (76%) versus bringing the troops home (Obama 78%). What the poll does not update is the popular view as which should be the priority. From what I can see Rasmussen last polled on that issue in May before there was any reporting on the success of the Surge.
Therein lies the rub for Obama. As the potential, if not the actuality, of victory in Iraq becomes manifest to the people, they will shift their mood from one of pessimistic quagmire to let's get 'er done. If that happens, Obama will have an uphill slog to cobble together an electoral majority: he will have lost the raison d’être for his candidacy.
At that point all of his tacking to the right on social issues will work to further defeat him as his base begins to crumble.
Indeed there are already signs of it. The feminist wing of the party is far from being convinced and they represent the largest voting bloc available to him. Read Marie Coco's piece on yesterday's RCP website.
There are already rumblings that he is financial trouble. Look for more.
The race is tightening.
I'm not going to pretend to be an expert on methodology, but I'm wondering if the polls don't represent another phenomenon: the relative softness of McCain support. I.e., Obama voters have by and large -- esp. in southern states -- already made up their mind, but conservatives and moderates who normally vote Republican have not made up their minds.
Look at the numbers for just Obama. They are entirely plausible, while McCain's numbers look too low in almost every state.
Consider South Carolina: African-Americans were 30% of the electorate in 2004 and Kerry won only 85% of them. Let's expect that number to jump to 90% of a 33% share, and Obama is looking at 30% of the statewide vote right there. Even if he only gets 20% of the rest of the vote, he'll be hovering around 45%. This poll shows him at 42%.
Again, not defending the methodology of the poll, but other polls and much of the commentary on this site and elsewhere has not reflected a clear understanding of the math here in the South.
I'm not going to pretend to be an expert on methodology, but I'm wondering if the polls don't represent another phenomenon: the relative softness of McCain support. I.e., Obama voters have by and large -- esp. in southern states -- already made up their mind, but conservatives and moderates who normally vote Republican have not made up their minds.
Look at the numbers for just Obama. They are entirely plausible, while McCain's numbers look too low in almost every state.
Consider South Carolina: African-Americans were 30% of the electorate in 2004 and Kerry won only 85% of them. Let's expect that number to jump to 90% of a 33% share, and Obama is looking at 30% of the statewide vote right there. Even if he only gets 20% of the rest of the vote, he'll be hovering around 45%. This poll shows him at 42%.
Again, not defending the methodology of the poll, but other polls and much of the commentary on this site and elsewhere has not reflected a clear understanding of the math here in the South.
I'm not going to pretend to be an expert on methodology, but I'm wondering if the polls don't represent another phenomenon: the relative softness of McCain support. I.e., Obama voters have by and large -- esp. in southern states -- already made up their mind, but conservatives and moderates who normally vote Republican have not made up their minds.
Look at the numbers for just Obama. They are entirely plausible, while McCain's numbers look too low in almost every state.
Consider South Carolina: African-Americans were 30% of the electorate in 2004 and Kerry won only 85% of them. Let's expect that number to jump to 90% of a 33% share, and Obama is looking at 30% of the statewide vote right there. Even if he only gets 20% of the rest of the vote, he'll be hovering around 45%. This poll shows him at 42%.
Again, not defending the methodology of the poll, but other polls and much of the commentary on this site and elsewhere has not reflected a clear understanding of the math here in the South.
I never thought I'd be caught out defending Zogby, but:
Despite the self-selection problem, every result on there could be defended, if you assume a large margin of error, of about 8.5% or so. In fact, if you about double the normal margin of error Zogby isn't THAT awful.
If you look at the pollster accuracy chart, you see a error of 5.73% between Zogby's predicted results and the actual results. Unless there's some increased self-selection problem with the general as opposed to the primary campaign, I'd expect the real error margin isn't much more than that now.
So, these are useful data-points. As long as they are not given the same weight as ordinary polls.
People here focus too much on individual pollsters. The more data is added to the model the more robust it is. So, Zogby is entitled to SOME weight, just not a lot.
Since Zogby's margin of error is about 3 times that of the Average pollster (including Zogby!) i.e. 5.73% versus 1.97%, Nate is actually being VERY CONSERVATIVE in giving them a 1/4 weight, instead of a 1/3 weight.
Matt,
I think YouGov's approach has considerable promise.
Halfdan,
I'm pretty sure there have been studies that show that cellphone-only people have lower income than average. It wouldn't surprise me to learn that landline-only people also have a lower income than average. But these are probably different demographics; my entirely uninformed stereotype is that cellphone-only people tend to be younger and landline-only people tend to be older.
Also, it wouldn't surprise me to learn that cellphone-only people move more often. When you move, you have to call the phone company and so on to get a new land line, but you can keep the cell phone you already had. The obvious effect here is that they might be less likely to be registered to vote. In the case of students, they may be registered to vote in their home states. Second, this means that cellphone-only people are the sort of people who move often; I'm not sure what this translates into politically.
Finally, it occurs to me that polling cell phone users might be complicated by the fact that people often have cell phone numbers with area codes that aren't in the state they reside in. But that might not be as much of a problem as I think it is -- I'm a graduate student at a university that attracts people from far away, so my view is a bit skewed.
@anonymous
Oh I know.
pete kent,
If you don't think the methodology is sound, why bother with this site?
You are not offering any suggestions to Nate to improve the methodology. Instead, you are using it as a forum to cherry-pick numbers from a single poll to "prove" that Obama is in trouble. You assert (without any factual support) that the race is tightening.
You accuse this site of bias. Fine, most of us here do support Obama, but this thread is about methodology. If you cannot offer a suggestion to improve Nate's methodology, then kindly refrain from the partisan noise. Enough of that can be found outside this site.
@isabel
Interesting thought about voter registration issues.
Normally, ignoring cell phone-only people doesn't make a difference. But in this election, there's a young, tech-savvy candidate whose strategy has been mobilizing young, tech-savvy people, vs an old, tech-illiterate candidate whose strategy has been trying to play to traditional groups.
If cell-phone-only voters had distinct preferences in any election, it'd be this one.
He is seen as the candidate most likely to raise taxes, while McCain is the one seen as most likely to control spending.
And how many report "high taxes" as being even in the top 5 of issues? Maybe 10%?
Also by 20 points the voters see McCain as having the view that America is a fair and decent society. Only 45% think Obama shares that view. If the people really have that assessment of Obama he has got some work to do before he passes a crucial credibility test for a Presidential candidate: Love of Country.
I would agree that McCain expresses more blind nationalism than Obama. That's one reason I'm voting for Obama.
McCain is of course seen as the candidate who will finish the job in Iraq (76%) versus bringing the troops home (Obama 78%). What the poll does not update is the popular view as which should be the priority. From what I can see Rasmussen last polled on that issue in May before there was any reporting on the success of the Surge.
The surge has been reported as successful roughly every 2 weeks for the last year.
People SCREEN OUT POLLSTERS?!
For the love of GOD. WHY CAN'T THEY CALL ME?!?!?!
I have always burned to take part in polls and never get called. It drives me crazy how some people are throwing their chance away.
I think Bill in GA might be on to something -- the margins are out to lunch on a few states, but Obama's share of the vote is generally pretty reasonable.
Actually, the least reasonable results are for Barr, IMHO. Zogby has Barr at 9% in OK, for example -- even though, as I understand it, OK has extremely strict rules on ballot access and Barr won't be on the ballot there at all!
What all of this suggests is that quite a few Republican-identified voters (perhaps those who incline to being isolationist and Christianist) don't really like McCain very much, and jump at the chance to show their support for an alternative. They're highly unlikely to vote for Barr in such numbers in Nov, but I wouldn't really be surprised to see that a large number of Republicans in Oklahoma kind of wish they had a different sort of Republican candidate...
"I came here for some hard facts, but am becoming disillusioned by the statistical bias that is being built into the reporting/modeling."
Why don't you just leave then? What's holding you up?
Don't let the door hit you on the way out, 'cause nobody is stopping you. You're wasting way too much time with your lengthy and pointless posts.
" anon@8:24,
Those results are pretty screwy, I agree, though one of the two polls done of SC had McCain up by only 3, so Obama up by one isn't totally out of line.
A lot of the other states look pretty reasonable (CA, FL, NY, MA, CT, IL, CO, IA, MN...).
There are some real inconsistencies, of course. If Obama were really leading in AZ, and ahead by 16(!) in NM, he'd be ahead by more than 2 points in CO.
I'll be interested to see what effect these numbers have on the overall results of the model!
"
Actually even the "outliers" don´t move much. About 1 point in the states with not much polling, about .5 points, in states like NC.
Pete Kent
There is no victory in Iraq, just a new set of conditions. You'll find your victory under the leprechauns pot of gold. You can torture numbers all you want, Iraq gets Krusty nothing.
I was willing to maybe give these a look-see, but then I got to the second state. C'mon. Not even a 25% rating is weak enough to make up for Obama being AHEAD in ARKANSAS by 2 pts.
Worthless.
I would love to see these results in November...but I have to be realistic about it and say that these Zogby polls are very inflated. There were some good points on the youth vote as well as the fluidity of John McCain's vote %.
I would almost disregard these results, but the sample size makes it some credit to what's going on.
Evie,
I think Rasmus' post just above addresses your concern. Adding these numbers in isn't going to mean big changes to the predictions -- they'll move by a percent or two, nothing more.
The cost of counting these polls is much lower than the cost of leaving polls out of the model just because they give an unexpected result. Once you do THAT, you might as well give up altogether.
These seem like reasonable results to me, but I still question their inclusion. For example, if Ron Paul were in the race, do you think they'd be worth anything at all? Obviously the strength of support for Barr doesn't compare, but I think the Paul campaign has provided ample evidence that internet polls are thoroughly useless.
when you have one candidate who's been powered by the internet seemingly to their nomination while the other has likely never 'googled' his own name - it seems pretty dumb to poll via the internet.
I'm realizing that this site isn't worth shit after seeing that these worthless polls are being included.
What's next? Including a poll on cnn.com where Obama wins 90%-10%?
I'm an Obama supporter, but including these are just incredibly stupid.
Because ZI samples are non-random and, as someone said above, self-selecting -- you can sign up on the site to take them, or forward the e-mailed link (presumably to your friends and family) -- I am not sure it's appropriate to use regression analysis on them.
Even Harris Interactive probably has a more representative cross-section of the public because at least they pay points for responding that you can accumulate and cash in for prizes. Zogby Interactive doesn't pay, so you have to just be interested in the subject.
BTW I know Zogby polls are non-random because my husband and I both like to take them. We take a lot of the same polls!
These numbers are way too friendly to Obama but if he leads by 14 in MI, i doubt he is behind in the reality.
Comparing ZI results to the current Snapshot:
AL +6.5
AR +14.4
AZ + 10.3
CA +0.9
CO -2.9
CT - 3.4
FL -2.1
GA -1.0
IA -4.7
IL -3.5
IN -2.1
KY +8.9
LA +1.3
MA +5.9
MD +7.8
MI +8.6
MN +3.6
MO +2.4
NC +11.6
NH -6.7
NJ +0.4
NM +8.9
NV +0.7
NY +0.3
OH -0.7
OK +15
OR +6.9
PA +0.6
SC +8.9
TN +8.8
TX +4.4
VA +3.5
WA -0.6
WI +0.1
A lot of these are close -- I count 18/34 within four points of the Snapshot, and that subset is about evenly split. But look at the states with the biggest differences:
OK +15
AR +14.4
NC +11.6
AZ +10.3
SC +8.9
KY +8.9
NM +8.9
TN +8.8
MI +8.6
MD +7.8
By contrast, the most McCain friendly result is NH at 6.7 better than the Snapshot, followed by IA at 4.7.
these polls aren't great but they can get a quarter rating. A lot of the numbers are inline with others we've seen and some are not. Give them a reduced reliability and let them stand. They won't matter much over all.
Completely bunk. McCain voters are more likely to be like McCain: non-computer users. Considering half the population does not rely on Internet access for daily life, I wouldn't put even a quarter weight on this poll.
I know that these polls are extremely Obama friendly, and that many don't line up with averages we currently see. However, it is only recent history that has shown elections to be very close. In the past there have been many instances of blowouts, and it seems unlikely that we will have 3 very close elections in a row. Given that, these polls have some merit becasue they are a survey of voters and do give us some idea of how people may be thinking.
I think Nate is right to include these polls with low weight, as long as they provide some level of information. The reason this site is so good is because Nate has a methodology that quantifies exactly how bad Zogby Interactive really is. He doesn't just go with his gut.
Here is the Zogby Interactive methology, for those who want to have an informed discussion of its merits.
Nate:
It seems that polls with fewer "Undecided/other" reponses, such as a typical SurveyUSA poll, might outperform. Have you explored the relationship between the number of "Undecided/other" in a given poll and the poll's accuracy?
Cugel:
I think Nate calculates the weights differently from how you suggest. The variance for a given poll is given by:
VariancePoll = VarianceSS + VariancePIE
Where VarianceSS is due to the sample size, and VariancePIE is due pollster-introduced error. VariancePIE should be proportional to the square of the PIE listed in Nates' chart. The weight for a given poll is then (1 / VariancePoll). (Ignoring time-decay for now.)
So on PIE alone, Zogby should receive about 1/8 the weight of other polls. The reason it's closer to 1/4 is because finite sample sizes introduce additional error that is more evenly shared between good and bad pollsters.
For example, if a given poll has a PIE of 2, and Zogby Interactive has a PIE of 6, but the "margin of error" due to sample size of both polls is 3, then the unscaled weights will be:
Zogby: 1/(36+9) = 1/45
Other: (1/4+9) = 1/13
Someone correct me if I'm wrong.
MN:
You can sign up here. :-)
DU, as to your post at 8:30, please retry your math on 21/5.
You have this result at 5.2. No matter how many times I try it it comes out to 4.2. While not earthshattering I stand by my statement that this is significant. Here we go again with the Democrats fuzzy math. LOL
Sorry to disappoint agains DU. On RealClearPolitics, as of today Obama's trend is downward while McCain's remains upward. Look again. And the LA Times poll is still on there. It will be even greater when that drops off.
Just checked all of Zogby's numbers against Nate's current snaphots. The results are essentially confirming 'bill in Georgia's thesis of indicating soft McCain support, not overestimating Obama's strength. In average across all states (unweighted by EV, population or anything similar), Zogby comes out with 2.5% less for Obama, and 5.7% less for McCain.
There is just one positive 'outlier' (i.e. Zogby's result differing by more than 2.5% from Nate's snaphot), which is NC. For a number of states, Zogby is reporting lower Obama shares compared to Nate's snapshot. These are primarily New England (NH, CT) and Mid-Western (IA, IL, IN, WI, MN) states, plus CO, OH, WA. Dissatisifaction of Internet users with the Obama stand on FISA?
McCain is strongly underreported by Zogby in comparison to the snapshot in a number of states, especially in 'larger Appalachia' (AR, OK, TN, KY, MO, NC, SC, GA, VA), in the South-West (AZ, CO, NM, NV, TX), and parts of the Rust Belt/ Mid-West (MI, MN, OH, IN). This geographic pattern corresponds to what one would expect if the conservative GOP base is still unsure about McCain.
I am not sure whether self-selection is the main problem with these polls - any poll is somehow self-selected, as people may refuse to participate. Rather, I would assume a bias towards well-informed population groups, or, in other words, hardly representation of low education / low information respondents, as the latter are less likely to use the internet.
In this sense, the polls definitely need to be down-weighted. However, there have been a number of questionable polls over the last weeks, that appeared to underweight AAs, younger voters or other demographic group. If those polls were included, the Zogby polls should be included as well.
@ Isabel: "Second, this means that cellphone-only people are the sort of people who move often; I'm not sure what this translates into politically." I would say, such people appreciate change, or are at least not afraid of it.
Even if Zogby is not the best pollster, these polls still represent the opinions of 46,274 likely voters.
Moreover, most of these polls are very much in line with other polls for June. And some are more favorable to McCain. For example, compare the difference between Obama's lead (or deficit) in Zogby's new polls to the unweighted average of the other June polls in the following states:
CA - No difference
CO - McCain +1.5
CT - McCain +5.5
FL - McCain +2.167
GA - Obama +1.25
IN - McCain +2
IA - McCain +1
NV - Obama +2.5
NH - McCain +8.5
NJ - Obama +0.5
NY - McCain +2
OH - Obama +0.5
PA - Obama +2
VA - Obama +3
WA - McCain +1.5
WI - McCain +1.67
There are also a couple states (IL and MD) that were not polled in June by other pollsters, but seem to be in line with expectations.
So although there are some outliers in other states, it would be unreasonable not to use these numbers. Rather, all of these polls should be included after being appropriately weighted.
Just first glances can tell you that some of those polls came out as garbage. Obama up by 3 in Arizona? Up by 2 in Arkansas? Down by only 5 in OKLAHOMA? Give me a break. I'm a total Obama homer and I can tell those polls are junk.
Cugel,
the weight of a poll is as follows:
=0,5^(d/30)*((6400*(SQRT((80*n^(-0,5))^2+PIE^2)^-2))/440(roughly)), where d is the number of days transpired since the poll was in the field. This formula does not account for the punishment of polls, where more recent polls exist.
One thing of course is that these are simply MANY polls. Any set of 34 polls is bound to have several funny results and 1-2 bizarre results.
Bear this in mind, and allow for a possible Obama bias of 2-4 points (and most polling firms will have some bias, intentionally or not), and this all appears a fairly "normal", even if fairly "wrong", batch of polls, which certainly should be included with a low weight.
If I do an in-my-house poll will you weight it for me, Nate?
From everything we have seen McCain is down about 5 points. There hasn't been much evidence to the contrary from any group of polls yet. The Rasmussen poll is very stable, the Gallup bounces around from Obama +6 to even and most other polls are in that range. Its pretty close, but consistent. Until we see a group of polls to the contrary, thats what we must assume. Theres no trend in any direction. Since Obama getting the nomination its been very stable.
None of these polls mean a row of beans until after the conventions. Nobody knows what events will occur between now and Nov 4, all predictions are just opinion based on current knowledge not future knowledge.
I get a kick out of the media including the well respected Chuck Todd comparing this race to 76 and 80. I think its human nature to have to compare current events to past events. This race has similarities to many races in the past, and they have no bearing on this race whatsoever. All comparisons are just punditry. McCain could win in a landslide, however improbable that appears today. Obama could win in a landslide or it could be close.
I come to this site to see facts as do all of you. Nate has given us the best projection of how the race looks right now across the board and we should believe it. Its the best we have.
I'd really like to see an enormous amount more data on how they do the weighting process, but I agree with Nate. I'm not ready to completely close the door on online polls. There is obviously a self selection bias, but this isn't a case of 20 people clicking on a Vote here link on a single webpage. If he is advertising across the political and non-political spectrum, he is getting a decent cross-section, especially if there are large numbers of respondents. Somebody said McCain's voters are likely like McCain (read: old). I'm sure that the system weights for this.
If the weightings are correct and the respondent numbers large enough, I think you can get some decent numbers out of this. The problem is that this whole methodology is still in the learning phase and will take a couple of cycles to iron out.
I agree with Nate: Include them but with a very light weight.
Nate, how about some discussion, preferably in English translation, of whether conventional polls are adequately sampling Obama's young and minority constituencies? And whether "likely voter" models (presumably from past turnout) are outdated?
My guess is yes to both, but I'm not a methodology geek.
The polls sure look way over optimistic for Obama. But, it would be wrong for Nate to just ignore them because they do not seem right. When Nate starts to make subjective judgments about which polls to include, which to exclude, and how much weight to give polls, then he destroys the site's credibility. OTOH, unexpected results may call for re-assessing the model to see if is the best it can be. Any decision to exclude the Zogby polls would need to be systematic, rather than the ad hoc decisions some call for here.
I'm surprised that no one has yet pointed out the fact that 3rd party candidates poll MUCH better in the summer and their numbers go down steadily until election day. This happened with Wallace ('68), Anderson ('80), Perot ('92/'96) and Nader ('00). I think it will happen with Barr as well.
Why does it work out this way? I think a lot of voters who are dissatisfied with the choice that the two major parties present toy with the idea of registering their feelings by voting 3rd party -- but as the election draws closer they arrive at the conclusion that they would rather have a vote that counts and end up supporting "the lesser of two evils".
A lot of Barr's (seeming) support is coming from conservatives who are eventually going to come around and vote for McCain.
As to whether we should count Zogby interactive, I think the question will be moot when the legitimate pollsters start polling more often. Right now the number junkies who hang out at this 'site (I include myself) are getting impatient to read any results.
What Zogby lacks in quality they more than make up for in quantity! ;-)
asmodeus: Such a poll, assuming a sampling size of 5, would get a weight of about .01 and would be dropped immediately because of being not significant.
Rasmus,
Is that the same as:
weight =
0,5^(d/30) : Exponential time decay
* (6400/440(roughly)) : Arbitrary scale factor
/ ((6400/n)+PIE^2)) : Inverse variance
I take it you are using 6400/n as an approximation for the sample-size variance. Does Nate use this approximation? Wouldn't (2500/n) be more reasonable?
I believe that these polls become much more plausible if we give 40 - 80% of Barr's support to McCain, depending on the perceived partisan lean of the independents within that state. The rest could be split 50/50 to Obama and Barr.
Perhaps that could be a factor in the weights.
I am very skeptical about these "Voting 2.0" scenarios (the election being decided by young, hip, Facebookers). All the exact same arguments (sans Facebook) were made in 2004 and it led to a great deal of disappointment. It's very exciting to think about, yes, but it's wishful thinking to imagine millions of surprise voters overwhelming the polls in November.
Nate,
Following up on my previous question about "Undecided/other" voters: what would happen if the poll were adjusted to reflect the number of people who actually chose a candidate?
For example, consider if a poll samples 1000 people and the responses are:
Obama 45%
McCain 35%
Undecided 20%
Would it be better to treat this poll as a poll of 800 people with
Obama 56.25%
McCain 43.65%
In other words, just pretend the undecided people weren't polled. This would effectively give more weight to polls with fewer undecideds.
McCain +5 in MO with Rasmussen poll out today.
I share the skepticism about these poll results expressed in many comments here. I do have a question about interactive internet polling, however.
As I understand it, Zogby attempts to overcome the sample self-selection problem with his interactive polls by weighting the sample based upon other information supplied by participants, such as party identification, income level, etc. Is this, in principle, different from the sort of weighting that's done in conventional telephone polls to deal with issues like cell-phone-only people, Caller-ID screening, people who simply refuse to participate, people who aren't home during the evening, etc?
In other words, apparently no pollster relies on raw random selection and all to some degree construct their samples "by hand" to overcome biases that are inherent in the telephone sampling process. Is there something about the self-selection bias of Zogby's Interactive internet polls that cannot be overcome by the sorts of techniques that pollsters use to deal with the other sorts of bias? BTW, that's not a rhetorical question -- I'm not a polling professional or a statistician -- I'd really like to know the answer.
Peter Kent,
Even if you are right that public opinion is going to turn in favor of the Iraq war I doubt it will help McCain very much. Especially with the Iraqi governemnt very publicly demanding a timeline for withdrawal. With the economy in shambles and the increasingly dire reports that it will continue to tank all the way into 2009 all that a decrease in violence in Iraq will do is allow it to fade away for voters as an issue. Practically I think this will mean the percentage of people naming the economy as the top issue will rise and Obama is clearly favored in that category.
And whether or not you agree with it Obama's increasing emphasis on Afganistan and Pakistan makes him seem Presidential and helps him pass the commander-in-chief test. The more he reminds people that that is where the people that attacked us reside and positions himself as the champion of that war the better off he'll be. The "we took our eye of the ball when we went to Iraq argument" works better the more stable Iraq gets because Obama says this allows us to fight the people we should have been fighting in the first place.
And all the hand wringing about Obama underperforming the "generic democrat" has little to do with him and everything to do with McCain. Democrats aren't surging because this country has all of a sudden become more liberal. They are flocking to democrats because of the failures of the Republicans and they have no other choice than the Democrats. But McCain still has some of his maverick image so those disgusted with the Republicans but not very enthusiastic about the Democrats feel like they can support McCain because he's not your typical Republican.
anon 11:01 -
I agree that each election is unique, and since Obama has been compared with everyone from Carter 76 to Carter 80 to Reagan 80 to Dukakis 88 to JFK 60 to Clinton 92-96 to Kerry 04, it does get ludicrous.
That said, I think some things can be learned from all of these:
Dukakis and Kerry: Respond forcefully and quickly to slime and smear.
Carter 76 Carter/Reagan 80, the change candidate wins in times of national discontent with the incumbent.
JFK 60-Clinton 92
the young charismatic change candidate wins but it is a big leap from the status quo
Clinton-Bush
Clinton-Dole
Young positive candidate beats old cranky one.
As to Zogby, can't add much, except to say that SOME attention must be paid to such large sample sizes, though with a major helping of salt. I mean if you polled the people on Freeper or Lucianne McCain would be winning Massachusetts 95-5. Probably a good rule of thumb would be to take about 10 points off states like NC, Arizona and Oklahoma, which would still mean a tight race in NC, McCain more concerned than he would like about his home state and Oklahoma about as red as it can be...I think Nate's weighting this at 1/4 is reasonable and conservative. I don't recall the howls of outrage when surveyusa was included, despite wildly varying party ids from month to month. Zogby is just one more bit of information.
Lastly, couldn't agree more with Counselben's reply to Peter Kent (please stop sullying the name of a great Canadian newsman by using his handle, PK!). Of course, most of the folks on here want to see Obama win. However, the point of this site is to look at all the numbers and (for me, being a layman) learn from all the statistical wizards on here, esp. Nate, as to how these numbers can be crunched.
With a massive flood of polling every week, I am grateful to Nate and Sean for this truly innovative and groundbreaking site, and would love to see the hate speech and vitriol left to other places. It is boring and offensive to read and has no place here.
BTW, that goes for the Obama side, too, though the vast majority of insults and slurs on this site come from the right.
Rasmussen MO is out at McCain 47 Obama 42; and 50 to 45 with leaners.
If you only look at the results of, say, where Obama has bought airtime, the results are actually not that great for him (for those saying they're way over biased for Obama). The reason it looks skewed is because of the states that people though he had basically no chance in: AZ, SC, AR, for example. But, let's look at the states that he bought air time in:
CO +2
FL -4
GA -6
IA +4
IN +1
MI +14
MO +2
NC +9
NH +3
NM +16
NV +0
OH +5
PA +10
VA +5
WI +10
Honestly, does anything look completely out of whack for the states that even Obama thinks he needs to target? Here's the divergence (for Obama) from Nate's "Snapshot" for these states:
CO -2.9
FL -2.1
GA +1
IA -3
IN -2.1
MI +8.6
MO +2.4
NC +6.4
NH -6.7
NM +8.9
NV +.7
OH -.7
PA +.6
VA +3.5
WI +1.5
This shows me a couple things. First, Obama is basically exactly where other polls have pegged him in the big states like OH and PA. However, he's doing a bit worse off in states where it looked like he was doing well and, if these polls are really accurate, should be slightly worrying, like IA, CO, and NH. If you add up all the divergence here you end up with 16.1, or an average of about +1.1 Obama. Is this really getting people that worked up? Is it that inconceivable that Obama's ads could have moved him an average of +1.1 in these states?
As for the other states, since Obama doesn't consider them to be competitive yet, I guess we'll need more polling to see whether this poll is good or not and whether he should invest some resources there, but as it is, this is really not all that far off from where the state of the race is right now, and seems entirely plausible.
Modeler @ 11:34.
This stuff gets fairly complicated, but it appears that you *wouldn't* and to treat the undecideds that way.
If anything, a lead needs to be discounted more when there are a high number of undecideds, which implies giving some majority of undecideds to the trailing candidate, whereas that other approach implicitly gives more undecideds to the leading candidate.
One possible exception is in Senate races that are particularly lopsided. If a poll comes out showing a Senator leading 60-30 or something, the race will often end up being closer to 70-30 than to 65-35. We do actually account for this on the Senate side, but not on the Presidential side.
Just WOW at Obama being ahead of McCain in Arizona. I want to believe in your methodology, Nate, but that just seems ridiculous. Other than that and a few other outliers (SC, NM, etc.), the data seems reasonable though. Part of me wishes that you would just throw out the polling data that is obviously wrong.
y-j:
Nate didn't determine that Obama was ahead in AZ. A scientific poll did. It could very well be crap (which is what most people are saying here), but Nate had nothing to do with the result.
I think Nate's doing the right thing in including this with only a bit of weight. Once you start throwing out polls, where do you draw the line? I suppose you could just say you'll exclude all "outliers", but you have to define what you consider an outlier (1 SD away based on all other polling?), and then you always run the risk of just missing trend. I think it's better just to pick up all the polls, but give weight based on poll performance, which is exactly what Nate is doing here.
I am very skeptical about these "Voting 2.0" scenarios (the election being decided by young, hip, Facebookers). All the exact same arguments (sans Facebook) were made in 2004 and it led to a great deal of disappointment.
Disappointment? Youth turnout was extremely high in 2004, 51.6%, up from 42.3% in 2000 and 47.9% in 1992.
Overall turnout was higher in 2004 than 2000, by 6.4%, so youth turnout grew by 50% more than the national average.
That's a major difference. Sure, it didn't swing the election. So what?
weight =
0,5^(d/30) : Exponential time decay yep
weight= Time decay*Pollster Introduced Error*Exspected Sampling Error.
The Exspected sampling error is 80*n^(-0,5), you find the Pollster Introduced error in the Pollster Ratings, and the approximation is 6400*(SUM OF THE SQUARES(ESE, PIE))^(-2). That one is divided by 440 to get a ratio, and then mulitplied with the time decay.
If you have any further questions concerning the methodology, you can send me an email to "rasmus.pianowski@googlemail.com".
Nate,
Thanks, that's very interesting. It still seems like there should be some way of discounting polls with a large number of undecideds. Maybe add in another variance term proportional to the square of the number of undecideds? That would reflect the probability distribution of undecideds breaking for one candidate or the other, assuming we really don't know which way they lean.
To explain why they might sometimes break for the favorite and sometimes break against, I can imagine modeling undecided voters as something like this:
1) Informed undecideds. These people know a lot about the candidates but still can't make up their minds. I'd imagine this happens often in situations in which a voter's personal opinion is at odds with the opinions of those around them (or the media narrative). These undecideds might tend to break against the favorite.
2) Uninformed undecideds. These are people who haven't decided because the candidates are little more than names to them. If they are still uninformed come election day, they'll probably go with the favorite, trusting the judgment of the media and those around them.
In lopsided senate races, the uninformed undecided model might dominate. However in races like the NH democratic primary, the informed undecided model seemed to dominate.
"Just WOW at Obama being ahead of McCain in Arizona. I want to believe in your methodology, Nate, but that just seems ridiculous."
Is it really all that ridiculous?
According to Nate, the three states most similar to Arizona are New Mexico, Texas, and Nevada. Obama leads in New Mexico and Nevada's a tossup.
Obama does very well in general in the Southwest - Colorado, NM, NV, CA. It seems to me that the only reason Arizona doesn't get thrown into that mix is because Obama's opponent is the senior senator from the state. If McCain weren't from Arizona, would anybody be surprised if Obama were polling well there? Al Gore couldn't carry his home state in 2000, why are we so sure that McCain will carry his?
Nate's regression - WITHOUT including the Zogby numbers - was showing McCain only up +4.3 in Arizona and Nate's projection for Arizona - again, BEFORE the Zogby numbers were added - was McCain +9.1 with a MOE of 9.1, which was giving Obama a 16% chance of winning Arizona.
I agree that the most likely explanation of this result is simply that the Zogby numbers are wrong. But I think that there's some chance that there's something there.
@Totalino:
Before you criticize how Nate's model works, take the trouble to learn how it is constructed. If you do so, those objections you raised fade into insignificance.
Specifically, the ZI polls are only one of several polls and they are being given very little weight. Their impact on the final projection is small and will diminsh further over time.
Nate,
I think that I have come to the point where I have no respect for your methods. You are continually changing what goes into your results, and in the end it certainly can't be more accurate in a scientific sense even though your adjustments generally are bringing your results back to the averages.
The biggest thing that gets me is the suggestion that you can predict November's results with today's polls. YOU CAN'T ACCURATELY PREDICT THE FUTURE, YOU CAN ONLY MEASURE THE PRESENT. It's like predicting the weather on November 4th based on today's temperatures and averages. It isn't anything that I would bank on, and I don't know of a single pollster that would claim that they can predict the future 4 months out.
Rasmus,
I think you meant to write:
weight= Time decay * (Inverse variance)
Inverse variance is approximated by (6400/440)*(SUM OF THE SQUARES(ESE, PIE))^(-1).
I still don't understand where the expected sampling error of 80 / SQRT(N) comes from. Why not just use:
Variance(vote share) =
p(1-p)/N
or variance(margin) =
Var(O) + Var(M) - 2Cov(O,M) =
p(1-p)/N + q(1-q)/N + 2pq/N =
(p + q - (p - q)^2) / N
where p and q are Obama and McCain's vote shares.
Brambster,
Two points:
1) Scientists are always refining their theories. That's how theories improve.
2) No one can say for certain what would go on in November, but Nate's not doing that. He's assigning probabilities. My guess is that if you asked a pollster whether one candidate was more likely than the other to win in November, most would say yes. But you are right in that no one knows for sure.
What a pathetic set of polls. Weigh these polls .0000001 if you could. And also take some of my advice and weight the PPP from Ohio low like .25 too, the poll that has Obama up 11. Considering 50% of those polled were Dems, 30% were Republicans, and the others were Independents.... Obama MAY be up 2-3 points in Ohio, not 11.
Finally if you eliminate that dumb LA Times poll on the RCP Average, I come up with Obama at 47.6 vs. McCain's 44.2; a much smaller 3.4 difference.
The Zogby polls are surveys of subsets of State voters--those who actively make use of computers and the internet.
On the other hand, Gallup/Rasmussen... polls are also surveys of subsets of State voters--those who possess landline telephones.
Zogby likely overpolls Obama supporters while the others overpoll McCain supporters.
Michael @1145: The Maliki government is posturing with regard to its timetable and even if it were not,that doesn't change the fact that Obama was wrong about the Surge and if he had had his way the war would have gone unfunded and the country abandoned to chaos. Poor judgment.
You make the point that the economy is in "shambles". This has been the Democrat dialectic fueled by the mainstream media ever since there was a whiff of economic trouble during this election year.
Unemployment reamins relatively low by historic standards at 5.5 or 5.4%. We lost 63,000 jobs last month. Can you guess how many jobs are typically lost monthly during a true recession?
I am waiting . . . .
Mmmmmm . . . .
250,000 was the monthly number --during the relatively mild recession of 2001 (remember that historic collapse?).
While you could get rich predicting macro economic trends --could be up, could be down -- it is a gross overstatement to say that the current economy is in "shambles".
It is that sort of shameless political posturing that has contributed to the downturn by depressing consumer confidence. Its anti-American as well as deceptive.
Pete, I appreciate your passion. However on this site you might better serve your cause by relying more on facts and figures and less on rhetoric. A lot of us come here because we find little value in this sort of posturing and spin:
Pete Kent
While you could get rich predicting macro economic trends --could be up, could be down -- it is a gross overstatement to say that the current economy is in "shambles". It is that sort of shameless political posturing that has contributed to the downturn by depressing consumer confidence. Its anti-American as well as deceptive.
John McCain's new ad
[John McCain] believes our world is dangerous, our economy in shambles.
Mr Kent, you don't seem to have noticed that there are a lot of very numerate people around here, many of whom work in fields like economics and finance. You might want to be more careful about randomly throwing around statistics like the unemployment rate if I were you, since there are surely people here who have forgotten more about how it's calcluated and how to use it than you or I have ever known.
In fact, I'm getting the impression that you may not understand all that much about most of the discussion on this site, but I am willing to be proven wrong.
Brambster @ 1:18 PM
"YOU CAN'T ACCURATELY PREDICT THE FUTURE, YOU CAN ONLY MEASURE THE PRESENT."
What made you click on a link to a website that ***PROJECTS*** elections? "FiveThirtyEight.com: Electoral Projections Done Right"
If you don't believe in dragons, don't click on a dragon web page and tell the person running it he's an idiot for doing so. A lot of people think today tells you something about tomorrow (for instance, if McCain's up by 25 points in every national poll today, would it not be insane to say that Obama might be leading tomorrow?). If you are not one of those people, then this site will probably continue to be as unsatisfying to you as your comments are to it.
Pete Kent wrote:
it is a gross overstatement to say that the current economy is in "shambles". It is that sort of shameless political posturing that has contributed to the downturn by depressing consumer confidence. Its anti-American as well as deceptive.
Modeler replied:
John McCain's new ad:
[John McCain] believes our world is dangerous, our economy in shambles.
Oh, SNAP!
Sure, Zogby interactive is almost useless. The key word here, however, is "almost". The people accusing Nate of bias by including them are, in fact, the people who are biased. Nate doesn't arbitrarily weight anything, people, it is based on how well the polsters have done in past elections. Not including these at all would be to set the weight to zero, which is unsupported by past data.
This poll shares a fatal polling flaw that led to the famous "dewey defeats truman" headline debacle of the 50's. In that case, the paper relied on polling data from telephone surveys which showed dewey up over truman to print their headline in advance. The reason that those telephone polls didn't reflect what actually happened on the ground in the elections was that in the 1950's, there were still not telephones in every American home. The ones that did have them were more likeley then not to be of a higher socio-economic class, which tended then to vote republican. Therefore, the people who were called for polling were a biased sampling from the beginning and the results were slanted.
I think these internet surveys are the same way today. While I agree that cell phones are making polling difficult, assuming that internet polling is any more reliable is probably not correct. I would guess that there were simply more mid to upper class dems answering the online polling then there were middle to upper class republicans.
Maybe ass republicans are always alleging, the internet/media really are dominated by liberals.
modeler: I think 538 found this formula by empirical measures.
Currently I am working on an own model, which will be ready in some weeks. The poll weighting part will be roughly the same, but the variance/simulation part not.
What a pathetic set of polls. Weigh these polls .0000001 if you could. And also take some of my advice and weight the PPP from Ohio low like .25 too, the poll that has Obama up 11. Considering 50% of those polled were Dems, 30% were Republicans, and the others were Independents.... Obama MAY be up 2-3 points in Ohio, not 11.
@MVRed. Too bad that Nate uses FORMULAS to determine the weight of the polls, not any biased intention.
Zogby Interactive IS bad, but to get such a high PIE that their polls get a weight of .00000001 they probably have to predict a 100% vote share for candidate 1 when candidate 2 actually gets 100%. And that in every election.
And PPP has actually a good tracking record and the most transparent methodology I have ever seen. If you want to manipulate and reweight the polls even further as this site is doing with the "trend-adjustment", then it really is not objective anymore. Some of SUSAs VP matchups had really unrealistic Party ID´s too. Or they showed McCain getting 25% of the Black vote. But if every poll was reweighted by such standards, well, you could do that at rushlimbaugh.com or Huffington Post. Not on a -mostly- unbiased website, please.
Y'all got me on the McCain ad! Still I reject the hyperbole being associated with the present economy.
I must say I am impressed with how "numerate" many of you are and I confess I don't have the time to really analyze what is being said with regard to quantitiave matters in depth. That said, I do know what the unemployment rate is and have a farily good idea of what economists traditionally define as a recession and how this present condition compares to past ones.
Many of you are very smart, some of you are very snide. I am very stubborn - - especially when I know I am right!
Modeler & Jim,
I have seen Nate write twice recently that he was attempting to project the result of the election. The first time that I saw this was when he introduced the "Projection" line to the Super Tracker which moderates the point-in time lead. I think that this did a lot to bring his overall projections inline with all other poll trackers, but in reality he was only balancing out his Super Tracker method which was giving too much weight to Obama post Hillary.
Right now I think the map is dead-on with the exception of Nevada which is notoriously hard to poll and I believe that Obama is stronger there than the pink color suggests. If you look at Intrade prices, this is also much in line with that.
What is happening is Nate is taking polling results, then mixing in all sorts of other stuff, and then essentially he is coming back to the same numbers established by the poll results. These other things may have reason behind them, but they surely aren't any more accurate than an actual poll is.
I stand by my point that you can no better predict the outcome of a race 4 month out than you can predict the weather regardless of how sound your methods are. News/coverage, debates, ads and all sort of other stuff can't be predicted by polls in advance. Polls can't predict the future going out 4 months, and this can't be fixed by way of mixing in other measures.
Brambster,
I think you'll find this post and discussion interesting. The evidence suggests that polls aren't entirely meaningless at this point.
A bit late to the party, but I'm inclined to agree with many of the others here: Zogby telephone polls are fine to include in the analysis (yeah, they're mediocre, but not actively bad), but the Zogby Interactive polls should be ignored.
I suspect the results will be more accurate without ZI in the mix, though it's possible that the noisy nature of them will have little effect on the ultimate results - though it seems likely that ZI overrepresents support of third party candidates like Barr, Nader, and Paul.
.
Modeler
I didn't say that polls were "meaningless" in predicting the outcome of an election. Nate certainly understands that they become more and more accurate to the actual result the closer to the election that you get for instance, but that doesn't mean that you can refine the results with some historical measures and use today's data to accurately predict the exact results in 4 months.
I used weather as an example because it very closely resembles what he claims to be trying. We all know the averages 4 months out, but knowing those averages as well as the weather today will not get us any closer to the weather 4 months from now than the averages predict, and the 95% degree of confidence interval is quite wide.
Regarding including Zogby Interactive polls, I can't recall anyone that I respected in the media or blogs ever giving any credibility to their results. If you look at the map that they drew up, while the red and blue states are typical, and the tossups are a bit overstepping of reality (AZ, SC, and AR), they put states as Red with 3 and 4 point margins while placing states as tossup that were 5 point margins or higher. There's no science to their map either, it's just opinion and it doesn't agree with their own skewed numbers.
I think Nate should old-school his methodology a bit more. Even if he gets it mostly or even all right in the end, the poll averages themselves pretty much get it right without such external factors.
Brambster:
(A little lengthy obviously, so this is in response to two Brambster posts ago, but still applies)
A) You must've started reading very recently then, because he's been writing the entire time that he's "attempting to project the result of the election." Again. This is the entire premise of the site. How did you get here without seeing that (look at the top of your browser)?
B) He included the "Projection" line, because he made the (admitted) mistake of assuming that the election will not tighten as election day nears. Well, it turns out that historically, elections almost always get closer as we approach the finish line (only noticably so, though, when the gap is large). To compensate for this, he included in his model a regression that accounts for this tightening in the race. Read the post where it happened. The details are all very clear as to why he did it and you have misrepresented his purpose and intention by your description.
C) I stand by the fact that historically and mathematically, you absolutely can predict the outcome of a race 4 months out. Obviously, the farther from election day you are, the less accurate your prediction will be. What about 1 day before the election. Can you predict things about the outcome of the race then? What about 1 week? 1 month? Where's the cutoff line? What methodology did you use to calculate it? Or are you just guessing based on what your gut feels? I'll take Nate's math over your gut 100% of the time.
D) The polls aren't predicting the future. Read the methodology and you'll understand how the polls today can be used to imply things about the results on election day. If Nate was just telling us what the "polling results, then mixing in all sorts of other stuff," then how would he come up with McCain having a ~1/3 chance to win? If he was just using polls today to predict the outcome of the election and nothing else, that number would be very close to zero. The fact that McCain has such large odds to win stands as pretty strong evidence that Nate's system agrees with you that there's a lot of "News/coverage, debates, ads and all sort of other stuff (that) can't be predicted by polls" still left between now and the election.
E) You appear to be the embodiment of a major problem in our country (and possibly world) right now. When you don't understand something and it doesn't align with things as you perceive them to be, you attack it and try to undermine it. Well, most people who are drawn to a site like this will prefer understanding and learning. If you don't find yourself desiring these things, that's fine, but please do some research around the site and come back with some claims that are at least educated about what you're attacking. It is clear to me you do not understand Nate's methodology and I've read only a handful of words you've typed. Lemme guess... you call the local sports talk radio show and complain when your manager doesn't have Adam Dunn bunt the runner over with a man on 1st and no outs down a run in the 9th inning, right? If you refuse to believe evidence and logic, what do you base decisions on?
Ross said...
This poll shares a fatal polling flaw that led to the famous "dewey defeats truman" headline debacle of the 50's. In that case, the paper relied on polling data from telephone surveys which showed dewey up over truman to print their headline in advance. The reason that those telephone polls didn't reflect what actually happened on the ground in the elections was that in the 1950's, there were still not telephones in every American home..."
Sorry, Ross, not quite. The Dewey/Truman polling debacle didn't stem from bad samples, it stemmed from failing to poll in the weeks leading up to the election. Since "everyone knew" that Dewey would win, the media stopped supporting the then young polling efforts of folks like Roper and Gallup. In fact, the last few national polls suggested a surge for Truman three to four weeks ahead of the election but no one believed it.
I suspect you're confusing the facts with the story of the 1936 Literary Digest "poll" that sent out 10 million "ballots" to respondents drawn from auto registration and telephone subscriber lists in 1936. Based on the 2.3 million (!) responses they received, the magazine predicted Landon would beat FDR 57% to 43%.
That debacle led to the death of the Literary Digest.
Unfortunately, Zogby's internet polling operation is the modern day equivalent of the Literary Digest.
After reading through the comments, a couple of points come to mind.
Zogby's results cannot be defended based on the fact that they are "close to" other polls. If, in fact, they confirm the results of other polls, they add no new informaton.
On the other hand, if they suggest different results, there is simply no way to know whether the sample bias inherent in a volunteer sample is the source of the difference or not.
Either way, the results are not "informative" in any sense. The fact that they represent a straw poll of over 46,000 voters is worthless unless the sample is chosen randomly from the appropriate population.
All the inclusion of Zogby's internet-based results does is to introduce a source of error into the model. If the results are close to the average of other polls, the error is minor. If they are not, the error is both potentially greater and cannot be estimated using statistical techniques.
I'm sure Nate would not consider introducing a straw poll of fans' views of a player value into a measure of that player's performance, even if he gave it minimal weight. The Zogby results have no more place in this context.
Peter Kent
First, I'm going to be generous (a very american trait) and ignore your "un-american" comment. I will assume you weren't referring to me directly.
The unemployment rate is only one way of measuring the health of an economy. The sub-prime mortgage crisis and its aftermath have severely restricted the flow of credit, the lifeblood of our economy, and the consequent drop in home prices has had a significant negative impact on consumer spending and confidence. Inflation in fuel and food markets is having an outsized impact on lower income americans and high oil prices are also starting to drive up the prices of other products made with petroleum or that have to be shipped long distances. Peripheral but still connected to the economy is the soaring costs of health care and higher education. Basically, Americans are getting pinched from every direction. I stand by my assertion that the economy is in shambles. It's not a disaster but there are quite a few large problems inhibiting growth and eating into the income of your average american.
However, if the economy is not in as bad shape as I think it is, my point still stands. In essence I was disagreeing with this quote of yours.
As the potential, if not the actuality, of victory in Iraq becomes manifest to the people, they will shift their mood from one of pessimistic quagmire to let's get 'er done. If that happens, Obama will have an uphill slog to cobble together an electoral majority: he will have lost the raison d’être for his candidacy.
This will not happen. Politically McCain has little to gain from Iraq no matter which way it goes because the economy is perceived as being in bad shape. Thus, everyones attention will shift from Iraq to who they think will do a better job with the economy and every poll gives Obama the owerwhelming advantage on that issue.
If there wasn't a feeling that the economy is in bad shape then I think a turn in public opinion on the war would benefit McCain alot, but with the economy as issue #1 unless Iraq falls into utter chaos or the Iraqis throw Bush a ticker tape parade, a change in Iraq will have little effect on the outcome in November.
And you do know that dialectic basically means logical argument right?
Jim,
Can I disagree with you without you turning around and painting me as "embodiment of a major problem in the our country." That's ridiculous, and if anything, it speaks more to who you are than to who I am.
Stick to the facts. I think you are confusing the fact that polls are obviously tools for prediction, but they aren't predictive into themselves until you negate the effect of time. Unlike baseball teams and players, this country is so evenly divided into red and blue that 537 votes and a bunch of messed up ballots threw an entire election to the popular vote loser in 2000. We probably wouldn't even remember Florida if Nader wasn't running, or if a few days before the election news of Bush's drunk driving arrest didn't surface. Polls can't predict that stuff. What do you think would happen if Ron Paul endorsed Barr? What if Obama chose Hillary as a running mate, or maybe Colin Powell? My money's on Bayh, but polls can't predict such things.
If the election was tomorrow, Intrade would be probably be trading Obama contracts at 90, but being that it is 4 months out, they are trading at 65. The difference is due to the inherent uncertainty about what is going to happen between now and November 4th.
Nate's massaging of today's numbers in a way that he claims is an attempt to predict the actual state by state outcomes on November 4th has no better of a confidence interval than those that know Obama would win if the election was today, but give a 35% chance of it not happening 4 months out.
To include these polls is bad for the credibility of the site. Regardless of how they compare to other polls, it is a self-selected sample taken from a population that is online.
I would expect well-informed voters to be very pro-Obama.
The NC poll doesn't really surprise me. I live here, and I see Obama Signs and Stickers probably a 4:1 over McCain. If Obama can keep huge margins in the Urban areas of NC. He may very well win NC by 2-6%.
My apologies if what I'm saying has already been said in one of the comments or on a previous post. There are a lot of comments, but I read most of them.
I was wondering if it could be considered more of a problem, for lack of a better word, if polls by a particular group tended to favor a particular candidate by a certain amount, rather than just being consistently off by that amount, but not tending to favor either candidate.
I'd love to believe the Zogby polls are correct... But they are hard to believe. That's for sure.
情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,
^^ nice blog!! thanks a lot! ^^
徵信, 徵信社,徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 外遇, 抓姦, 離婚, 外遇,離婚,
徵信, 外遇, 離婚, 徵信社, 徵信, 外遇, 抓姦, 徵信社, 徵信, 徵信社, 徵信, 外遇, 徵信社, 徵信, 外遇, 抓姦, 徵信社, 征信, 征信, 徵信, 徵信社, 徵信, 徵信社, 征信, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信社, 徵信社, 徵信, 外遇, 抓姦, 徵信, 徵信社, 徵信, 徵信社,
^^ nice blog!! ^@^
徵信, 徵信, 徵信, 徵信社, 徵信社, 徵信社, 感情挽回, 婚姻挽回, 挽回婚姻, 挽回感情, 徵信, 徵信社, 徵信, 徵信, 捉姦, 徵信公司, 通姦, 通姦罪, 抓姦, 抓猴, 捉猴, 捉姦, 監聽, 調查跟蹤, 反跟蹤, 外遇問題, 徵信, 捉姦, 女人徵信, 女子徵信, 外遇問題, 女子徵信, 徵信社, 外遇, 徵信公司, 徵信網, 外遇蒐證, 抓姦, 抓猴, 捉猴, 調查跟蹤, 反跟蹤, 感情挽回, 挽回感情, 婚姻挽回, 挽回婚姻, 外遇沖開, 抓姦, 女子徵信, 外遇蒐證, 外遇, 通姦, 通姦罪, 贍養費, 徵信, 徵信社, 抓姦, 徵信社, 徵信, 徵信公司, 徵信社, 徵信, 徵信公司, 徵信社, 徵信公司, 女人徵信, 外遇
徵信, 徵信網, 徵信社, 徵信網, 外遇, 徵信, 徵信社, 抓姦, 徵信, 女人徵信, 徵信社, 女人徵信社, 外遇, 抓姦, 徵信公司, 徵信社, 徵信社, 徵信社, 徵信社, 徵信社, 徵信社, 女人徵信社, 徵信社, 徵信, 徵信社, 徵信, 女子徵信社, 女子徵信社, 女子徵信社, 女子徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社,
艾葳酒店經紀公司提供專業的酒店經紀, 酒店上班小姐,八大行業,酒店兼職,傳播妹,或者想要到打工兼差、打工,兼差,或者八大行業,酒店兼職,想去酒店上班, 日式酒店,制服酒店,ktv酒店,禮服店,整天穿得水水漂漂的,還是想去制服店當上班小姐,水水們如果想要擁有打工工作、晚上兼差工作、兼差打工、假日兼職、兼職工作、酒店兼差、兼差、打工兼差、日領工作、晚上兼差工作、酒店工作、酒店上班、酒店打工、兼職、兼差、兼差工作、酒店上班等,想了解酒店相關工作和特種行業內容,想兼職工作日領、假日兼職、兼差打工、或晚班兼職想擁有快速賺錢又有保障的工作嗎???又可以現領請找專業又有保障的艾葳酒店經紀公司!
艾葳酒店經紀是合法的公司工作環境高雅時尚，無業績壓力，無脫秀無喝酒壓力，高層次會員制客源，工作輕鬆，可日領、現領。
一般的酒店經紀只會在水水們第一次上班和領薪水時出現而已，對水水們的上班安全一點保障都沒有！艾葳酒店經紀公司的水水們上班時全程媽咪作陪，不需擔心！只提供最優質的酒店上班,酒店上班,酒店打工環境、上班條件給水水們。心動嗎!? 趕快來填寫你的酒店上班履歷表
水水們妳有缺現領、有兼職、缺錢卡奴的煩腦嗎?想到日本留學缺錢嗎?妳是傳播妹??想要擁有高時薪又輕鬆的夜間兼職工作,打工機會和,假日打工,假日兼職賺錢的機會嗎??想實現夢想卻又缺錢沒錢嗎!??
艾葳酒店台北酒店經紀招兵買馬!!徵專業的酒店打工,想要去酒店的水水,想要短期日領,酒店日領,禮服酒店,制服店,酒店經紀,ktv酒店,便服店,酒店工作,禮服店,酒店小姐,酒店經紀人,
等相關服務 幫您快速的實現您的夢想~!!
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店經紀,
酒店打工經紀,
制服酒店工作,
專業酒店經紀,
合法酒店經紀,
酒店暑假打工,
酒店寒假打工,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店工作,
酒店打工經紀,
制服酒店經紀,
專業酒店經紀,
合法酒店經紀,
酒店暑假打工,
酒店寒假打工,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店工作,
酒店打工經紀,
制服酒店經紀,
酒店經紀,
菲
梵,
Post a Comment