Dick,
It has been a long and hard-fought primary campaign. We've both had our share of successes, and made our share of mistakes. Granted, you made a few more than I did. But at the end of the day, we're really after the same thing: to apply rigorous, bias-free, well-documented, and internally consistent methodology to the study of public opinion.
So let's put this all behind us. In the meantime, let me know if you want me to book this flight for you. Minimum $1,000 per state.
6.05.2008
A Call for Pollster-Poblano Unity
@ 6:10 PM
Subscribe to:
Post Comments (Atom)
33 comments
pwned.
Does ARG get to subtract his PIE? Or can Dick just say that with these 8 samples the 1.8 difference is within the margin of error.
pwned indeed
Wow! So demographically isn't exactly destiny? Still, Poblano's record is pretty darn good considering the vagaries of primaries/caucuses, etc. Take out the two outliers, however, and poblano's model(s) do extremely well.
So why were KY and SD such outliers?
Nate,
Are you challenging Dick for the General Election on a state-by-state basis (at 1K each state)?
That should be demography (not demographically).
Does the demographic approach work better on open primaries?
Really interesting. I might be reading that wrong, but in every one of those states, ARG underestimated Obama's % -- occasionally only by a little, but consistently throughout.
So what's wrong with their system? Not catching younger people? Minorities? Cell phone-only users?
No new polls today?
Those results argue strongly for including some form of pollster bias, even if it is just regional, instead of party-based.
The fact that ARG was so vastly more accurate in KY and WV doesn't seem like a statistical fluke. I'm not sure what method you could use to account for that sort of bias, but it certainly seems like your model would be more accurate if it weighted ARG's results in those areas more strongly (and if it accounted for whatever was causing that pro-Clinton bias).
OT:New Missouri poll from Rasmussen. What effect? Inquiring minds want to know...
In the very first paragraph, you really ought to change the comma to a period.
It's a better introduction as a declarative than a salutation.
Very interesting, though, that you were off by so much in just two states. It'd be worth finding out if there were any common factors in KY/SD that didn't exist in WV or other states.
I kind of side with Dick Bennett a little on this one. (Even though ARG is a horrible polling company. And beating their polling data by 1.8% is nothing to write home about.) Why do you project the raw vote totals then act like it doesn't matter that your projections are so horribly wrong every time?
Also, are you going to adjust the weighting of your regressions based on your accuracy in predicting primary contests?
Nate, are you going to do a polling post today? I don't think there were any state polls yesterday, but there were a couple today (Kansas and Missouri).
Now, now, boys. Stop fighting. (unless ARG is a republican outfit, then go ahead).
But really, it's amazing a general stat guy did measurably better than an established pollster. The sample size is small, though.
I thought the fight was between Pollster.com and 538. That would have made me sad.
Keep it up, poblano! We love you!
With your powers combined (a simple average of the 2 estimates), your "average error" would have been only 3.7 pts. over these 8 states. Talk about a Dream Ticket.
So why were KY and SD such outliers?
Good question. If I had to guess at an explanation, the best one I can think of is that Obama spent little time campaigning on those states. Another variable to add in?
That's a pretty dramatic bias on each side.
@18:20. We did account for the amount of time spent on the ground, but it's possible that we didn't account for it enough, and/or accounted for it in the wrong way. For example, it's very different to make 10 visits to South Dakota, which has one congressional district, than to make 10 to North Carolina, which has 13.
@17:58. The average error made by a pollster in the primaries was right about at 7 points. So the notion that the demographic analysis is roughly as accurate as an average poll (which is what our general election model assumes) seems to have at least some basis in reality.
One interesting quote from New Hampshire Business Report (from the linked ajacksonian blogspot entry):
"Dick Bennett: The Manchester-based pollster defends the omission of cell phone users in polling, saying that the people without landlines – mostly young people – “don’t vote.”"
What happens if you regress ARG's error in each state against difference in Clinton's performance vs. Obama between old and young voters?
Nate:
Tell em like it 'tis...bro...
icebergslim
I don't really get what Dick Bennett's problem is, but it's not just you (although he does seem to have developed something of a preoccupation with you). I remember earlier this year he complained about bloggers misusing the term "outlier" when talking about polls (which I find a bit silly to begin with because the term doesn't have an accepted technical definition), and then proceeded to give an utterly nonsensical definition of his own (he uses a warped version of the interquartile range).
I think he's of the opinion that anyone who isn't a pollster is an amateur who should step aside.
It's amazing to see him criticize your prediction in SD though, when he missed the mark by just as much. And of course, ARG doesn't try to predict turnout at all, so there's really no basis for comparison there.
The particular galling bit is this paragraph:
Mr. Silver's predictions provide great fuel for speculation on the Internet, but even he admits choosing from many possible outcomes when making his final predictions. His final predictions reveal his personal biases as he continues to incorrectly make causal claims from statistical associations.
I admit to not reading every one of your posts, but I don't recall these causal claims that Bennett is talking about. Also, I was under the impression that using multiple regression frequently involves choosing from among several different outcomes. The accusations about personal bias are uncalled for.
One further question:
Your model does not (as far as I can tell) incorporate methodological bias (as opposed to just methodologial error) in the Pollster-induced-error. Does this matter in the overall accuracy of the final result?
It's even better than you list, Nate. You have your error in SD as 15.7, it should be 5.7.
So two of ARG's three biggest errors came in Obama's biggest wins. Meanwhile, two of 538's three biggest errors came in Clinton's biggest wins. So what would you say in response to a straw-man critic who suggests that this just a case of pundits' biases coming out on both sides?
It's great how Dick makes up measures of accuracy; "89% of the difference in turnout was for Clinton."
"Anonymous
It's even better than you list, Nate. You have your error in SD as 15.7, it should be 5.7."
No. It´s 15.7.
My error is (too bad I can´t participate here, because I was away the 5 days before West Virginia and not able to post a prediction):
Pennsylvania: 0,3
Indiana: 4,9
North Carolina: 0,2
West Virginia: /
Kentucky: 5,4
Oregon: 5,8
South Dakota: 28,6 (lol)
Montana: 0,9
Error: 6,6
That DAMN North Dakota -.- Everyone got it wrong, so it´s not such a shame, but 30 POINTS??
w/o SD my error would have been 2,9%
Did Dick Bennet not read the post on South Dakota, or did he just ignore it?
"Senator Obama did not win in South Dakota as Mr. Silver so confidently predicted." - www.americanresearchgroup.com
"...she could very easily win South Dakota (it should probably be thought of as a "toss-up"). Even a win of some magnitude (high single digits or very low double digits) would not completely stock me..."
http://www.fivethirtyeight.com/2008/06/south-dakota-projection-obama-by-5.html
He read it.
"...but even he admits choosing from many possible outcomes when making his final predictions. His final predictions reveal his personal biases as he continues to incorrectly make causal claims from statistical association".
Well, it seems Dick continues to make incorrect causal claims from biased and less-than-semi-professional polls...
Don't look at the means, look at the medians (with SD there's skew!). You're doing much better; your median error is 2.9 to the other guy's 7.1.
Good job. Nice post on Schweitzer!
情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店經紀,
酒店打工經紀,
制服酒店工作,
專業酒店經紀,
合法酒店經紀,
酒店暑假打工,
酒店寒假打工,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店工作,
酒店打工經紀,
制服酒店經紀,
專業酒店經紀,
合法酒店經紀,
酒店暑假打工,
酒店寒假打工,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店工作,
酒店打工經紀,
制服酒店經紀,
酒店經紀,
菲
梵,
Post a Comment