Comments on FiveThirtyEight: Politics Done Right: Internal Polls

酒店經紀人, 菲梵酒店經紀, 酒店經紀, 禮服酒店上班, 酒店小姐兼職, 便服酒店經紀, ...

2009-10-08T08:18:10.367-04:00

酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店經紀,
酒店打工經紀,
制服酒店工作,
專業酒店經紀,
合法酒店經紀,
酒店暑假打工,
酒店寒假打工,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店工作,
酒店打工經紀,
制服酒店經紀,
專業酒店經紀,
合法酒店經紀,
酒店暑假打工,
酒店寒假打工,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店工作,
酒店打工經紀,
制服酒店經紀,
酒店經紀,
菲
梵,

艾葳酒店經紀公司提供專業的酒店經紀, 酒店上班小姐,八大行業,酒店兼職,傳播妹,或者想要到打工兼差、...

2009-09-11T01:24:06.282-04:00

艾葳酒店經紀公司提供專業的酒店經紀, 酒店上班小姐,八大行業,酒店兼職,傳播妹,或者想要到打工兼差、打工,兼差,或者八大行業,酒店兼職,想去酒店上班, 日式酒店,制服酒店,ktv酒店,禮服店,整天穿得水水漂漂的,還是想去制服店當上班小姐,水水們如果想要擁有打工工作、晚上兼差工作、兼差打工、假日兼職、兼職工作、酒店兼差、兼差、打工兼差、日領工作、晚上兼差工作、酒店工作、酒店上班、酒店打工、兼職、兼差、兼差工作、酒店上班等,想了解酒店相關工作和特種行業內容,想兼職工作日領、假日兼職、兼差打工、或晚班兼職想擁有快速賺錢又有保障的工作嗎???又可以現領請找專業又有保障的艾葳酒店經紀公司!

艾葳酒店經紀是合法的公司工作環境高雅時尚，無業績壓力，無脫秀無喝酒壓力，高層次會員制客源，工作輕鬆，可日領、現領。
一般的酒店經紀只會在水水們第一次上班和領薪水時出現而已，對水水們的上班安全一點保障都沒有！艾葳酒店經紀公司的水水們上班時全程媽咪作陪，不需擔心！只提供最優質的酒店上班,酒店上班,酒店打工環境、上班條件給水水們。心動嗎!? 趕快來填寫你的酒店上班履歷表

水水們妳有缺現領、有兼職、缺錢卡奴的煩腦嗎?想到日本留學缺錢嗎?妳是傳播妹??想要擁有高時薪又輕鬆的夜間兼職工作,打工機會和,假日打工,假日兼職賺錢的機會嗎??想實現夢想卻又缺錢沒錢嗎!??
艾葳酒店台北酒店經紀招兵買馬!!徵專業的酒店打工,想要去酒店的水水,想要短期日領,酒店日領,禮服酒店,制服店,酒店經紀,ktv酒店,便服店,酒店工作,禮服店,酒店小姐,酒店經紀人,
等相關服務幫您快速的實現您的夢想~!!

^^ nice blog!! ^@^徵信, 徵信, 徵信, 徵信社, 徵信社, 徵信社...

2009-03-11T05:56:00.001-04:00

^^ nice blog!! thanks a lot! ^^徵信, 徵信社,徵信, 徵信社, 徵信...

2009-03-11T05:56:00.000-04:00

情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,...

2009-02-12T23:02:00.000-05:00

Nate, I'd say err in the side of caution and exclu...

2008-07-08T00:26:00.000-04:00

Nate,
I'd say err in the side of caution and exclude polls from all 5 groups.
The only exception to this rule would be to MAYBE consider including polls from groups #4 or #5 in sparsely polled areas. Some States that are solidly in the Republican or Democratic column (e.g. Idaho, South Dakota, Illinois, Vermont) were last polled in January/February, way before the candidates were chosen by either party. Since we can be pretty certain about the winner in these States, I'd say chances are that biased polls are less likely to come from groups #4 or #5 (groups 1-3 should still be excluded). In these places we are probably better off with a snapshot of July rather than January.

I think the risk in bias due to selective release ...

2008-07-07T23:41:00.000-04:00

I think the risk in bias due to selective release is minimal. If a candidate or party committee releases an internal poll that contradicts an opponent's poll, then almost invariably the opponent will release a competing poll. If no such competing poll is released, then that's a pretty good indicator that the first poll squares with reality.

Sure, there might be times when you don't want to counter, or don't have a (relatively) recent poll to counter with. But last cycle, we saw numerous Dem polls go unchallenged, especially late in the cycle. And guess what? A lot of Dems won.

You should do what Pollster.com does. Mark pollste...

2008-07-03T15:56:00.000-04:00

You should do what Pollster.com does. Mark pollsters like PPP (D) and Strategic Vision (R). You should also consider their bias when chosing these polls' weight.

Nate,This is an awesome website. As an engineer a...

2008-07-03T15:00:00.000-04:00

Nate,

This is an awesome website. As an engineer and fellow statistician I find your approach and analysis fundamentally sound. I'm a data geek too.

I was wondering if you have seen or done any work to estimate the "Bradley Effect" (non-white candidates underperforming their poll predictions on election day) on the 2008 Presidential Election? I've seen conflicting reports as to whether this occurred in New Hampshire. I'm certain that Obama's campaign has analyzed this. I for one, think Hillary Clinton did Obama a favor staying in the primaries by allowing him to test the "Bradley Effect" effect in all states.

You are misinterpreting what I said just to try to...

2008-07-01T18:04:00.000-04:00

You are misinterpreting what I said just to try to score points. What I was saying was that if a reliable pollster like Rasmussen will sometimes game its questions to reflect a bias on the part of the organization that commissioned the poll, then any poll commissioned by an organization with a known bias should be viewed with suspicion.

I don't think you have the slightest idea what ad hominem ("to the man") means. I didn't attack you personally so quit your whining.

Your link is broken but the question order on the Rassmussen poll is:

#1: How do you rate the way that George W. Bush is performing his role as President? Excellent, good, fair, or poor?

If THAT doesn't prejudice the poll, I don't know what would! Start them off thinking about how much they hate Bush, then ask them whether they support "Republican John McCain or Democrat Barack Obama."

Still, overall it looks like a very good poll to me. Bad news for Obama which sucks, but pretty accurate.

I think a new study is in order... where not only ...

2008-07-01T14:20:00.000-04:00

I think a new study is in order... where not only the pollsters themselves are measured, but also the customers who commission the polls and how accurate the polls that they release tend to be.

Obviously this is going to be problematic when it comes to individual customers who don't order that many polls.

So maybe they can be lumped into categories for convenience an assessed on that basis.

I dunno, maybe I'm talking out my ass.

I'd draw the line between 4 and 5. Or just take a...

2008-07-01T13:26:00.000-04:00

I'd draw the line between 4 and 5.

Or just take all of them and figure it balances out - a dangerous assumption.

I think the issue is more consequential in the new US Senate analysis where you may have a harder time getting the volume of polls you'd like.

There are several good arguments for including pol...

2008-07-01T08:02:00.000-04:00

There are several good arguments for including polls commissioned by interest groups, and possibly even by campaigns, provided that they are conducted by reputable polling outfits, and that the full wording and order of the questions are released. (BTW, is that even the standard among the major independent pollsters?)

But the risk of introducing a grave selection bias keeps me unconvinced. After all, many polls by interest groups will probably use rather small samples, resulting in a large MoE. If we can't be reasonably sure that we're not only hearing about polls that look "positive outliers" among the internal data, I would probably err on the side of caution.

Whichever route you ultimately take, I would still find it helpful if your "Today's Polls" posts mentioned any new polls that fails to qualify for inclusion.

Cugel said "But if the best pollsters like Rasmuss...

2008-07-01T07:08:00.000-04:00

Cugel said "But if the best pollsters like Rasmussen are problematic for you, then who's left?"

You are misinterpreting what I said just to try to score points. What I was saying was that if a reliable pollster like Rasmussen will sometimes game its questions to reflect a bias on the part of the organization that commissioned the poll, then any poll commissioned by an organization with a known bias should be viewed with suspicion.

All of the polls Nate asked about are from organizations with a bias, and should be excluded as noise, unless you can design a metric to analyze the poll which will permit you to (i) adjust for the poll's bias, or (ii) determine that the poll is (relatively) free from bias.

Finally, Cugel, did you review the question I linked to? If so, do you agree or disagree with my comment about that question? A reasoned response to my point would be more productive than your ad hominem dismissal.

It has been fascinating to watch the methodology d...

2008-07-01T06:19:00.000-04:00

It has been fascinating to watch the methodology develop over time, and this may be, going forward, the most important question yet.

I'm not sure that the reputable independent pollsters can eliminate bias consistently, therefore I'd have to draw the line UNDER 5, excluding everyone with a potential reason to sneak bias into the survey.

I started by musing on the issue of what, if anything, constitutes an outlier. The only thing one can say with certainty is that a sample which may be deliberately, however subtly, doctored, doesn't even get to the starting line.

Even if backtesting some of these categories improves your accuracy, I'd still be nervous about it--what about the next one?

Ultimately, I'm more concerned about which surveys that are automatically accepted for your model may include unacceptable bias or trial design, so again, as for the five categories you mentioned, in true 1974/2008 style, I'd say, "throw the bums out!"

Just sayiing.

Keep up the awesome work!

John

Data is data. If they conducted three simultaneous...

2008-07-01T03:37:00.000-04:00

Data is data. If they conducted three simultaneous polls and released only the best to them, it would be questionable. But even if that is the case, if the poll was done properly and the data released then it is a valid poll and should be treated as any other. Your PIE should effectively compensate for any bias.

And yet, I can't get over a certain uneasiness I feel about including a candidate's own internals in the model.

In stats though, I just have to let the numbers do the work. If you can test a category of polls and they increase the accuracy then use them. If they don't, don't.

"Responding to cugel, as has been discussed in sev...

2008-07-01T00:43:00.000-04:00

"Responding to cugel, as has been discussed in several different threads, a poll can appear to have a sound methodology based solely on the numbers, but internal pollsters can introduce bias by question order and phrasing."

Of course. But, then it wouldn't be a very good poll would it? If they release the results you can SEE the question order. If for instance they ask a question that would prejudice voters for or against a candidate just before asking voter preference, that would be pretty bad polling. But, it would also be pretty obvious and easy to weed out polls that do this.

Once again, if you CAN'T see the poll, then you would have to take their word that the methodology was sound. In your example it flat isn't so it should be excluded of course. So, that's pretty much beating a dead horse.

But if the best pollsters like Rasmussen are problematic for you, then who's left?

I would definitely err on the side of exclusion of...

2008-07-01T00:25:00.000-04:00

I would definitely err on the side of exclusion of all 5 for statistical reasons. Even if all categories are done soundly on methodological grounds, if you are using each survey as a case in a regression model, then it is not only the scientific methodology of the survey that is necessary, but also the surveys as a universe should also not be biased in terms of their viewpoint or rationale. For instance, it may be that certain types of unaffiliated orgs commission surveys while others do not--this introduced bias and including these polls may drag the regression line in one particular direction.
I understand the desire to use them in Senate races where more limited data are avaliable, and if this is needed for the purposes of informing us as your audience I'd encourage you to flag this for us.

Thanks for a great site.

Exclude, exclude, exclude, exclude, exclude...

2008-06-30T23:43:00.000-04:00

Exclude, exclude, exclude, exclude, exclude...

If the poll's sponsor can choose to suppress a par...

2008-06-30T23:40:00.000-04:00

If the poll's sponsor can choose to suppress a particular poll, then their polls should be excluded, even if they have to date chosen to publish all of their polls (to our knowledge).

Nate:I would err on the side of excluding all five...

2008-06-30T23:12:00.000-04:00

Nate:

I would err on the side of excluding all five categories.

Responding to cugel, as has been discussed in several different threads, a poll can appear to have a sound methodology based solely on the numbers, but internal pollsters can introduce bias by question order and phrasing. By way of example, check out the questions for this Rasmussen Florida poll. I would argue that the phrasing of question 4 is designed to introduce a bias; in fact, it appears to be best described in the journalistic term "burying the lead." McCain is trying to "help" and Obama "opposes." Yet this is regarded as an authoritative poll. After all, what would we expect from a poll commissioned by Fox (Not Really) News. The only positive is that the bias was introduced *AFTER* the voting question, so it was introduced for purposes of making offshore drilling appear more popular, rather than to "suggest" a bias against Obama.

As internal polls are likely to use even more biased phrasing and question order, I don't think that they will aide your model. Rather, they will only introduce noise.

I wouldn't include any polls from groups 1-5. Only...

2008-06-30T20:20:00.000-04:00

I wouldn't include any polls from groups 1-5.

Only pollsters that conduct consistent operations and release the results of all their polls will reliably avoid biasing your result.

Also, in response to Cudgel:

Frankly, if a pollster took 3 internal polls and only released one, as long as that one was scientifically reliable, then it's just another data point. You can't rely on it completely, but it provides some evidence.

This argument is wrong. If you make a series of measurements but only release the ones you like, then you're inherently biasing your result. This is how schools can make the claim that all their students are above average.

Imagine a 13 year old boy who measures his height 3 times and gets 5'5", 5'8" and 5'3". He wants to be tall so he tells you he's 5'8". Are you really going to believe that?

Anonymous Said: "I used to work for a campaign. Wh...

2008-06-30T20:00:00.000-04:00

Anonymous Said: "I used to work for a campaign. While we never truly made up poll numbers when we released them, we would re-release our point estimates to be at the bounds of the margin of error so it would look better for our candidate.

So if our poll showed our candidate at 45% and the other candidate at 42% w/ a MofE of +/-4%, we would release a poll showing us at 49%, other candidate at 38%. Technically, we were giving you the 'correct' data."

This is a perfect example of what I'm talking about! Tricks like this wouldn't work if you got the raw numbers of responses. Anyone can reconstruct a poll given these responses and tricks like adding the margin of error in a completely fraudulent way to the poll results is simply totally dishonest!

It's outright lying in fact. As we know, the margin of error cuts both ways and only the most EXTREME and unlikely interpretation is that your candidate is at the extreme upward edge of the margin of error and your opponent at the extreme lower boundary. It's just intellectually dishonest to present it that way.

If you have to rely on the the internal pollster to "interpret" the results and the raw numbers aren't available, and there's no track record and we don't even have all their polls released, then the poll is completely useless and should be excluded.

Anonymous former campaign worker gives us a perfect example of WHY that should be done. In his case the campaign was flat lying. Adding the margin of error to boost your candidate and denigrate the opponent is just scientifically bogus as we all know.

They might just as well have made the whole thing up!

So, if all we get are the "results" of the poll and no actual numbers, then ABSOLUTELY YES! I would agree with everybody here that such polls cannot be included. (and quite frankly, I don't think it much matters what source category #1 or Category #5 -- NO private polling ought to be used if there's no way to verify it).

I talked in my earlier post about reliability. But, how can we judge reliability if all we get is "my candidate is up by 5 points!" and no way to assess how likely that is? I was assuming that the internal pollsters released their raw numbers. If they don't then forget about it!

Is there a possible intermediate solution here? Na...

2008-06-30T18:58:00.000-04:00

Is there a possible intermediate solution here? Nate's already weighting polls on the basis of the reliability of the pollster; might it be possible to "penalize" the reliability score for internal polling, say by weighting internal polls about the same as a poll from a company of particularly low reliability? In races which are heavily polled, internal polls then wouldn't matter very much, but in lightly-polled races (including, for example, Senate races which haven't yet attracted national attention), an internal poll would count for a bit more, just because there's so little else to go by.
All things being equal, I'd rather see as much available information incorporated into the model, as long as it's weighted appropriately.

In response to Cugel and others, please DO NOT use...

2008-06-30T18:55:00.000-04:00

In response to Cugel and others, please DO NOT use campaign internal polls, party committee internal polls, and be very way of #4 and #5 too.

I used to work for a campaign. While we never truly made up poll numbers when we released them, we would re-release our point estimates to be at the bounds of the margin of error so it would look better for our candidate.

So if our poll showed our candidate at 45% and the other candidate at 42% w/ a MofE of +/-4%, we would release a poll showing us at 49%, other candidate at 38%. Technically, we were giving you the 'correct' data.

Be careful with any of these polls. They are not representative, and they aren't even always truly correct.