No general election polls today, but Rasmussen finds Obama up 56-33 in the North Carolina primary. The arguably more interesting finding is that 56% of Clinton voters report they are not likely to vote for Barack Obama in the general election. As Rasmussen reports, "A month ago, 45% of Clinton voters said they were not likely to vote for Obama against McCain."
So should this trend be troubling for Obama?
It's not good news, certainly. There are some Clinton supporters who won't vote for Obama in the general election -- and there are also some Obama supporters who won't vote for Clinton in the general election. However, there is no evidence from this poll that their numbers are increasing.
Let's do some simple math. A month ago, 40% of North Carolina voters supported Clinton in the primaries, according to Rasmussen, and 45% of those voters said they wouldn't vote for Obama in the general election. That means that 18% of likely primary voters in North Carolina both intended to vote for Clinton in the primaries and intended to vote for McCain in the general election:
40% x 45% = 18%
Now? 56% of Clinton's supporters say they won't vote for Obama in November. But -- and this is the important point -- the percentage of Clinton supporters has gone down, from 40% to 33%. If we multiply 33% by 56% we get...
33% x 56% = 18%
...we get 18%! (Technically 18.48%). In other words, exactly the same fraction of the electorate are Type 3 voters (Clinton-McCain-Obama) as we had before.
Here's the general rule to keep in mind: when a candidate is gaining support, their support tends to be soft. That means the candidate has a lot of newly-minted support, fresh out of the oven, and such support inherently tends to be soft. Conversely, when a candidate is losing support, their support tends to be hard. Clinton now has the support of only 33% of North Carolina primary voters. But those people who are left in her camp are hard-core; all the soft support, the swing support in the state, has swung over to Obama. And it's not surprising that her remaining supporters have strong preferences for Clinton over Obama -- enough so that many of them insert McCain's name somewhere between the two Democrats.
4.05.2008
4.04.2008
Today's Polls, 4/4
@ 9:52 AM
Rasmussen has a poll out in my favorite state in the country. Obama has a 10-point over advantage, and Hillary a 5-point advantage, over John McCain in the state of Maine.
Maine is a state I'd been curious about as being a potential swing state in a Hillary-McCain matchup. Maine has a substantial anti-establishment streak, which probably works better for McCain or Obama than for Clinton, and meanwhile, Hillary's fundraising has been slack there. On the other hand, this is a state that has no trouble electing female politicians, and the polling numbers give Hillary no real reason to worry as of yet.
Also, you'll notice that the weights of the polls have changed; this is a response to the good discussion we had in this thread about the relationship between sample sizes and the pollster reliability ratings. I've done a lot of thinking about this and have come up with a solution that I think is pretty good, but am going to ask your indulgence as it's a busy time and it will probably be a couple of days before I get the chance to explain everything.
Maine is a state I'd been curious about as being a potential swing state in a Hillary-McCain matchup. Maine has a substantial anti-establishment streak, which probably works better for McCain or Obama than for Clinton, and meanwhile, Hillary's fundraising has been slack there. On the other hand, this is a state that has no trouble electing female politicians, and the polling numbers give Hillary no real reason to worry as of yet.
Also, you'll notice that the weights of the polls have changed; this is a response to the good discussion we had in this thread about the relationship between sample sizes and the pollster reliability ratings. I've done a lot of thinking about this and have come up with a solution that I think is pretty good, but am going to ask your indulgence as it's a busy time and it will probably be a couple of days before I get the chance to explain everything.
4.03.2008
Today's Polls, 4/3: A 30,000-foot look at Pennsylvania
@ 5:48 PM
Two new polls today. In New Jersey, Farleigh-Dicksinson has both Democrats leading John McCain by 5 points. This is similar to FDU's last poll of the state, which showed Clinton ahead by 5 and Obama by 4, and qualifies as slightly reassuring news for the Dems in a state where polls have shown McCain being reasonably competitive.
But the bigger news for our purposes is the Strategic Vision poll of Pennsylvania that's being reported by MSNBC and Political Wire. The poll shows Obama closing the gap in the Democratic primary matchup to 8 points -- but it also shows Hillary leading her general election trial heat by 6 points, whereas Obama trails his by 5. The poll has not been released officially yet, but I'm assuming that the leaked reports are accurate and that the sample size is the same as Strategic Vision's last poll of the state (1,200). A little bit of snap analysis below the fold.
If we look at the two polls, Strategic Vision and Quinnipiac, where we can extrapolate trendlines in Pennsylvania, we see two things:
1. Obama improved by about 10 points in his primary matchup, a result also seen in other polls of the state.
2. Hillary has improved her general election standing by a solid margin against John McCain. She went from a +2 against McCain in Quinnipiac's last poll to a +8 in yesterday's, and a -6 in Strategic Vision to a +6. Obama's general election numbers, meanwhile, have essentially been static. He went from a +2 in Quinnipiac to a +4, but fell from a -3 in Strategic Vision to a -5.
So what's going on? In the primaries, Obama is likely gaining support with some of those white Catholics that Chris Matthews likes to talk about. Frankly, I never bought that Obama was at risk of losing Pennsylvania by 20 points. Pennsylvania has generally run about 10 points behind the national polling averages for Obama. Right now, he's ahead by 3 or 4 points in the national primary polls against Clinton, and behind by 6 or 7 in the Pennsylvania polling averages, so that would account for the 10-point margin.
But Pennsylvania has a closed primary, and if I'm right, those Chris Matthews Catholics were never the vote that Obama had to worry about losing in the general election anyway. He's converting Type 1 voters (Clinton-Obama-McCain) to Type 2 voters (Obama-Clinton-McCain).
So where is Hillary gaining her general election support from? From independents? Not exactly. Take a look at the Quinnpiac internals by party identification:
There is essentially no difference in any of the individual numbers for Hillary -- she has gained two points each among independents and Republicans, but McCain has gained two points each among Democrats and Republicans as the number of undecideds has gone down. That sounds like a wash.
Unless Quinnipiac has shifted its party identification model -- which it has every reason to do, because of the surge of Democratic registration in the state. So Hillary hasn't moved up in any of the individual groups. But the size of the Democratic group is now larger, and she benefits from that more than Obama because there are still some Democrats in Pennsylvania that won't vote for Obama in the general: likely not the Chris Matthews Catholics, but the conservative Type 3 (Clinton-McCain-Obama) Democrats in the Pennsyltucky parts of the state. Pennsylvania does not have a lot of Southern Baptists, but 21.5% of its population consists of white evangelicals of other types, who may behave similarly.
At least that's what I think is going on. We should have no lack of polling data in Pennsylvania between now and the primary to track everyone's progress.
But the bigger news for our purposes is the Strategic Vision poll of Pennsylvania that's being reported by MSNBC and Political Wire. The poll shows Obama closing the gap in the Democratic primary matchup to 8 points -- but it also shows Hillary leading her general election trial heat by 6 points, whereas Obama trails his by 5. The poll has not been released officially yet, but I'm assuming that the leaked reports are accurate and that the sample size is the same as Strategic Vision's last poll of the state (1,200). A little bit of snap analysis below the fold.
If we look at the two polls, Strategic Vision and Quinnipiac, where we can extrapolate trendlines in Pennsylvania, we see two things:
1. Obama improved by about 10 points in his primary matchup, a result also seen in other polls of the state.
2. Hillary has improved her general election standing by a solid margin against John McCain. She went from a +2 against McCain in Quinnipiac's last poll to a +8 in yesterday's, and a -6 in Strategic Vision to a +6. Obama's general election numbers, meanwhile, have essentially been static. He went from a +2 in Quinnipiac to a +4, but fell from a -3 in Strategic Vision to a -5.
So what's going on? In the primaries, Obama is likely gaining support with some of those white Catholics that Chris Matthews likes to talk about. Frankly, I never bought that Obama was at risk of losing Pennsylvania by 20 points. Pennsylvania has generally run about 10 points behind the national polling averages for Obama. Right now, he's ahead by 3 or 4 points in the national primary polls against Clinton, and behind by 6 or 7 in the Pennsylvania polling averages, so that would account for the 10-point margin.
But Pennsylvania has a closed primary, and if I'm right, those Chris Matthews Catholics were never the vote that Obama had to worry about losing in the general election anyway. He's converting Type 1 voters (Clinton-Obama-McCain) to Type 2 voters (Obama-Clinton-McCain).
So where is Hillary gaining her general election support from? From independents? Not exactly. Take a look at the Quinnpiac internals by party identification:
There is essentially no difference in any of the individual numbers for Hillary -- she has gained two points each among independents and Republicans, but McCain has gained two points each among Democrats and Republicans as the number of undecideds has gone down. That sounds like a wash.
Unless Quinnipiac has shifted its party identification model -- which it has every reason to do, because of the surge of Democratic registration in the state. So Hillary hasn't moved up in any of the individual groups. But the size of the Democratic group is now larger, and she benefits from that more than Obama because there are still some Democrats in Pennsylvania that won't vote for Obama in the general: likely not the Chris Matthews Catholics, but the conservative Type 3 (Clinton-McCain-Obama) Democrats in the Pennsyltucky parts of the state. Pennsylvania does not have a lot of Southern Baptists, but 21.5% of its population consists of white evangelicals of other types, who may behave similarly.
At least that's what I think is going on. We should have no lack of polling data in Pennsylvania between now and the primary to track everyone's progress.
Polls Now Weighted by Sample Size
@ 4:07 AM
Let me show you a catchy new graphic and then I'll explain what's going on.
You'll see that, in addition to listing the date of the poll, we also list the sample size. And not only do we list the sample size, but we also weight the poll in part based on its sample size.
This is something I'd thought about doing all along, but a couple of the commenters had been prodding me on this issue in light of some recent polling that had particularly large sample sizes. The recent PPIC poll in California, for instance, had more than 1,000 respondents, and Quinnipiac's sample sizes have been enormous -- they surveyed 3,484 people in their latest Pennslvania poll!
Now, I don't quite assume a linear relationship between sample size and a poll's reliability. For one thing, different pollsters have different habits about how many respondents they include -- all of Rasmussen's trial heat polls have included exactly 500 people, for example, while Strategic Vision favors sample sizes of either 800 or 1,200. And so, the information about the sample sizes may already be embedded to a certain extent in the pollster's reliability rating, meaning that we might wind up double-counting the impact of sample size if we're not careful. For another thing, sample sizes are just one source of error, with methodological error perhaps be the more important kind. I'm convinced that Zogby or ARG could survey all 325 million American citizens and still manage to fuck things up somehow.
Clearly though, if one poll is going to survey 3,484 voters and another poll is going to survey 348, the bigger survey ought to get some kind of extra credit. So the formula I use is
(sample/600) ^ 0.5
e.g. the poll's sample size, divided by 600, and taken to its square root. The number 600 is chosen because it represents an "average" sample size; it also represents a margin of error of exactly +/- 4. And taking things to the square root has the effect of accounting for diminishing returns. The Quinnipiac poll of Pennsylvania comes out to 2.41 under this formula, for instance, whereas a typical Rasmussen poll of 500 respondents works out to 0.91.
So we now have three factors we use to determine the weight a poll is given in our model:
1. Its sample size, subject to the calculation I described above.
2. Its pollster reliability rating.
3. Its recentness rating, as described in the FAQ. Note that "old" polls from the same polling firm are punished under our recentness formula (FYI, I have validated based on data from previous election cycles that this tends to improve the accuracy of our estimates more than either throwing out "old" polls from the same agency, or not penalizing them at all).
The Quinnipiac poll, for instance, has a sample size rating of 2.41, a reliability rating of 0.95, and a recentness rating of 0.91; we multiply these three numbers together to get its overall weighting, which is 2.10. In the interests of transparency, I'm now explicitly listing the weighting I'm using for each and every poll in our data table (and I'm now calling it 'weight' rather than 'reliability', since the pollster's reliability score is just one of three factors we use in the weighting calculation).
Yes, I know this stuff gets complicated. But that's why you come here, right?
And it does turn out to make some difference. Texas, for instance, had been showing up on Obama's Swing State List in part because of a Survey USA poll that showed him trailing McCain by just 1 in that state. This is the most recent Survey USA poll, and Survey USA is a very reliable polling agency, so this poll does deserve some weight. However, this poll had a sample size of exactly 600, whereas there were a number of Texas polls, taken right around the same time, that have exceptionally large sample sizes. An IVR poll that showed Obama trailing McCain by 22 had more than 2,900 respondents; CNN conducted a poll with 1,500 respondents, and a previous Survey USA poll had 1,725. These large sample size polls now have a comparatively larger weight, and as such Obama's numbers have dropped a couple points in Texas, enough to take it off his Swing State List.
Overall, however, including the sample sizes has turned out to be helpful to both Democrats, particularly on the strength of the large-sample Quinnipiac polls in places like Ohio, Pennsylvania, and New Jersey.
You'll see that, in addition to listing the date of the poll, we also list the sample size. And not only do we list the sample size, but we also weight the poll in part based on its sample size.
This is something I'd thought about doing all along, but a couple of the commenters had been prodding me on this issue in light of some recent polling that had particularly large sample sizes. The recent PPIC poll in California, for instance, had more than 1,000 respondents, and Quinnipiac's sample sizes have been enormous -- they surveyed 3,484 people in their latest Pennslvania poll!
Now, I don't quite assume a linear relationship between sample size and a poll's reliability. For one thing, different pollsters have different habits about how many respondents they include -- all of Rasmussen's trial heat polls have included exactly 500 people, for example, while Strategic Vision favors sample sizes of either 800 or 1,200. And so, the information about the sample sizes may already be embedded to a certain extent in the pollster's reliability rating, meaning that we might wind up double-counting the impact of sample size if we're not careful. For another thing, sample sizes are just one source of error, with methodological error perhaps be the more important kind. I'm convinced that Zogby or ARG could survey all 325 million American citizens and still manage to fuck things up somehow.
Clearly though, if one poll is going to survey 3,484 voters and another poll is going to survey 348, the bigger survey ought to get some kind of extra credit. So the formula I use is
(sample/600) ^ 0.5
e.g. the poll's sample size, divided by 600, and taken to its square root. The number 600 is chosen because it represents an "average" sample size; it also represents a margin of error of exactly +/- 4. And taking things to the square root has the effect of accounting for diminishing returns. The Quinnipiac poll of Pennsylvania comes out to 2.41 under this formula, for instance, whereas a typical Rasmussen poll of 500 respondents works out to 0.91.
So we now have three factors we use to determine the weight a poll is given in our model:
1. Its sample size, subject to the calculation I described above.
2. Its pollster reliability rating.
3. Its recentness rating, as described in the FAQ. Note that "old" polls from the same polling firm are punished under our recentness formula (FYI, I have validated based on data from previous election cycles that this tends to improve the accuracy of our estimates more than either throwing out "old" polls from the same agency, or not penalizing them at all).
The Quinnipiac poll, for instance, has a sample size rating of 2.41, a reliability rating of 0.95, and a recentness rating of 0.91; we multiply these three numbers together to get its overall weighting, which is 2.10. In the interests of transparency, I'm now explicitly listing the weighting I'm using for each and every poll in our data table (and I'm now calling it 'weight' rather than 'reliability', since the pollster's reliability score is just one of three factors we use in the weighting calculation).
Yes, I know this stuff gets complicated. But that's why you come here, right?
And it does turn out to make some difference. Texas, for instance, had been showing up on Obama's Swing State List in part because of a Survey USA poll that showed him trailing McCain by just 1 in that state. This is the most recent Survey USA poll, and Survey USA is a very reliable polling agency, so this poll does deserve some weight. However, this poll had a sample size of exactly 600, whereas there were a number of Texas polls, taken right around the same time, that have exceptionally large sample sizes. An IVR poll that showed Obama trailing McCain by 22 had more than 2,900 respondents; CNN conducted a poll with 1,500 respondents, and a previous Survey USA poll had 1,725. These large sample size polls now have a comparatively larger weight, and as such Obama's numbers have dropped a couple points in Texas, enough to take it off his Swing State List.
Overall, however, including the sample sizes has turned out to be helpful to both Democrats, particularly on the strength of the large-sample Quinnipiac polls in places like Ohio, Pennsylvania, and New Jersey.
It's 3 AM: Time to Fire Your Creative Director
@ 12:18 AM
See if you can pick out the four real advertisements from the four parodies. Tougher than you'd think!
4.02.2008
Today's Polls: Great News for Hillary
@ 9:08 PM
Most of the interesting polling lately has been on the primary side of the ledger, but the Clinton campaign has to be heartened by today's series of Quinnipiac polls in Ohio, Florida, and Pennsylvania:
While these polls aren't bad news for Obama -- his Florida result is poor, but that is a state in which we'd already projected an uphill battle for him -- these are nevertheless exactly the sort of numbers that Clinton is going to want to see if she is to make a credible case about electability to the superdelegates.
The caution is that polling in each of these states has been fairly abundant, and we have Quinnipiac rated as a good-but-not-great pollster, so they don't move the state-by-state estimates all that much. Still, every little bit counts in big states like these. On the strength of these numbers, Hillary's win percentage has improved from 53% to 59% in Ohio; from 47% to 52% in Pennsylvania, and from 37% to 41% in Florida. Obama's win percentage has fallen from 20% to 17% in Florida, but risen from 37% to 39% on Ohio, and from 45% to 47% in Pennsylvania.
| State | Agency | Reliability | Date | Obama | Clinton |
| FL | Quinnipiac | **** | 3/28 | McCain +9 | Clinton +2 |
| OH | Quinnipiac | **** | 3/28 | Obama +1 | Clinton +9 |
| PA | Quinnipiac | **** | 3/28 | Obama +4 | Clinton +8 |
While these polls aren't bad news for Obama -- his Florida result is poor, but that is a state in which we'd already projected an uphill battle for him -- these are nevertheless exactly the sort of numbers that Clinton is going to want to see if she is to make a credible case about electability to the superdelegates.
The caution is that polling in each of these states has been fairly abundant, and we have Quinnipiac rated as a good-but-not-great pollster, so they don't move the state-by-state estimates all that much. Still, every little bit counts in big states like these. On the strength of these numbers, Hillary's win percentage has improved from 53% to 59% in Ohio; from 47% to 52% in Pennsylvania, and from 37% to 41% in Florida. Obama's win percentage has fallen from 20% to 17% in Florida, but risen from 37% to 39% on Ohio, and from 45% to 47% in Pennsylvania.
Site Fixes
@ 4:21 AM
Our regular readers should notice a few differences ... I had to do something to expand the width of the main column and make this look like a real blog. But there are also some goodies; you should now be able to expand and contract both posts and comments from the main page.
NOTE: Internet Explorer 6 users: Please download Internet Explorer 7.
NOTE: Internet Explorer 6 users: Please download Internet Explorer 7.
4.01.2008
The Reverse Bradley Effect: Fact or Fiction?
@ 8:25 PM
Over at The Stump, Noam Scheiber tracked down a Pew Research study that purports to identify the presence of both a Bradley Effect and a Reverse Bradley Effect in the results to date in the Democratic primaries. For those of you who are not familiar with the Bradley Effect, the long and short of it is the idea that black candidates tend to systematically underperform their polls on election day, perhaps because people lie to interviewers in surveys so as to be politically correct. The Reverse Bradley Effect would be -- well, just the reverse -- a black candidate systematically overperforming his poll numbers for some or another reason.
The gist of the Pew study can be found in the chart below (larger version here), which looks at the extent to which Barack Obama overperformed or underperformed his polls in a number of primary states, and compares that to the African-American population in the state. The authors report that Obama has outperformed his polls in states with high African-American populations, and underperformed them in states with low ones. They find a correlation of .74, which is quite high -- it would imply that more than half of the state-by-state polling errors are explained by the racial composition of that state alone.
I have a couple of issues with the way this study was conducted. Actually, just one issue, but it's a pretty big one. The authors seem to have cherry-picked their states. They exclude states like Connecticut, Maryland, and New York that had closed primaries, but give no explanation as to why. They do not include Vermont, even though it appears to have met their standard of having three polls conducted in the week before the election. They don't include Florida which, officially-sanctioned primary or no, would seem to be as useful as any other data point insofar as the Bradley Effect goes. They don't include the caucus states of Iowa and Nevada (there is actually a decent argument for that, since caucuses are conducted in public rather than the privacy of a voting booth, but I would tend to be inclusive rather than exclusive when testing my own hypothesis). They do include the "outlier" of Wisconsin, but seem to be annoyed by it -- as though it's Wisconsin's fault for not conforming to their hypothesis.
So, I attempted to recreate their analysis, pulling my own numbers from Pollster.com, and adding back in the blacklisted states. My imitation of their graph is below:
Putting these other states back in turns out not to make all that much difference; the correlation drops from .74 to .62. Still, we notice the presence of a few more states, like Vermont and Iowa, that don't seem to fit the hypothesis.
You'll notice I've done something else too, which is to color code the graph. The states in blue are those that we define as Southern (this includes Florida and Virginia, but not Maryland or Missouri), whereas those in red are the rest of the country. The claim for the Reverse Bradley Effect is really just based on the strong pull exerted by five states: Virginia, Alabama, South Carolina, Georgia, and Mississippi. All those states have high black populations, but they also have another thing in common, which is that they are all Southern.
But what if we look at other states that have relatively high black populations, but are not in the South? New York and Illinois have fairly substantial black populations -- but the polls were spot-on in each of those locations. In Maryland -- which I consider a Northern state, and which demographically has much more in common with other Northern states than anything in the South -- Obama outperformed his polls, but only barely so. New Jersey has a relatively large black population and Obama slightly underperformed his polls there.
So instead of drawing one regression line, let's draw two: one to represent the South and the other to represent the rest of the country.
Well, it looks like we are dealing with two completely different sets of behaviors. The relationship in the South is quite strong -- just as strong as the Pew authors found originally. But there is virtually no relationship between race and Obama's performance at the ballot booth elsewhere in the country -- the slight correlation you see is nowhere near statistically significant. The polls overestimated Obama's performance in New Hampshire and Massachusetts -- but underestimated it in Vermont and Wisconsin. They were largely accurate in states like New York and Maryland that have substantial black populations. There is just nothing happening with the Bradley Effect outside of the South, at least so far as this data can tell us.
Also of note: my study identifies the presence of a Reverse Bradley Effect in the South (Obama outperforming his polls in states with higher percentages of black voters), but not the presence of the plain ol' Bradley Effect in either the North or the South. At no point do my regression lines for either region run substantially above zero, which is the point at which Obama would begin to underperform his polls.
Put differently: there is nowhere in the country where we have reason to subtract from Obama's poll numbers based on the Bradley Effect. (Yes, Obama has underperformed his polls in some "white" states -- but he has overperformed them in others that are whiter than Kurt Rambis). On the other hand, there is one specific group of states where we might want to add to Obama's polls based on the Reverse Bradley Effect, which are Southern states with high African-American populations.
If the Reverse Bradley Effect is real, what is its raison d'être? There is a fair amount of academic literature on the effects of the race of the interviewer on survey results. People can guess quite accurately the race of the person on the other end of the line, and they might respond differently depending on that perception. It is probably safe to assume that the majority of interviewers are white, and correctly perceived as white. A white voter might not want to tell a (presumably) white interviewer that they're voting for the black guy. Or, a black voter might feel intimidated by a (presumably) white interviewer, and not want to tell him they're voting for the black guy. These effects may be far more tangible in the South, in which race is a much more explicit consideration in everyday life. Three other quick comments on this:
1. If I had to guess, I would guess that black voters might be more likely not to want to reveal their true candidate choice than white voters. This is because I have noticed that a lot of black voters tend to be classified as 'undecided' in pre-primary surveys, which might indicate the hedging of bets.
2. As Scheiber and others have noted, there appears to be less of a Reverse Bradley Effect in polls conducted by agencies like Rasmussen, Survey USA and PPP that use automated calling scripts (robopolls). This requires further research -- but if true it would tend to validate my hypothesis.
3. Something else that is worth mentioning: Americans tend to think that other people have more racial hangups than they claim to have themselves. This might be why there is a Reverse Bradley Effect, rather than a Bradley Effect. You don't think you're a racist -- but you think the person on the other end of the line might be, and so you lie about your candidate choice so as not to offend them.
With all that having been said, I may be overselling the Reverse Bradley Effect. What we know is that in Southern states with large black populations, Obama has outperformed his polls by a statistically significant margin. But we're just guessing at why this is the case. It could be because the pollster's turnout models are screwed up. It could be because minority voters are more likely to use cellphones as their primary line, and therefore won't make it into surveys. It could be because of something I call the Frontrunner Effect, which is that there is some tendency for candidates who are already ahead in the polls to run up the score on election day.
Nor do we know if we have identified a universal effect, or whether it's something specific to Obama and Clinton. Perhaps Southern voters feel badly voting against Hillary Clinton, who claims some heritage in the region. But the same might not be true when Obama is matched against John McCain, whom Southerners tend to feel lukewarm about. Or, it could be that there is a Reverse Bradley Effect among Democrats but a real Bradley Effect among Republicans, and the two things will cancel one another out in the general election. If you had to bet against the spread, however, you might want to take the over on Obama's numbers in states like North Carolina and Virginia.
The gist of the Pew study can be found in the chart below (larger version here), which looks at the extent to which Barack Obama overperformed or underperformed his polls in a number of primary states, and compares that to the African-American population in the state. The authors report that Obama has outperformed his polls in states with high African-American populations, and underperformed them in states with low ones. They find a correlation of .74, which is quite high -- it would imply that more than half of the state-by-state polling errors are explained by the racial composition of that state alone.
I have a couple of issues with the way this study was conducted. Actually, just one issue, but it's a pretty big one. The authors seem to have cherry-picked their states. They exclude states like Connecticut, Maryland, and New York that had closed primaries, but give no explanation as to why. They do not include Vermont, even though it appears to have met their standard of having three polls conducted in the week before the election. They don't include Florida which, officially-sanctioned primary or no, would seem to be as useful as any other data point insofar as the Bradley Effect goes. They don't include the caucus states of Iowa and Nevada (there is actually a decent argument for that, since caucuses are conducted in public rather than the privacy of a voting booth, but I would tend to be inclusive rather than exclusive when testing my own hypothesis). They do include the "outlier" of Wisconsin, but seem to be annoyed by it -- as though it's Wisconsin's fault for not conforming to their hypothesis.
So, I attempted to recreate their analysis, pulling my own numbers from Pollster.com, and adding back in the blacklisted states. My imitation of their graph is below:
Putting these other states back in turns out not to make all that much difference; the correlation drops from .74 to .62. Still, we notice the presence of a few more states, like Vermont and Iowa, that don't seem to fit the hypothesis.
You'll notice I've done something else too, which is to color code the graph. The states in blue are those that we define as Southern (this includes Florida and Virginia, but not Maryland or Missouri), whereas those in red are the rest of the country. The claim for the Reverse Bradley Effect is really just based on the strong pull exerted by five states: Virginia, Alabama, South Carolina, Georgia, and Mississippi. All those states have high black populations, but they also have another thing in common, which is that they are all Southern.
But what if we look at other states that have relatively high black populations, but are not in the South? New York and Illinois have fairly substantial black populations -- but the polls were spot-on in each of those locations. In Maryland -- which I consider a Northern state, and which demographically has much more in common with other Northern states than anything in the South -- Obama outperformed his polls, but only barely so. New Jersey has a relatively large black population and Obama slightly underperformed his polls there.
So instead of drawing one regression line, let's draw two: one to represent the South and the other to represent the rest of the country.
Well, it looks like we are dealing with two completely different sets of behaviors. The relationship in the South is quite strong -- just as strong as the Pew authors found originally. But there is virtually no relationship between race and Obama's performance at the ballot booth elsewhere in the country -- the slight correlation you see is nowhere near statistically significant. The polls overestimated Obama's performance in New Hampshire and Massachusetts -- but underestimated it in Vermont and Wisconsin. They were largely accurate in states like New York and Maryland that have substantial black populations. There is just nothing happening with the Bradley Effect outside of the South, at least so far as this data can tell us.
Also of note: my study identifies the presence of a Reverse Bradley Effect in the South (Obama outperforming his polls in states with higher percentages of black voters), but not the presence of the plain ol' Bradley Effect in either the North or the South. At no point do my regression lines for either region run substantially above zero, which is the point at which Obama would begin to underperform his polls.
Put differently: there is nowhere in the country where we have reason to subtract from Obama's poll numbers based on the Bradley Effect. (Yes, Obama has underperformed his polls in some "white" states -- but he has overperformed them in others that are whiter than Kurt Rambis). On the other hand, there is one specific group of states where we might want to add to Obama's polls based on the Reverse Bradley Effect, which are Southern states with high African-American populations.
If the Reverse Bradley Effect is real, what is its raison d'être? There is a fair amount of academic literature on the effects of the race of the interviewer on survey results. People can guess quite accurately the race of the person on the other end of the line, and they might respond differently depending on that perception. It is probably safe to assume that the majority of interviewers are white, and correctly perceived as white. A white voter might not want to tell a (presumably) white interviewer that they're voting for the black guy. Or, a black voter might feel intimidated by a (presumably) white interviewer, and not want to tell him they're voting for the black guy. These effects may be far more tangible in the South, in which race is a much more explicit consideration in everyday life. Three other quick comments on this:
1. If I had to guess, I would guess that black voters might be more likely not to want to reveal their true candidate choice than white voters. This is because I have noticed that a lot of black voters tend to be classified as 'undecided' in pre-primary surveys, which might indicate the hedging of bets.
2. As Scheiber and others have noted, there appears to be less of a Reverse Bradley Effect in polls conducted by agencies like Rasmussen, Survey USA and PPP that use automated calling scripts (robopolls). This requires further research -- but if true it would tend to validate my hypothesis.
3. Something else that is worth mentioning: Americans tend to think that other people have more racial hangups than they claim to have themselves. This might be why there is a Reverse Bradley Effect, rather than a Bradley Effect. You don't think you're a racist -- but you think the person on the other end of the line might be, and so you lie about your candidate choice so as not to offend them.
With all that having been said, I may be overselling the Reverse Bradley Effect. What we know is that in Southern states with large black populations, Obama has outperformed his polls by a statistically significant margin. But we're just guessing at why this is the case. It could be because the pollster's turnout models are screwed up. It could be because minority voters are more likely to use cellphones as their primary line, and therefore won't make it into surveys. It could be because of something I call the Frontrunner Effect, which is that there is some tendency for candidates who are already ahead in the polls to run up the score on election day.
Nor do we know if we have identified a universal effect, or whether it's something specific to Obama and Clinton. Perhaps Southern voters feel badly voting against Hillary Clinton, who claims some heritage in the region. But the same might not be true when Obama is matched against John McCain, whom Southerners tend to feel lukewarm about. Or, it could be that there is a Reverse Bradley Effect among Democrats but a real Bradley Effect among Republicans, and the two things will cancel one another out in the general election. If you had to bet against the spread, however, you might want to take the over on Obama's numbers in states like North Carolina and Virginia.
A Game-Changer in Ohio?
@ 7:48 AM
Okay, so it's not what you're thinking. It's not a poll.
But sometimes in our frantic search for polls, we uncover other interesting things.
There's a piece of legislation in Ohio called the Ohio Healthy Families Act. The legislation would require Ohio employers to give their employees a minimum of seven paid sick days per year.
You might think this seems like a fairly commonsensical and populist piece of policy in tough economic times. And you'd be right that it has the support of the majority of the state. According to the Zanesville Times Recorder, 76 percent of Democrats and 45 percent of Republicans support the initiative.
But, also according to the Zanesville Times Recorder, the legislation is likely to be rejected by the Ohio General Assembly, which tilts heavily Republican, especially in the State Senate (where it's R 21, D 12).
And so, assuming that the proposal is rejected, the SEIU is going to try and collect 120,000 signatures to put it on the November ballot. Given the organizational resources of a union like the SEIU, and the historical affection for unions in the industrial Midwest, this would seem to be little problem.
Now, I don't think that the presence of this ballot initiative is likely to increase turnout all that much. But I do think it's likely to trigger some reasonably significant backlash against Republicans, and give either Democratic candidate an easier time casting themselves as the champion of Ohio's working class. This is especially the case, obviously, if McCain comes out against the proposal, which I have no idea about.
Anyway, just something to keep an eye out for. It's a little thing, but just the sort of little thing that can swing elections.
But sometimes in our frantic search for polls, we uncover other interesting things.
There's a piece of legislation in Ohio called the Ohio Healthy Families Act. The legislation would require Ohio employers to give their employees a minimum of seven paid sick days per year.
You might think this seems like a fairly commonsensical and populist piece of policy in tough economic times. And you'd be right that it has the support of the majority of the state. According to the Zanesville Times Recorder, 76 percent of Democrats and 45 percent of Republicans support the initiative.
But, also according to the Zanesville Times Recorder, the legislation is likely to be rejected by the Ohio General Assembly, which tilts heavily Republican, especially in the State Senate (where it's R 21, D 12).
And so, assuming that the proposal is rejected, the SEIU is going to try and collect 120,000 signatures to put it on the November ballot. Given the organizational resources of a union like the SEIU, and the historical affection for unions in the industrial Midwest, this would seem to be little problem.
Now, I don't think that the presence of this ballot initiative is likely to increase turnout all that much. But I do think it's likely to trigger some reasonably significant backlash against Republicans, and give either Democratic candidate an easier time casting themselves as the champion of Ohio's working class. This is especially the case, obviously, if McCain comes out against the proposal, which I have no idea about.
Anyway, just something to keep an eye out for. It's a little thing, but just the sort of little thing that can swing elections.
3.31.2008
State-by-State Fundraising Numbers
@ 11:35 PM
One of the metrics we use in our regression analysis is the state-by-state fundraising totals received by each of the three major candidates. These fundraising numbers turn out have a highly statistically significant relationship with their standing in the polls, even after accounting for things like the party identification in the state, and the Kerry-Bush voting tally in 2004.
The FEC has recently updated its website, including fresh fundraising data through 2/29/08. February was a huge fundraising month for both Barack Obama and Hillary Clinton -- less so for John McCain -- so this gives us a better idea of where each candidate's strengths and weaknesses lie.
The way I look at fundraising is in terms of dollars raised per 2004 general election voter. So in Missouri, for example, Obama has raised just over $1.3 million dollars, and whereas there were 2.7 voters in the 2004 general election. This works out to about 50¢ per voter (actually 49¢).
In the abstract, the fundraising numbers are very impressive for the Democrats. McCain has outfundraised Obama in only four states: Arizona, Michigan, Mississippi, and South Carolina. He's outraised Clinton in each of those states, plus Idaho, Kansas, Montana, Wyoming, and North and South Dakota. However, it's the relative numbers that count for purposes of our regression model. On that front, here are Obama's Top 10 -- and bottom 5 -- fundraising states.
Barack Obama, Funds Raised per GE voter
1. Illinois $2.85
2. New York $2.24
3. California $2.15
4. Connecticut $2.07
5. Vermont $2.01
6. Massachusetts $1.83
7. Hawaii $1.65
8. Maryland $1.61
9. Virginia $1.38
10. Washington $1.38
46. Mississippi $0.19
47. Louisiana $0.16
48. West Virginia $0.14
49. Arkansas $0.14
50. North Dakota $0.14
No real surprises on the top of the list -- these are mainline, Democratic states that also happen to be quite wealthy. The one exception is possibly Virginia, but Virginians tend to donate a lot of money to all Presidential candidates because of their connections with the Beltway. At the bottom of the list are several states from our Deep South region, plus North Dakota -- where, even though Obama hasn't raised very much money, he's still raised more money than Clinton and McCain combined. North Dakotans just have better things to do than to part with their money for Presidential candidates, I guess.
Next, Hillary's list:
Hillary Clinton, Funds Raised per GE voter
1. New York $3.77
2. California $1.88
3. Maryland $1.77
4. New Jersey $1.67
5. Connecticut $1.54
6. Rhode Island $1.35
7. Virginia $1.30
8. Arkansas $1.28
9. Massachusetts $1.10g
10. Florida $1.09
46. Wisconsin $0.16
47. Montana $0.11
48. South Dakota $0.10
49. Mississippi $0.09
50. North Dakota $0.06
Hillary's fundraising prowess in New York is astounding -- it's as though every single voter in the state decided to pony up and buy her a Value Meal. That's what happens when you combine a state that inherently should be kind to any Democrat with Manhattan money and being that state's representative in the Senate. Also, her Florida numbers should be considered encouraging. The shocker is Wisconsin, which ranks at #46 on Hillary's list. I don't know what it is about Wisconsin. The money in the state, I guess, is in Madison, Milwaukee, and parts of the Fox River Valley. Madison is an Obama town, the Fox River Valley is sort of half-way between NASCAR and Country Club Republican, and perhaps the business elites in Milwaukee have a lot of ties to Chicago? Either way, it does not portend well for Hillary in Wisconsin, where her polling against McCain has been sluggish.
John McCain, Funds Raised per GE voter
1. Arizona $1.74
2. Virginia $0.96
3. Connecticut $0.95
4. New York $0.71
5. Texas $0.65
6. California $0.55
7. Florida $0.52
8. New Jersey $0.49
9. Nevada $0.49
10. South Carolina $0.48
46. Nebraska $0.10
47. Iowa $0.08
48. North Dakota $0.07
49. Kentucky $0.07
50. West Virginia $0.06
On John McCain's list, we see perhaps more relationship with the states in which he ran an active primary campaign -- South Carolina is 10th on his list (and New Hampshire is 11th), whereas voters in Iowa seem to have punished him for blowing off the state. Otherwise, we see a largely predictable list of states -- and a reminder that there is plenty of money to go around in blue states like California and New York.
The complete list of numbers is below. I've presented them two ways: first, the basic funds-raised-per-voter calculation I described above, and second, on a relative basis, where 100 represents the average state for that candidate. Thus we see that, for example, while Obama has raised more funds in Alabama than John McCain, it's relatively speaking a stronger state for McCain; Obama is so far ahead everywhere in the fundraising game that this will be the case any time McCain keeps it reasonably close.
A few thoughts on the above list:
Arkansas really seems to be almost punitive toward Obama -- this has also held in the polling numbers.
It's not clear that the lack of an officially-sanctioned primary has harmed the Democrats in Florida. Clinton's numbers are pretty strong. Obama has raised slightly less than his state average, but the demographics in Florida aren't great for him to begin with. In Michigan, on the other hand, there clearly seem to be some Democrats who are punishing their candidates for the lack of a competitive primary there. I don't know how else to explain how McCain outfundrasies both candidates in a state that has voted Blue each year since 1992, when he's otherwise been unable to outfundraise any Democrat, anywhere.
The Massachusetts numbers should be encouraging to Obama. Perhaps it's a bunch of Harvard Law grads contributing their $2,300, but this fundraising is a big reason why our regression model just doesn't buy that McCain can keep the state competitive against Obama, as some polls have suggested.
Everyone's numbers are low in Ohio, which seems to be suffering from a combination of economic hardship and Swing State Fatigue.
Another state where I'd be encouraged by these numbers if I were Obama: Colorado. Another state where I'd be worried: New Jersey.
p.s. Swapping in these new fundraising numbers does have some slight effect on our overall analysis, so I've refreshed the charts and graphs.
The FEC has recently updated its website, including fresh fundraising data through 2/29/08. February was a huge fundraising month for both Barack Obama and Hillary Clinton -- less so for John McCain -- so this gives us a better idea of where each candidate's strengths and weaknesses lie.
The way I look at fundraising is in terms of dollars raised per 2004 general election voter. So in Missouri, for example, Obama has raised just over $1.3 million dollars, and whereas there were 2.7 voters in the 2004 general election. This works out to about 50¢ per voter (actually 49¢).
In the abstract, the fundraising numbers are very impressive for the Democrats. McCain has outfundraised Obama in only four states: Arizona, Michigan, Mississippi, and South Carolina. He's outraised Clinton in each of those states, plus Idaho, Kansas, Montana, Wyoming, and North and South Dakota. However, it's the relative numbers that count for purposes of our regression model. On that front, here are Obama's Top 10 -- and bottom 5 -- fundraising states.
Barack Obama, Funds Raised per GE voter
1. Illinois $2.85
2. New York $2.24
3. California $2.15
4. Connecticut $2.07
5. Vermont $2.01
6. Massachusetts $1.83
7. Hawaii $1.65
8. Maryland $1.61
9. Virginia $1.38
10. Washington $1.38
46. Mississippi $0.19
47. Louisiana $0.16
48. West Virginia $0.14
49. Arkansas $0.14
50. North Dakota $0.14
No real surprises on the top of the list -- these are mainline, Democratic states that also happen to be quite wealthy. The one exception is possibly Virginia, but Virginians tend to donate a lot of money to all Presidential candidates because of their connections with the Beltway. At the bottom of the list are several states from our Deep South region, plus North Dakota -- where, even though Obama hasn't raised very much money, he's still raised more money than Clinton and McCain combined. North Dakotans just have better things to do than to part with their money for Presidential candidates, I guess.
Next, Hillary's list:
Hillary Clinton, Funds Raised per GE voter
1. New York $3.77
2. California $1.88
3. Maryland $1.77
4. New Jersey $1.67
5. Connecticut $1.54
6. Rhode Island $1.35
7. Virginia $1.30
8. Arkansas $1.28
9. Massachusetts $1.10g
10. Florida $1.09
46. Wisconsin $0.16
47. Montana $0.11
48. South Dakota $0.10
49. Mississippi $0.09
50. North Dakota $0.06
Hillary's fundraising prowess in New York is astounding -- it's as though every single voter in the state decided to pony up and buy her a Value Meal. That's what happens when you combine a state that inherently should be kind to any Democrat with Manhattan money and being that state's representative in the Senate. Also, her Florida numbers should be considered encouraging. The shocker is Wisconsin, which ranks at #46 on Hillary's list. I don't know what it is about Wisconsin. The money in the state, I guess, is in Madison, Milwaukee, and parts of the Fox River Valley. Madison is an Obama town, the Fox River Valley is sort of half-way between NASCAR and Country Club Republican, and perhaps the business elites in Milwaukee have a lot of ties to Chicago? Either way, it does not portend well for Hillary in Wisconsin, where her polling against McCain has been sluggish.
John McCain, Funds Raised per GE voter
1. Arizona $1.74
2. Virginia $0.96
3. Connecticut $0.95
4. New York $0.71
5. Texas $0.65
6. California $0.55
7. Florida $0.52
8. New Jersey $0.49
9. Nevada $0.49
10. South Carolina $0.48
46. Nebraska $0.10
47. Iowa $0.08
48. North Dakota $0.07
49. Kentucky $0.07
50. West Virginia $0.06
On John McCain's list, we see perhaps more relationship with the states in which he ran an active primary campaign -- South Carolina is 10th on his list (and New Hampshire is 11th), whereas voters in Iowa seem to have punished him for blowing off the state. Otherwise, we see a largely predictable list of states -- and a reminder that there is plenty of money to go around in blue states like California and New York.
The complete list of numbers is below. I've presented them two ways: first, the basic funds-raised-per-voter calculation I described above, and second, on a relative basis, where 100 represents the average state for that candidate. Thus we see that, for example, while Obama has raised more funds in Alabama than John McCain, it's relatively speaking a stronger state for McCain; Obama is so far ahead everywhere in the fundraising game that this will be the case any time McCain keeps it reasonably close.
A few thoughts on the above list:
Arkansas really seems to be almost punitive toward Obama -- this has also held in the polling numbers.
It's not clear that the lack of an officially-sanctioned primary has harmed the Democrats in Florida. Clinton's numbers are pretty strong. Obama has raised slightly less than his state average, but the demographics in Florida aren't great for him to begin with. In Michigan, on the other hand, there clearly seem to be some Democrats who are punishing their candidates for the lack of a competitive primary there. I don't know how else to explain how McCain outfundrasies both candidates in a state that has voted Blue each year since 1992, when he's otherwise been unable to outfundraise any Democrat, anywhere.
The Massachusetts numbers should be encouraging to Obama. Perhaps it's a bunch of Harvard Law grads contributing their $2,300, but this fundraising is a big reason why our regression model just doesn't buy that McCain can keep the state competitive against Obama, as some polls have suggested.
Everyone's numbers are low in Ohio, which seems to be suffering from a combination of economic hardship and Swing State Fatigue.
Another state where I'd be encouraged by these numbers if I were Obama: Colorado. Another state where I'd be worried: New Jersey.
p.s. Swapping in these new fundraising numbers does have some slight effect on our overall analysis, so I've refreshed the charts and graphs.
Rasmussen, Survey USA, Enthusiasm and Likely Voters
@ 11:37 AM
Rasmussen and Survey USA are responsible for the vast majority of polling data that you see on this site. Fortunately for us, they also happen to occupy a couple of the top spots in our pollster ratings.
However, as I've alluded to a couple of times, there have come to be some systematic differences in the numbers each Democrat has received from each agency. The difference is particularly notable for Hillary Clinton.
Let's run a couple of straightforward charts. First, these are Obama's numbers against McCain in the most recent Rasmussen and Survey USA polls, in states that both pollsters have surveyed since Super Tuesday.
Obama vs McCain
On average, Obama leads McCain by 1.2 points in the Survey USA polls, but trails him by 1.5 points in the Rasmussen polls -- a difference of 2.7 points. Although I think that difference would hold up as statistically significant if we ran a t-test, it's not a huge gap, especially considering that the Rasmussen surveys are on balance slightly more recent, and therefore more likely to have been impacted by the Jeremiah Wright fallout and so forth. (If we look only at those states that both pollsters have surveyed at some point in March, the gap between the two agencies falls to only about 1 point).
It's in Hillary Clinton's numbers where we see the bigger differences:
Clinton vs McCain
Clinton's Survey USA numbers have been about 7 points better on average than her Rasmussen numbers. Look at how devastating the Rasmussen electoral map would be to Hillary: she's losing Pennsylvania, Michigan, Florida and Ohio; she's losing New Mexico, Iowa, Virginia and Wisconsin by double digits. She's losing "core" Clinton states like Arkansas and New Jersey. By contrast, Survey USA has her tied or ahead in almost all of those states (Iowa and Colorado are the exceptions).
So, bellows the choir, who's right? I would caution against any categorical answer to this question (e.g. "Rasmussen is a Republican hack pollster" or "Survey USA's results are really goofy"). Both of these pollsters know what they're doing. Survey USA's polls were devastatingly accurate in this year's primaries, which is why they sit atop our pollster rankings. But primaries and general elections are completely different beasts, and Rasmussen performed better than Survey USA in the 2004 general. There are, however, two important differences between the two pollsters of which I'm aware.
Firstly, as has been helpfully pointed out by a couple of commenters, Rasmussen is already applying a likely voter model, whereas Survey USA includes all registered voters. I'm an agnostic on the issue likely voter models. On the one hand, you sometimes hear it said that "if everybody turned out, the Democrats would win every election in a landslide". I have no idea whether that's actually true or not, especially since Democrats have started to outperform Republicans among traditional high-turnout groups (e.g. high-education, high-income voters). But there's likely an element of truth in there. On the other hand, there is surprisingly little evidence that likely voter models actually work; indeed, Gallup, the pollster with the most (in)famous likely voter model, ranks toward the very bottom of our pollster rankings. On balance, I would tend not to want to apply a likely voter model this early in the election cycle, so that might be a point for Survey USA.
The other difference is that Survey USA tends to push leaners a lot harder. They quite literally make it hard to say you're undecided, since the way that their call script works, there's a pause of several seconds before the 'Press '9' for Undecided' option comes up. So they're going to pick up more soft support than other pollsters. Pushing leaners may be desirable late in an election cycle -- and may account for Survey USA's superior results in the primaries, where support is generally softer in than in general elections. But just as I don't like applying a likely voter model months ahead of an election, I also don't like pushing leaners this early -- especially if you aren't applying a likely voter model, which means that you may be getting people who are both indifferent toward the candidates and indifferent toward the act of voting in general, and forcing them into one or another candidate's column. So, point there for Rasmussen.
What's interesting is not so much in the result, but in what that result tells us about the nature of Clinton's and Obama's support. On balance, Rasmussen is likely to capture the most enthusiastic voters -- people who have turned out to vote before, and people who already established a strong candidate preference. Survey USA will cut from a wider cross-section.
The surprise might be that it's Clinton, and not Obama, for whom we see the big differences in the numbers. I say it's a surprise because the conventional wisdom seems to be that Obama is more dependent on a high turnout, particularly from among groups like blacks and youths that are thought to be unreliable voters. However, if we see the difference less as one of turnout and more as one of enthusiasm, the result becomes easier to understand. With her generally lower favorability ratings, Clinton may generate more lukewarm support; people who would probably vote for her if it came right down to it, but would do so only grudgingly. These voters would tend to get picked up by Survey USA, but not Rasmussen. It may also be the case that there there are more enthusiastic anti-Clinton voters than anti-Obama voters, in which case the same thing may be true.
What it's safe to say is that Rasmussen and Survey USA are presenting two different scenarios for how the election might go. The Rasmussen numbers are generally more strongly correlated with John Kerry's results in 2004. The devastating result for the Democrats is if you start with Kerry's numbers and subtract about 5 points across the board. If Kerry had polled 5 points lower nationally, he would have lost Wisconsin, New Hampshire, Pennsylvania, Michigan, Minnesota, and Oregon, representing 69 electoral votes in total. That would have made the result Bush 356, Kerry 182, which gets pretty close to Michael Dukakis territory. If Rasmussen is right, this sort of result might be somewhat more likely for Clinton than for Obama, who has a couple of regional dipoles of strength (Iowa/Minnesota; Washington/Oregon; Colorado/Nevada) that might hold up even if things went badly elsewhere. Contrary to the conventional wisdom, then, there may actually be more downside in a Clinton candidacy than an Obama one, as she appears to have vulnerable, lukewarm support in the event of poor GOTV operations or a badly-run campaign.
On the other hand, both Democrats have significant upside as represented by the Survey USA scenario: Clinton by sweeping the Kerry states and bringing in Southern states like Florida, Arkansas, West Virginia, and perhaps Missouri and Tennessee; Obama by doing the same but adding a number of states in the Interior West and the Prairie regions, as well as the increasingly "Northern" Virginia.
If you look at our electoral vote distribution chart, you can actually see this distinction: Obama's edge over Clinton comes more in averting downside scenarios than securing upside scenarios. According to our model, Clinton has a 17.6% chance, but Obama just an 8.5% chance, of winning fewer than 150 electoral votes. On the other hand, Clinton has a 15.5% chance of winning 350 or more electoral votes -- essentially the same as Obama's 17.0%. For the time being, however, both Democrats need to tie down their core support before we talk about 350-EV scenarios.
However, as I've alluded to a couple of times, there have come to be some systematic differences in the numbers each Democrat has received from each agency. The difference is particularly notable for Hillary Clinton.
Let's run a couple of straightforward charts. First, these are Obama's numbers against McCain in the most recent Rasmussen and Survey USA polls, in states that both pollsters have surveyed since Super Tuesday.
Obama vs McCain
| State | Rasmussen | Date | Survey USA | Date |
| AR | McCain +29 | 3/18/08 | McCain +20 | 2/27/08 |
| CA | Obama +15 | 3/13/08 | Obama +14 | 3/15/08 |
| CO | Tie | 3/17/08 | Obama +9 | 2/27/08 |
| CT | Obama +12 | 3/11/08 | Obama +21 | 2/27/08 |
| FL | McCain +4 | 3/12/08 | McCain +2 | 2/27/08 |
| GA | McCain +13 | 3/20/08 | McCain +13 | 2/27/08 |
| IA | Obama +3 | 2/18/08 | Obama +6 | 3/15/08 |
| MA | Obama +7 | 3/18/08 | Tie | 3/15/08 |
| MI | McCain +1 | 3/25/08 | Obama +1 | 2/27/08 |
| MN | Obama +4 | 3/19/08 | McCain +1 | 3/15/08 |
| MO | McCain +15 | 3/22/08 | McCain +14 | 3/15/08 |
| NC | McCain +9 | 3/20/08 | McCain +2 | 2/27/08 |
| NH | McCain +3 | 3/16/08 | Obama +2 | 2/27/08 |
| NJ | McCain +1 | 3/27/08 | Tie | 2/27/08 |
| NM | Tie | 2/23/08 | Obama +6 | 3/15/08 |
| NV | Obama +4 | 3/19/08 | Obama +5 | 2/27/08 |
| NY | Obama +13 | 3/11/08 | Obama +8 | 3/15/08 |
| OH | McCain +6 | 3/13/08 | McCain +7 | 3/15/08 |
| OR | Obama +6 | 3/26/08 | Obama +9 | 3/15/08 |
| PA | McCain +1 | 3/10/08 | McCain +5 | 2/27/08 |
| SD | McCain +10 | 3/4/08 | McCain +4 | 2/27/08 |
| VA | McCain +11 | 3/26/08 | Obama +1 | 3/15/08 |
| WA | Obama +5 | 3/27/08 | Obama +11 | 3/15/08 |
| WI | McCain +2 | 3/26/08 | Obama +4 | 3/15/08 |
| AVG | McCain +1.5 | Obama +1.2 |
On average, Obama leads McCain by 1.2 points in the Survey USA polls, but trails him by 1.5 points in the Rasmussen polls -- a difference of 2.7 points. Although I think that difference would hold up as statistically significant if we ran a t-test, it's not a huge gap, especially considering that the Rasmussen surveys are on balance slightly more recent, and therefore more likely to have been impacted by the Jeremiah Wright fallout and so forth. (If we look only at those states that both pollsters have surveyed at some point in March, the gap between the two agencies falls to only about 1 point).
It's in Hillary Clinton's numbers where we see the bigger differences:
Clinton vs McCain
| State | Rasmussen | Date | Survey USA | Date |
| AR | McCain +7 | 3/18/08 | Clinton +11 | 2/27/08 |
| CA | Clinton +7 | 3/13/08 | Clinton +18 | 3/15/08 |
| CO | McCain +14 | 3/17/08 | McCain +6 | 2/27/08 |
| CT | Clinton +3 | 3/11/08 | Clinton +10 | 2/27/08 |
| FL | McCain +7 | 3/12/08 | Clinton +9 | 2/27/08 |
| GA | McCain +20 | 3/20/08 | McCain +21 | 2/27/08 |
| IA | McCain +10 | 2/18/08 | McCain +4 | 3/15/08 |
| MA | Clinton +19 | 3/18/08 | Clinton +13 | 3/15/08 |
| MI | McCain +3 | 3/25/08 | Tie | 2/27/08 |
| MN | McCain +1 | 3/19/08 | Clinton +3 | 3/15/08 |
| MO | McCain +9 | 3/22/08 | McCain +2 | 3/15/08 |
| NC | McCain +16 | 3/20/08 | McCain +8 | 2/27/08 |
| NH | McCain +6 | 3/16/08 | McCain +8 | 2/27/08 |
| NJ | McCain +3 | 3/27/08 | Clinton +5 | 2/27/08 |
| NM | McCain +12 | 2/23/08 | Clinton +6 | 3/15/08 |
| NV | Clinton +1 | 3/19/08 | McCain +8 | 2/27/08 |
| NY | Clinton +12 | 3/11/08 | Clinton +13 | 3/15/08 |
| OH | McCain +6 | 3/13/08 | Clinton +6 | 3/15/08 |
| OR | McCain +6 | 3/26/08 | Clinton +6 | 3/15/08 |
| PA | McCain +2 | 3/10/08 | Clinton +1 | 2/27/08 |
| SD | McCain +12 | 3/4/08 | McCain +12 | 2/27/08 |
| VA | McCain +22 | 3/26/08 | Tie | 3/15/08 |
| WA | McCain +3 | 3/27/08 | Clinton +5 | 3/15/08 |
| WI | McCain +11 | 3/26/08 | Clinton +1 | 3/15/08 |
| AVG | McCain +5.3 | Clinton +1.6 |
Clinton's Survey USA numbers have been about 7 points better on average than her Rasmussen numbers. Look at how devastating the Rasmussen electoral map would be to Hillary: she's losing Pennsylvania, Michigan, Florida and Ohio; she's losing New Mexico, Iowa, Virginia and Wisconsin by double digits. She's losing "core" Clinton states like Arkansas and New Jersey. By contrast, Survey USA has her tied or ahead in almost all of those states (Iowa and Colorado are the exceptions).
So, bellows the choir, who's right? I would caution against any categorical answer to this question (e.g. "Rasmussen is a Republican hack pollster" or "Survey USA's results are really goofy"). Both of these pollsters know what they're doing. Survey USA's polls were devastatingly accurate in this year's primaries, which is why they sit atop our pollster rankings. But primaries and general elections are completely different beasts, and Rasmussen performed better than Survey USA in the 2004 general. There are, however, two important differences between the two pollsters of which I'm aware.
Firstly, as has been helpfully pointed out by a couple of commenters, Rasmussen is already applying a likely voter model, whereas Survey USA includes all registered voters. I'm an agnostic on the issue likely voter models. On the one hand, you sometimes hear it said that "if everybody turned out, the Democrats would win every election in a landslide". I have no idea whether that's actually true or not, especially since Democrats have started to outperform Republicans among traditional high-turnout groups (e.g. high-education, high-income voters). But there's likely an element of truth in there. On the other hand, there is surprisingly little evidence that likely voter models actually work; indeed, Gallup, the pollster with the most (in)famous likely voter model, ranks toward the very bottom of our pollster rankings. On balance, I would tend not to want to apply a likely voter model this early in the election cycle, so that might be a point for Survey USA.
The other difference is that Survey USA tends to push leaners a lot harder. They quite literally make it hard to say you're undecided, since the way that their call script works, there's a pause of several seconds before the 'Press '9' for Undecided' option comes up. So they're going to pick up more soft support than other pollsters. Pushing leaners may be desirable late in an election cycle -- and may account for Survey USA's superior results in the primaries, where support is generally softer in than in general elections. But just as I don't like applying a likely voter model months ahead of an election, I also don't like pushing leaners this early -- especially if you aren't applying a likely voter model, which means that you may be getting people who are both indifferent toward the candidates and indifferent toward the act of voting in general, and forcing them into one or another candidate's column. So, point there for Rasmussen.
What's interesting is not so much in the result, but in what that result tells us about the nature of Clinton's and Obama's support. On balance, Rasmussen is likely to capture the most enthusiastic voters -- people who have turned out to vote before, and people who already established a strong candidate preference. Survey USA will cut from a wider cross-section.
The surprise might be that it's Clinton, and not Obama, for whom we see the big differences in the numbers. I say it's a surprise because the conventional wisdom seems to be that Obama is more dependent on a high turnout, particularly from among groups like blacks and youths that are thought to be unreliable voters. However, if we see the difference less as one of turnout and more as one of enthusiasm, the result becomes easier to understand. With her generally lower favorability ratings, Clinton may generate more lukewarm support; people who would probably vote for her if it came right down to it, but would do so only grudgingly. These voters would tend to get picked up by Survey USA, but not Rasmussen. It may also be the case that there there are more enthusiastic anti-Clinton voters than anti-Obama voters, in which case the same thing may be true.
What it's safe to say is that Rasmussen and Survey USA are presenting two different scenarios for how the election might go. The Rasmussen numbers are generally more strongly correlated with John Kerry's results in 2004. The devastating result for the Democrats is if you start with Kerry's numbers and subtract about 5 points across the board. If Kerry had polled 5 points lower nationally, he would have lost Wisconsin, New Hampshire, Pennsylvania, Michigan, Minnesota, and Oregon, representing 69 electoral votes in total. That would have made the result Bush 356, Kerry 182, which gets pretty close to Michael Dukakis territory. If Rasmussen is right, this sort of result might be somewhat more likely for Clinton than for Obama, who has a couple of regional dipoles of strength (Iowa/Minnesota; Washington/Oregon; Colorado/Nevada) that might hold up even if things went badly elsewhere. Contrary to the conventional wisdom, then, there may actually be more downside in a Clinton candidacy than an Obama one, as she appears to have vulnerable, lukewarm support in the event of poor GOTV operations or a badly-run campaign.
On the other hand, both Democrats have significant upside as represented by the Survey USA scenario: Clinton by sweeping the Kerry states and bringing in Southern states like Florida, Arkansas, West Virginia, and perhaps Missouri and Tennessee; Obama by doing the same but adding a number of states in the Interior West and the Prairie regions, as well as the increasingly "Northern" Virginia.
If you look at our electoral vote distribution chart, you can actually see this distinction: Obama's edge over Clinton comes more in averting downside scenarios than securing upside scenarios. According to our model, Clinton has a 17.6% chance, but Obama just an 8.5% chance, of winning fewer than 150 electoral votes. On the other hand, Clinton has a 15.5% chance of winning 350 or more electoral votes -- essentially the same as Obama's 17.0%. For the time being, however, both Democrats need to tie down their core support before we talk about 350-EV scenarios.
Subscribe to:
Posts (Atom)
