Today's Polls, 10/30

Man, I thought I trained you guys better than this.

There is a lot of consternation in my inbox about two polls. One, from Mason-Dixon, shows John McCain just 4 points down in Pennsylvania. The other, from FOX News, shows McCain down just 3 points nationwide.

Let's start with the Pennsylvania result. Mason-Dixon is a pretty strong pollster. So, however, are many others from among the literally dozen or so agencies that have conducted polling within Pennsylvania over the past 72 hours. And none of those other pollsters shows the race that tight.

Mason-Dixon has also had a Republican "lean" this cycle of perhaps 2-3 points. They are quite frequently the most favorable number for John McCain in any given state. That doesn't mean that they are "biased", and it doesn't mean that they are wrong – there are many different (and legitimate!) ways to think about this election. But it does mean that their polls need to be interpreted in that context. Let's say the average poll in Pennsylvania has Obama ahead by 9.5 points. Mason-Dixon will probably start out seeing a 9.5-point state at a 7-point state. If they then end up toward the McCain side of their margin of error -- and they don't use huge sample sizes – that’s how you get to Obama +4.

Now, look. I don't think we need to be in the habit of ripping a poll apart every time that we don't like the result. There is nothing inherently "wrong" with this poll. It's simply that we need to look at in concert with the rest of the evidence. In this case, we have an abundance of evidence, and it suggests on balance that Pennsylvania is neither particularly close, nor is it particularly "tightening" (Mason-Dixon's prior poll of the state, in Mid-September, had Obama up by 2).

It might also help to come at this from the other direction. Here is one poll out of many, out of one "must-win" state out of many, that shows that John McCain is sorta kinda close? This is the best news he can muster? On Monday, I laid out specific criteria for what I'd want to see in order to conclude that the race has tightened materially:

John McCain polling within 2 points in 2 or more non-partisan polls ... in at least 2 out of the 3 following states: Colorado, Virginia, Pennsylvania.

We have yet to see any such results in any of these three states.

*-*

As for the FOX poll, I'm a little bit taken aback at the number of people who assume that, just because the poll is from FOX, it must somehow have been cooked. Sixteen times out of 20, an aberrant result (and I'm not sure you can really call this "aberrant", since a couple of other pollsters show the race at about 3 points right now) is the result of statistical noise. Perhaps 3 times out of 20, it might be the result of a poor sampling procedure. And then there might be that one case in 20 where the pollster feels compelled to put his finger on the scale in some way -- but these cases are extremely rare. And there's no particular reason to accuse FOX News of this behavior. Their polls haven't had much of a partisan lean this cycle, and for that matter, they were among the only pollsters to have John Kerry winning the popular vote in 2004. If there's a problem with FOX News polls, it's not that they're biased, but that they're simply not all that good.

It's true that FOX's sample included a materially higher percentage of Republicans this time around. FOX, however, does not choose its sample; its sample chooses itself. In this case, when they drew their ping-pong balls out of the jar, they came up with a slightly higher percentage of red ones. This kind of thing will happen all the time unless a pollster weights by party ID, which FOX News and many other pollsters do not. The Pew poll that came out the other day, for instance, had a big increase in the number of Democrats in its sample.

Nevertheless, the change in the partisan ID of their sample does cut against the notion that the race is tightening. What we are ultimately interested in is whether the same voters are starting to look at the race in a different way -- making up their minds, or changing their minds. In this case, however, it appears mostly that FOX was talking to different voters -- a more Republican-leaning set of voters -- rather than reaching the same sorts of voters and finding that they were thinking about the race differently. John McCain did pick up a few points among independents, but the numbers among Republicans and Democrats were essentially unchanged.

Numeros:



Boy, that's a lot of data. If there's anything that jumps out here, it's that we probably shouldn't be too quick to conclude that the race comes down to exactly Virgnia, Colorado and Pennsylvania. There are two polls out of Virginia today that show that state a little tighter than most of the other recent numbers; on the other hand, Ohio is becoming a real problem for John McCain, and perhaps North Carolina is too.

Lastly, although it's probably too early to conclude anything much about whether the Obama infomercial was successful, the Rasmussen and SurveyUSA state polling that was in the field last night seemed to contain pretty good numbers for him, slightly better than for much of the past week.