Party Identification in Ohio

There's been some discussion in the daily polling thread about the party ID metrics in SurveyUSA's new poll of Ohio. That poll showed Barack Obama leading John McCain by 8 points, but had a party identification breakdown of 52/28/18 (Democrat/Republican/Independent). Is such a result plausible?

Let's clarify a couple of things. Firstly, we should not refer to how SurveyUSA "weighted" the poll unless we know that they actually weighted it. According to SurveyUSA's statement of methodology, they weight their polls by a variety of demographic factors but not by party ID:

Where necessary, responses were weighted according to age, gender, ethnic origin, geographical area and number of adults and number of voice telephone lines in the household, so that the sample would reflect the actual demographic proportions in the population, using most recent U.S.Census estimates.

Why wouldn't a pollster weight their poll by partisan identification? There is a very long and very thorough discussion of this subject over at Mark Blumenthal's old site, and I would encourage you to consume that thoroughly.

But one fundamental issue is that unlike demographic factors, where one can cross-check the data against relatively hard-and-fast numbers from the Census Bureau, party ID is a nebulous concept. Broadly speaking, it can mean one of two things:

1. Which party you are actually registered with.
2. Which party you tend to identify with.

These are quite different concepts, and can produce quite different results. Unless a pollster uses a list-based sample obtained from a governmental agency (this happens very rarely in public polls) they cannot be absolutely certain about which party a voter is registered with. Moreover, some states like Illinois have nonpartisan registration, so there is no such thing as a "registered Democrat" or "registered Republican" in these states. Even in states (like Ohio) that do have partisan registration, asking the voter to provide that information may not produce a completely reliable result. The voter might not remember his registration properly, or might tend to identify with the party they intend to vote for in the upcoming election rather than the one they are registered with presently. In a primary election, moreover, the voter might intend to change their registration before election day or even at the polling place.

Even more importantly, in some states like Ohio, a voter's registration may automatically be changed once they vote in a party primary. So what is God's punishment for "Operation Chaos" voters who voted in the Democratic primary to screw with the Democrats? Well, for the time being, they technically speaking are Democrats. I am partial to using the term "vampire Democrats" to refer to these voters, but your mileage may vary.

And so, to get around these problems, many pollsters may instead prefer to ask voters which party they identify with. But this too can produce different responses depending on how the question is phrased. "Which party do you tend to vote for most of the time?" is a different question, for instance, than "which party do you generally tend to identify with?". For example, an Ohio voter who had voted for George W. Bush in 2004 and Mike DeWine in 2006 but had since become disenchanted with the Republicans might answer "Republican" to the former question but "Democrat" to the latter. Finally, as Blumenthal notes, a voter's response to the party ID question may be influenced by previous questions in the survey. If the voter has told you they intend to vote for Obama, and then you ask them about their party identification, they are probably more likely to identify as Democrat (or independent) then they might have been at the top of the survey. So the tail may somewhat wag the dog.

True, if you go by the numbers, you will find a more Democratic-leaning sample in this survey than you would in other surveys of Ohio. In the 2006 Senate race, for example, the Ohio electorate was identified in exit polling as 40/37/23 (D/R/I). Or, if you combine the exit polling results from the Republican and Democratic primaries in March, the party ID breakdown in the exit polling extrapolates to 48/32/20.

But you cannot and should not go strictly by the numbers when evaluating party ID unless you know that the question is framed exactly the same way between two different polls. You are too liable to wind up with an apples-to-oranges comparison; SurveyUSA's party identification is undoubtedly at least a little bit different than Edison/Mitofsky's was in their exit polling.

In short, there is no a priori reason to disregard this poll, or to place some kind of an asterisk by it. It is certainly possible, and perhaps even somewhat likely, that the party identification in this survey just so happened to lean more Democratic than the true nature of the Ohio electorate. But the best solution to that is to combine the numbers from several different polls, rather than to try and brand one or another of the polls as an "outlier". Indeed, even if we knew that this poll included more Democrats than the likely composition of the Ohio electorate in November, that would not be an indictment of SurveyUSA's methodology, so long as such a result emerged by random chance.