CNN Poll Suggests Trouble for Obama, But Omits Proper Context

A new CNN / Opinion Research released earlier this evening, and conducted yesterday and today in the wake of Barack Obama's announcement of Joe Biden as his running mate, shows a tied race at 47-47.

This is not good news for Obama, who had led by 7 points in a CNN poll released in late July. It would be nice to know why the decline occurred. However, the analysis of the poll omits several important pieces of context, and may come to a misleading conclusion about the reason for his decline.

CNN implies that the reason for the downtrend is a backlash among Hillary Clinton's supporters who had wanted Clinton to be the VP nominee:

In a new CNN/Opinion Research Corporation poll out Sunday night, 47 percent of those questioned are backing Obama with an equal amount supporting the Arizona senator.

“This looks like a step backward for Obama, who had a 51 to 44 percent advantage last month,” says CNN Polling Director Keating Holland.

“Even last week, just before his choice of Joe Biden as his running mate became known, most polls tended to show Obama with a single-digit advantage over McCain,” adds Holland.

So what’s the difference now?

It may be supporters of Hillary Clinton, who still would prefer the Senator from New York as the Democratic Party’s presidential nominee.

Sixty-six percent of Clinton supporters, registered Democrats who want Clinton as the nominee, are now backing Obama. That’s down from 75 percent in the end of June. Twenty-seven percent of them now say they’ll support McCain, up from 16 percent in late June.

“The number of Clinton Democrats who say they would vote for McCain has gone up 11 points since June, enough to account for most although not all of the support McCain has gained in that time,” says Holland.

There is a little bit of sleight-of-hand here. The analysis begins by comparing Obama's performance in this new poll to CNN's next-most-recent one, which had been conducted in late July. However, CNN then switches to discussing a different poll, one which was conducted in late June, and pulls several pieces of information about the preferences of Hillary Clinton supporters from that June version of its survey.

Why does this matter? The hypothesis suggested by the article is that Barack Obama's support has been impaired by the negative reactions of Hillary Clinton's supporters to his VP pick. The best way to test that would be to compare a poll conducted immediately before the VP pick to one conducted immediately after, before other events had a chance to intervene.

CNN is only in the field once a month or so, and so their most recent poll had been conducted three or four weeks ago, not quite as recent as we'd like. However, this would still be a lot better than a poll conducted seven or eight weeks ago. Why didn't CNN cite the preferences of Clinton supporters from its July poll instead of its June one?

Well, there are two possible reasons. Reason #1 is that they did not identify Clinton supporters in July, but had done so in June and then again now in August. This is entirely possible; most pollsters rotate different sorts of questions into and out of their polls in different months.

But we have no way to know, because CNN has not released any additional detail on at least its last three polls: no complete set of topline results, and certainly no detailed cross-tabular information. The only information we get is the information that their analysts decide to make available to us.

Essentially every other reputable polling organization, including Gallup, CBS/NYT, NBC/Wall Street Journal, ABC/Washington Post, Fox News/Opinion Dyanamics, LA Times/Bloomberg, Newsweek/Princeton Associates, Economist/YouGov, Rasmussen, SurveyUSA, Pew, Cook/RT Strategies, Public Policy Polling, Mason-Dixon, Quinnipiac, Tarrance/Battleground, IBD/TIPP, Hotline/FD and Democracy Corps, routinely makes this kind of information available. A handful of others are less consistent about it, however, they tend to strike a far less editorial tone in the presentation of their results than does CNN.

The other possibility, of course, is that CNN did identify Clinton supporters in its July poll, but chose not to cite those results because they didn't fit with its storyline. The number of Clinton-supporting Democrats will be fairly small in any given survey (probably about one-sixth of the total sample), and results from that subgroup will therefore shift around a lot, with or without reason.

All a poll really is is a series of statistics, and all statistics really are are facts, expressed numerically. As such, they deserve the same respect as any other series of facts reported in any other journalistic context. Too often pollsters think that they are making news by a conducting a poll, rather than simply reporting it.