Bad Math and the Bradley Effect

Theorem: The amount of time conservatives spend talking about the Bradley Effect is inversely proportional to the fortunes of their candidate.

Sean Oxendine at The Next Right purports to find evidence of a Bradley Effect in the Democratic primaries, something which I also looked for and did not find. The difference between my study and his is that I include all the states, whereas he excludes those which do not fit his argument.

Oxendine initially posts data from a large group of states, but then excludes those from what he calls the "Old Confederacy". The concept, however, is inconsistently applied. Texas (where Obama underperformed slightly) remains in his dataset. But also, the particular geographics of the Confederacy are not especially relevant electorally. Kentucky (where Obama underperformed) does not meet Oxendine's definition whereas Tennessee (where Obama overperformed) does, although the states are two peas in a pod demographically. Oxendine also excludes Iowa, where Obama significantly overperformed. True, Iowa was a caucus, and there is a reasonable argument for exlcuding it (likewise with Nevada, which he also excludes), but if you're trying to hypothesis-test, you ought to go with the more roubst and inclusive standard if you're hoping to affirm a positive finding.

In the 20 states that he does choose to include, Oxendine reports that Obama underperformed his polling margin by 2 points. This, by the way, is not a statistically signficant figure at either the 90th or the 95th percentile thresholds. Also, I actually get a different result when looking at that same set of states ... using the Pollster.com estimates rather than the RCP averages, as I did for my study, I found that Obama underperformed by 0.2 points rather than 2.0. Whether the Pollster.com or the RCP averages are superior is something we can take up at another time, but Oxendine's is not a very robust fidning if simply switching up the averaging mechanism that we use removes the positive finding entirely.

The other, more important question is why we should simply dismiss the results in the South, where Obama significantly overperformed his numbers, by 7.2 points on average, according to my definition of the region and by 9.9 points according to his -- numbers of a far greater magnitude than the Bradley Effect that he purports to find. Suppose that we conclude from this dataset that there was some sort of Bradley Effect outside of the South. We would also have to conclude, that within the South, there may be some sort of reverse Bradley Effect, perhaps resulting from black voters telling interviewers they are undecided when they really aren't. If Obama underperforms by, say, a point in Ohio, Michigan and Pennsylvania, but overperforms by 2-3 points in Virginia, Florida and North Carolina, it's not clear that this is a harmful trade for him.

So to summarize, Oxendine:

1. Cherry-picks states for his analysis;
2. Touts a finding that is not remotely statistically significant anyway;
3. Touts a finding that would entirely disappear if you used a different poll averaging mechanism, and,
4. Ignores, even if you excuse all of the above and take his claims at face value, the presence of an apparent reverse Bradley Effect that would benefit Obama in three highly electorally significant states.