[UPDATED] State Pegs Coleman Lead at 172 Votes; Challenges Increase for Fourth Straight Day

UPDATED at 9:50 PM with technical mumbo-jumbo.

The nightly, 8 PM update from the Minnesota Secretary of State now shows Norm Coleman with a nominal lead of 172 votes over Al Franken. According to the state's accounting, Franken has gained 43 votes on Coleman since the recount process began. However, the high number of challenges on both sides probably obscures any true movement in the vote counts.

Indeed, whereas as of midday, it appeared that the Coleman campaign was challenging significantly more ballots than Franken, Franken has fully caught up with Coleman's rate of challenges in the 8 PM count, with both campaigns challenging record-setting numbers of ballots. The Franken campaign's challenge rate in ballots counted today was 18.3 per 10,000 votes; Coleman's was 18.2 per 10,000 votes. The rates of challenged ballots have been increasing by an aggregate of about 50 percent per day over each day of the recount process.




TECHNICAL UPDATE: Different versions of the projection technique that we unveiled yesterday continue to peg Franken as a slight favorite. I am going to list four different versions here -- this will get technical:

'Gross' models evaluate each candidate's results individually, e.g. how much Franken gains in the absolute count. 'Net' models evaluate how much Franken gains relative to Coleman, without worrying about the absolute count. 'Simple' models include a maximum of three variables (plus a constant term): Franken's share of the two-way vote in that precinct, the frequency of challenges initiated by Coleman, and the frequency of challenges initiated by Franken. 'Complex' models account for two-way and three-way interactions (where statistically significant) between these independent variables. The regression is weighted based on the square root of the number of votes tabulated in that precinct. Variables are dropped if not statistically significant at the 95 percent certianty level.

Type    Depth      Franken  Coleman  Net           Result
Gross   Simple     +473     +138     Franken +335  Franken +120
Gross   Complex    +368     +17      Franken +351  Franken +136
Net     Simple     --        --      Franken +336  Franken +121
Net     Complex    --        --      Franken +263  Franken +48

The various versions of the model project a Franken win by between 48 and 136 votes once all ballots are re-counted and all challenges are resolved. However -- disclaimer! -- the margins of error on these regressions are HIGH, e.g. at least +/- 200 votes to achieve a 95 percent certainty level. The point is not really to project a precise margin of victory (or defeat) for Franken but to suggest Franken probably has made more progress in the recount than is implied by the state's in-progress totals and may in fact be the favorite to win it.

...and actually, there's a (relatively) simpler way to get at the same result. Taking a simple regression of Franken's challenge rate on Coleman's net total and vice versa, the models suggest that Franken is losing about .87 votes for every Coleman challenge, whereas Coleman is losing .74 votes for every Franken challenge. That would imply that about 87% of Coleman's challenges are to Franken votes that were initially ruled legal, and that 74% of Franken's challenges are to Coleman votes that were initially ruled legal. Challenges of this type result in a temporary deduction to the opposing candidate's total until the challenges are resolved.

Given each candidate's present number of challenges, this would suggest that Coleman has succeeded in (temporarily) deducting 1,211 from Franken's total by challenging legal ballots, whereas Franken has succeeded in deducting 1,033 votes from Coleman's total -- a gap of 178 votes. If all such challenges are rejected, Franken would presently have a lead of 6 ballots, and presumably would be on pace to make up additional ground as the final 26 percent of ballots are counted.

The key intuition is that, although the candidates have issued almost exactly the same number of challenges, Coleman is almost certainly making more of the type of challenge -- challenging ballots initially ruled to be legal -- that result in a deduction to the opponent's total. Conversely, Franken is probably appealing the local election official's decision more frequently on ballots ruled to be undervotes -- but this type of challenge has no immediate impact on the state's reported totals. This has nothing to do with which campaign's challenges are more frivolous or not -- it's just that much of Coleman's apparent lead is an artifact of the way that the state keeps score of different types of challenges.

EDIT: Well, let me qualify the above finding some more. Whether or not you arrive at the result I describe above -- that a higher percentage of Coleman's challenges are to legal Franken ballots -- depends heavily on whether and how you're weighting the regression to account for precinct size. I suspect this is because both campaigns' standards for challenging ballots have changed (i.e. loosened) with each subsequent day of the recount, and also with each subsequent day of the recount, we have tended to move into larger precincts. So, sprinkle in additional 'probablys' and 'maybes' as needed. This is confusing stuff.

EDIT #2: Another complication is that it appears as though types of challenges in a given precinct are also a function of the favorability of that precinct for Al Franken. In strong Coleman precincts, almost all of the challenges appear to involve legal ballots that the campaigns are trying to get thrown out. In strong Franken precincts, on the other hand, a more sizable percentage of the action appears to involve undervotes that were not counted initially.