The Somewhat Super Tuesday: Previews of the California, Iowa, and South Carolina Primaries and the Arkansas Senate Runoff

Tom Schaller's already done a preview of four (MT, NV, ND and SD) of the twelve states having elections tomorrow, and this post will cover four more: primaries in CA, IA and SC, and the Democratic U.S. Senate runoff in AR.

The first three states originally featured high-profile statewide barnburners that have cooled off a bit as candidates have built prohibitive leads, but as the carnival barkers say just before closing time, there's still plenty to see.

In California, the authoritative Field Poll (tops, BTW, in Nate's new pollster ratings) confirmed what observers already knew: Meg Whitman and Carly Fiorina have blown open once-tense Republican gubernatorial and Senate primaries.

Whitman's up 51-25 over Steve Poizner, after allowing him to get uncomfortably close in a couple of May polls. As you may have heard, this primary has featured unprecedented levels of spending (a total of $110 million), with Whitman spending over $70 million of her own money and Poizner chipping in $24 million. It appears that Whitman's powerfully redundant TV ads attacking Poizner as "just another liberal Sacramento politician" have trumped Poizner's attacks on her for unsavory associations with Goldman Sachs and for her failure to support Arizona's immigration law (his almost exclusive message down the stretch). Counter-intuitively, given Poizner's "true conservative" persona, Whitman's lead (according to Field) is above-average among those who "identify a lot" with the Tea Party Movement and among "strongly conservative" voters.

Poizner's challege, however, did force Whitman to abandon the technocratic, centrist tone of her early positive ads, and she's lost some serious ground to Democrat Jerry Brown in recent general election polls, particularly among independents and Latinos.

While Fiorina hasn't spent anything like Whitman's vast amounts of lucre, she did put $5.5 million of her own money into a late ad blitz, just as early front-runner Tom Campbell ran into money problems. She also seems to have benefitted from a shift among conservative voters reacting to independent advertising against Campbell, and abandoning Tea Partyish Chuck DeVore as nonviable. Field currently has Fiorina up over Campbell 37-22 (with Devore at 19), but her lead among early voters (over half the total) grows to 20%.

Like Whitman, Fiorina has suffered an erosion in general election standing during the primary battle; pollster.com's latest average has her trailing Barbara Boxer 46-39. Also like Whitman, Fiorina has baggage from her corporate career. And unlike Whitman, she's saddled with very conservative positions on abortion and on the Arizona immigration law--perilous in a state where Republicans need a decent Latino vote to win--which she endorsed (like the other two major GOP Senate candidates).

There are a host of competitive down-ballot races in California, including a Democratic LG contest featuring San Francisco mayor Gavin Newsom and LA city councilwoman Janice Hahn, a member of a powerful LA political family; and a Republican LG race between appointed incumbent Abel Maldonado--who supplied a key vote for a Schwarzenegger-backed budget deal that enraged conservatives--and conservative stalwart Sam Aanestad. Notable among several competitive congressional primaries are a battle to succeed retiring GOP Rep. George Radanovich featuring ex-congressman Richard Pombo, conservative former Fresno mayor Jim Patterson, and the incumbent's favorite, Jeff Denham; and another challenge to Blue Dog Democratic Rep. Jane Harman by 2006 candidate Marcy Winograd, in which Harman is the consensus favorite.

And this year there is just one California ballot initiative with national implications: Prop 14, which would create a Louisiana-style "jungle primary" system, essentially abolishing party primaries. The initiative campaign, which has commanded sizable pluralities in scattered polling, is being financed in part by Arnold Schwarzenegger's PAC, with support from the Chamber of Commerce and other corporate figures; the underfunded opposition mainly comes from Democratic and union groups, along with supporters of minor parties who would lose their general election ballot lines.

California turnout is expected to be anemic, with about half the total vote being cast early by mail.

Iowa's primaries frequently have future presidential implications, and that's been the case in the state's marquee 2010 primary, the Republican contest to oppose vulnerable Democratic incumbent governor Chet Culver. The frontrunner from the get-go has been former four-term Gov. Terry Branstad, an establishment figure with close ties to Mitt Romney. But in a state where social conservatives have been radicalized by a court decision legalizing same-sex marriages, challenger Bob Vander Plaats, who chaired Mike Huckabee's successful 2008 caucus bid, was thought to have a decent chance at an upset, particularly given longstanding unhappiness with Branstand in the Christian Right. A late development was the surprise endorsement (quite possibly unsolicited) of Branstad by Sarah Palin, which seemed to take the wind out of the sails of Vander Plaats' campaign. And over the weekend, the influential Des Moines Register poll showed Branstad holding a 2-1 lead over Vander Plaats.

In the Democratic Senate primary to choose a candidate for an uphill slog against Sen. Chuck Grassley, Iowa Democratic veteran Roxanne Conlin is expected to cruise to an easy win over two opponents.

The most interesting House primary is in central Iowa's 3d District, where Republicans have long targeted veteran Democratic Rep. Leonard Boswell. The national campaign committee is backing Jim Gibbons, a former wrestling coach at Iowa State (high school and college wrestling is a very big deal in Iowa) who is being challenged by a prominent state senator, Brad Zaun, and a tea party activist, Dave Funk. With four more candidates in the field, this primary could trigger Iowa's unusual provision allowing a party convention to choose the nominee if no primary candidate wins more than 35% of the vote.

South Carolina's Republican gubernatorial primary has been dominated lately by a nasty and confusing series of events in which state legislator Nikki Haley, a Mark Sanford protege and a favorite of conservative bloggers around the country, has been accused by two local political consultants (one associated with rival candidate Lt. Gov. Andre Bauer, the other being a former Sanford and Haley staffer) of having brief extramarital affairs with them. Absent any real evidence of these affairs, the allegations have convinced Haley backers (prominently including Sarah Palin and CNN/RedState pundit Erick Erickson) that it's all a cooked-up effort to beat Haley by a "good ol' boys" network of SC pols.

If so, it's not working so far, with the latest PPP poll showing Haley leading the four-major-candidate field with 43%, within possible striking distance of the 50% that would win the nomination without a runoff. The poll shows that a majority of likely primary voters don't believe the sexual allegations against Haley. But--perhaps reflecting Haley's own vow to fold her campaign or even resign as governor if the allegations prove to be true--42% of respondents think she should drop out if she did have an extramarital affair. With one of her accusers, blogger/consultant Will Folks, constantly hinting he will eventually produce definitive evidence of an affair, this possibility is, fairly or not, lurking in the background of the contest. Most recently, the Haley saga has morphed from sex to ethnicity, with Haley's own state senator, a Bauer ally named Jake Knotts, referring to her (and to President Obama) as a "raghead." (Haley is a second-generation Indian-American who converted a number of years ago from her Sikh heritage to evangelical Christianity).

The Haley saga has soaked up much of the oxygen in the race, largely neutering the financial advantage of her three rivals. PPP shows congressman Gresham Barrett, who has a strong regional base in the Upcountry area of northwestern SC, running second to Haley with 23% of the vote; Barrett's main problem has been a vote for TARP. Early frontrunner Henry McMaster, SC's Attorney General, is running third at 16%, while Bauer, whose already high unfavorable ratings have clearly been boosted by his alleged complicity in the "smears" against Haley, is running fourth at 12%.

Meanwhile, state representative Vincent Sheheen has used a big financial advantage and an upbeat campaign to overtake early frontrunner Jim Rex (the state school superintendent) for the Democratic gubernatorial nomination, which may have more value than originally thought given the chaos and nastiness of the GOP race. A third candidate, however, state senator Robert Ford, could definitely force a runoff, in part because he's the only African-American in a primary where African-Americans could represent close to half the vote.

The downballot race in SC that's drawing the most attention is a Tea Party-fed challenge to incumbent GOP Rep. Bob Inglis (another conservative congressman who voted for TARP) by Spartanburg County prosecutor Trey Gowdy. At a minimum, it looks like Gowdy will knock Ingles into a runoff.

Finally, Arkansas' runoff election between Sen. Blanche Lincoln and Lt. Gov Bill Halter is tomorrow, and while precedent and general atmospherics favor the challenger, it will all come down to a turnout battle. The one independent public poll, by R2K/DKos, showed Halter up narrowly (49-45), but the poll showed Halter with a huge lead among African-Americans which doesn't seem to accord with what happened in the primary. Certainly unions have stayed the course impressively with Halter, with SEIU and a labor-backed coalition called Working America spending an estimated $1.7 million for the runoff. But Halter's prospects will probably depend on his ability to turn out the vote in southern Arkansas, where he ran up big margins against Lincoln in the primary.

There's More...

Contract Post

Two Reids, Two Dakotas and a Quiet Montana

Four states in the Midwest/West hold primaries tomorrow: Montana, Nevada and the Dakotas. Let’s take a look at some of the key races and candidates in each.

Montana. Montana is an historically progressive western state that in recent years was depicted as one of the proving grounds for a new Democratic majority. Bill Clinton actually carried it in 1992, Democrat Brian Schweitzer attracted a lot of attention with his gubernatorial victory in 2004, and Barack Obama came close to swinging it back into the blue column in 2008. (It was also one of the five states I visited and focused on in Whistling.) Montana also played host to the dramatic saga of then-Senator Conrad Burns, who eventually lost his seat in 2006 to liberal netroots darling Jon Tester.

But this year Montana is largely out of the national conversation. This is the one federal cycle every 12 years when neither of the state’s two Senate seats, nor the governor’s office, is on the ballot. As the Great Falls Tribune reports, tomorrow’s primary outcomes may have the most impact on the state legislature. The only notable federal race tomorrow is the Democratic primary to see who will face Republican Denny Rehberg, the state’s at-large congressman. But Rehberg, despite a recent boating accident, is pretty safe, and he has raised more cash than all of his potential Democratic opponents combined. For more on the crowded Democratic primary field, I recommend this guest post over at DailyKos.

Nevada. This small state may provide the biggest story lines of the 2010 cycle. If Democrat Harry Reid loses his seat, and thus his majority leader’s position, that will be the big national story of the night. Reid's saving grace may be that the Republicans seem poised to nominate surging tea partier Sharron Angle to face Reid in November--even though the same Las Vegas Review-Journal poll showing Angle in the lead also confirms that Danny Tarkanian (yes, son of former UNLV coach) would be more competitive in the general against Reid. If Angle wins tomorrow, this race becomes a big, big, big, big national battle with all sorts of implications, and will be a major focus this November--regardless of outcome.

But in the Silver State, which Republicans had dominated in recent presidential elections until Obama beat John McCain rather comfortably in 2008, Harry may not be the only Reid on the fall ballot: His son Rory is the favorite to win the Democratic nomination for governor tomorrow. Presuming he wins tomorrow, the younger Reid will face either embattled, scandal-plagued Republican incumbent Jim Gibbons or, more likely, Republican Attorney General Brian Sandoval, who in recent polls enjoys a double-digit lead over Gibbons.

And so, while the Silver State’s senate race may provide the single biggest national storyline of Election 2010 (if H.Reid loses, that is, and especially if he loses to Angle), the governor’s race is also rather compelling. (Almost...in terms of pure sizzle, a match-up between a tea party favorite and the senate majority leader is virtually impossible to beat.) Incidentally, presuming Sandoval wins—which would be the wise choice for Nevada Republicans, given that Reid polls well in a potential match-up with Gibbons—Sandoval will join Susana Martinez as the second Latino Republican in consecutive weeks to be nominated for governor by the Republicans, signaling that they recognize the power of Latino votes in the southwest.

As for a Reid-Sandoval final this November, that could be fun because father Harry’s name on the ballot could seriously complicate matters for the son, who will undoubtedly have a tough fight against the young, attractive state attorney general. Polls suggest that Sandoval should be the favorite; although Reid has outraised Sandoval overall, they’ve raised the same amount since January.

South Dakota. Popular incumbent Republican Gov. Mike Rounds is term-limited, and with at-large Democratic Congresswoman Stephanie Herseth Sandlin not running, Larry Sabato ranks this a likely hold for the GOP. Which means the key contest tomorrow is the Republican primary between Lt. Gov. Dennis Daugaard and state senator Dave Knudson. Though Daugaard is the favorite and has the stronger numbers over expected Democratic nominee Scott Heidepriem, Knudson is surging in the late stages of this primary contest; he received the endorsement of the Sioux Falls Argus Leader yesterday. If Daugaard holds off Knudson, watch for the Daugaard-Heidepriem general election to be the contest with the most name misspells this cycle.

North Dakota. If, as Nate projects (see rankings, left), North Dakota is a certain pickup for Republicans in the wake of Democrat Byron Dorgan's retirement, the real question there is not so much whether John Hoeven, the state's popular governor, will win the open Senate seat, but by how much. So, there's not much more to say about ND than, "Congrats nominee-to-be and Senator-elect Hoeven."

There's More...

Contract Post

Pollster Scorecard: SurveyUSA

Over the coming days and weeks, I'm going to be posting 'scorecards' for most of the major polling firms. These are part and parcel of our new pollster ratings and will give you some idea on how we arrive at the numbers that we do. Some polling firms excel at general elections, but struggle in primaries; for others, the opposite is true. Some polling firms have a steep difficulty curve, and might be strong in Presidential elections, but see their polling deteriorate the further downballot one gets; others thrive when polling in more challenging contexts.

Let me show you the scorecard for SurveyUSA, which has an exemplary track record, and then we'll discuss what everything means. I should warn you that, although we've tried to design these charts as ergonomically as possible, it'll be a bit of data overload for the uninitiated.



The pollster scorecard consists of six mini-charts, each one providing a series of statistics broken down by the type of election, and the electoral cycle in which the survey was conducted. There are six types of races that we track: general elections for (i) President, (ii) the U.S. Senate, (iii) the U.S. House, and for (iv) governor, (v) Presidential primaries, and (vi) Senate/gubernatorial primaries, which are grouped together and which we've begun tracking only as of this year. Our database begins in 1998; note that odd-year elections are grouped with the proceeding even-numbered year -- for instance, last year's gubernatorial elections in Virginia and New Jersey are classified as being in the 2010 cycle.

The first two charts are straightforward, and reflect a simple count of the number of polls in our database for the firm (those which were conducted within 21 days of the Election), and the percentage of the weighted sample that they constitute. Note that more recent election cycles are weighted more heavily.

The next chart is "raw error", which is simply how much the pollster erred on forecasting the margin between the two leading candidates, on average.

There are two ways to screw up your polling, however. You can be imprecise, or you can be biased. The chart to the right of the raw error table gets at the latter problem. It shows the direction in which the error occurred, on average. For instance, a bias score of "D 1.2" would indicate that the poll missed to the Democratic side by an average of 1.2 points. We should distinguish this from a house effect, which measures the partisan direction in which a poll leans relative to other polls; the bias score instead measures how the polls fared as compared to the actual outcome of the elections. Let's say that Rasmussen is right that the Democrats are going to get utterly slaughtered in November, and they continue to be much more aggressive about predicting this than other polling firms. If they turn out to be on the money, then their polling -- even though it shows a strong house effect -- will not be 'biased', as we define it here; on the contrary, the results from the other polling firms will be biased! Although house effects are probably predictive of bias, there are also cases in which the entire polling industry has missed the mark. For instance, the average of all polls in the 1998 cycle displayed a 4.6-point Republican bias, while polling in 2002 had a 2.3 point Democratic bias.

The chart in the bottom left-hand corner -- "relative error" -- is probably the most important one. It shows how a polling firm fared as compared to other firms which surveyed the same elections, or the same types of elections, as determined by the regression analysis embedded in our pollster ratings. Negative numbers (green) are good news: they indicate that the polling firm had less error than its counterparts in the races that it surveyed; positive numbers (red) are bad news. Like the pollster ratings, the relative error chart is intended to reflect true pollster skill, and to strip out errors accounting to sample variance, or temporal variance.

Finally, we have a chart called 'contribution to score', which technically speaking is the relative error chart in the bottom LH corner multiplied by the 'percent of weighted sample' chart in the top RH corner. It tells you how we arrive at a polling firm's rawscore (i.e., our estimate of its error/skill before any reversion to the mean). For instance, SurveyUSA's strong performance in the 2008 Presidential primaries contributed -0.22 points toward its strong rawscore of -0.84.

Now, to be a bit less abstract, let's notate several features of SurveyUSA's performance:

-- SurveyUSA is very consistent. In each of the six types of elections that we consider, they've done better than their counterparts. However, their skill is most concentrated in the more difficult types of races. For instance, they've been 1.2 points better than their counterparts when polling House races, and about the same margin when polling primaries. This is oftentimes the emblem of a truly skilled pollster.

-- SurveyUSA has also held up well over different political cycles. They kicked butt in 1998, an environment in which many other pollsters struggled. And they've been very good from 2004 onward. That leaves 2000 and 2002, in which their performance was only average. But no polling firm is going to nail every cycle; if it can manage to be average, rather than below average, in its off years, that is usually a good sign.

-- Over the long run, SurveyUSA has had essentially zero partisan bias. It did have a clear Republican lean in 2000, and a clear Democratic lean in 2002 -- not coincidentally, these were its relatively weaker cycles. But generally, its polls have been straight down the middle, and especially so in recent election cycles.

As I hope is obvious, SurveyUSA is a very strong polling firm; no company has done more to contradict the notion that a "robopollster" need be inferior. Although it's not my place to make any endorsements, it would certainly make the life of electoral forecasters easier if SurveyUSA were to get more business.

There's More...

Contract Post

Pollster Ratings v4.0: Results

For a complete methodological discussion, see here. I'm just going to reiterate just a few very high-level bullet points.

-- These ratings pertain to just one particular type of poll: those for which the field work was conducted within the 21 days preceding a public election, which surveyed people about their voting intention in that election, and which was released into the public domain in advance of the election.**

-- These ratings reflect polling for President (general and primary elections), U.S. Senate, U.S. House, and gubernatorial races since 1998. More recent cycles are weighted more heavily. This is a truly massive amount of data: roughly 4,700 polls.

-- The variable called rawscore is the most direct measure of a pollster's track record. However, it is much inferior to PIE -- or Pollster-Introduced Error -- for evaluating the effectiveness of different pollsters on a going-forward basis. Because polling involves a great deal of luck in the near-term, rawscores must be substantially regressed toward the mean. However, different types of pollsters are regressed to different means. In particular, pollsters that have made a commitment to transparency and disclosure have been shown to have superior results over the long-run. The way we measure this is whether the pollster was a member of either the NCPP or the AAPOR Transparency Initiative as of 6/1/10.

-- PIE is expressed as a positive number and reflects the amount of error that a pollster introduces above and beyond that which is unavoidable due to things like sampling variance. The lower a firm's PIE the better.

The list below provides ratings for all firms with a minimum of 10 polls. A complete list of ratings (including for firms with fewer than 10 polls) follows it below the fold.

Ratings for firms with at least 10 polls



Ratings for all firms (no minimum number of polls)



** Correction: This language previously read: "These ratings pertain to just one particular type of poll: those which attempt to forecast election outcomes, and do so in the public domain." This had been improperly stated. This exercise accounts for all polls released into the public domain, whether or not it was the pollster's intention to forecast the election outcome. Some polling organizations do not regard their polls as constituting "predictions" or "forecasts", and do not recommend that their polls be used for forecasting purposes.

There's More...

Contract Post

Pollster Ratings v4.0: Methodology

Rating pollsters is at the core of FiveThirtyEight's mission, and forms the backbone of our forecasting models. But, it has been two years since we last revised our ratings. Here, at last, is an update. We have both substantially increased the amount of data that we are evaluating, and significantly refined our methodology.

Before we proceed, an important point of context. These ratings are designed to form an objective assessment of pollster quality with respect to one particular function: their aptitude in accurately forecasting election outcomes, when they release polls into the public domain in the period immediately prior to an election. The emphasis is ours, of course. The ratings may not tell you very much about how accurate a pollster is when probing non-electoral public policy questions, in which case things like proper question wording and ordering become much more important. The ratings may not tell you very much about how accurate a pollster is far in advance an election, when definitions of things like "likely voters" are much more ambiguous. And they may not tell you very much about how accurate the pollsters are when acting as internal pollsters on behalf of campaigns, and when their work is kept private; some of the pollsters with the lowest-rated publicly-released polling are some of the first I'd hire if I were a campaign director.

Nevertheless, this is a pretty massive undertaking. Not only have I updated the database with results from the 2008 General Election (as well as elections to date in 2009 and 2010), but I've also backfilled the data going back to 1998, substantially improving coverage and quality. The database now contains 4,670 distinct polls from 264 distinct pollsters covering 869 distinct electoral contests -- about treble what it did before.

Which Polls are Included

The database covers all minimally competitive [1] general elections for the offices of President, U.S. Senate, U.S. Representative and Governor in the period from November 3, 1998 through June, 1, 2010, including run-offs, "jungle primaries", and special elections, in which at least one poll was released into the public domain in the final 21 days of the campaign. It also covers all Presidential primary elections during this period. Senate and Gubernatorial primaries are included as of 2010 (but not before); primaries for the U.S. House are not included. Although the vast majority (93 percent) of the database covers polling at the state or district level, national polls for the office of President are also included, as is generic ballot polling for the U.S. House, compared against the outcome of the national popular vote. [2]

Polls are included if their median field date was within 21 days of the election. But, as will be described below, the closer a poll is taken to the election, the higher the standards we have for it; that is to say, "newer" polls are penalized, and older poll receive a bonus. There are a handful of exceptions to the 21-day rule. For instance, no Presidential primary polls are included from states other than Iowa until the Iowa caucus has been conducted, and no polls are included for states other than New Hampshire states until the New Hampshire primary has been conducted. In addition, a more advanced cut-off date may be applied in cases where a candidate who had a tangible chance of winning the election drops out of it prematurely.

All polls from a particular pollster are included, even if they have surveyed the race multiple times within the 21-day window.[3] Polls, especially those from weaker pollsters, sometimes bounce around a lot before "magically" falling in line with the broad consensus of other pollsters. In the 2008 Democratic primary in Wisconsin, for instance, which Barack Obama won by 17 points, American Research Group had released a poll on the Saturday prior to the election showing Obama losing to Hillary Clinton by 6 points; it then released a new poll 48 hours later showing Obama beating Clinton by 10 points. (It is very unlikely that there was in fact such dramatic late movement toward Obama, as most other pollsters had shown him well ahead the whole time). In any event, our model now accounts for this sort of issue. The only exception is that in the case of tracking polls, for which results must be non-overlapping. For example, in the event of a 4-day tracking poll, we'd take the final poll that the pollster released prior to the election, but then go back 4 days to include the poll before that, and so forth.

We flag polls which meet our definition of a partisan poll, which is quite narrow and quite specific: "a poll ... conducted [on behalf of] any current candidate for office, a registered campaign committee, a Political Action Committee, or a 527 group." Nevertheless, they are included in the ratings. If a pollster releases a poll into the public domain, we assume that they are interested in doing their best and most accurate work, regardless of whom the poll is conducted for.

The quality of the data is generally fairly high. However it is compiled from eight different data sources, and some minor mistakes are unavoidable. We cross-checked all polls which appeared to show abnormally large errors to ensure that they were recorded properly in the database. For a small portion (under 10 percent) of the database, the sample size was not known and was estimated based on the margin of error listed by the pollster, or by the sample size which that pollster generally uses. For about 5 percent of the database, the field dates of the poll were unknown and were estimated instead from the release date.

It is sometimes tricky to figure out whom to attribute the poll to. Many polls are conducted on behalf of media organizations, e.g. Zogby International on behalf of the Toledo Blade. In general, we are more inclined to assign the poll to the firm which conducted the field work, rather than the sponsor. But some large media organizations conduct their own field work, or otherwise put a strong methodological imprint upon the poll. There are simply some judgment calls that have to be made, the principle being to assign the poll to the organization which is likely to have added the most value to it (or subtracted the most value from it).

There are some cases in which a media organization has switched field providers; we would generally place these polls into separate groups. For example, in 1998-2000, CNN used Yankelovich to conduct its polls; these polls are classified as "Yankelovich". In 2002 and 2004, it partnered with Gallup; these polls (along with all polls Gallup has conducted on behalf of itself or USA Today) are classified as "Gallup". Since 2006, it has partnered with Opinion Research, these polls are classified as "CNN / Opinion Research". To take another example, the Los Angeles Times recently discontinued its in-house polling operation and now sponsors polls conducted by USC; these polls will be classified as "USC", and the pollster rating historically associated with "Los Angeles Times" polls will not be applied to them.

Another tricky case is that of CBS/New York Times and ABC/Washington Post, as these organizations very often collaborate on polls but occasionally release polls with just one of the two brand names attached. We have traditionally grouped polls conducted by these organizations together and will continue to do so. In the two cases, however -- these are Harris International and Zogby -- of a firm which releases both telephone-based and Internet-based polls, they are accounted for separately.

Calculation of 'Raw' Ratings

Some types of elections are significantly easier to forecast than others. For instance, among all the polls in our database, the average miss on the outcome of the U.S. Presidential national popular vote was just just 2.8 points, which approaches the theoretical maximum that might be achieved given the limitations owing to sample variation.[4] On the other hand, polls of Presidential primaries have missed by an average of 7.5 points, and for Senate and gubernatorial primaries, by 7.8 points. Moreover, these differences can vary from year to year. For instance, in 2000, there was a fair amount of late movement toward Al Gore, whereas the polling was quite stable during the last three weeks of the 2004 and 2008 Presidential campaigns.

Average Error, All Polls in FiveThirtyEight Database (1998-Present)


The idea, therefore, is to rate a pollster's accuracy in the context of the "degree of difficulty" of the races that it is covering. A pollster deserves more credit for correctly forecasting the balloting in a difficult primary, or for a U.S. House seat in an obscure district, than it would for calling the outcome of the Presidential popular vote (and it deserves more tolerance when it is wrong).

The procedure that we use to accomplish this consists of a regression analysis which attempts to understand the error incumbent to a poll, in consideration of (i) the pollster in question; (ii) the election, or type of election; (iii) other characteristics of the poll, such as its recency and its sample size. The measure of error which we use is the so-called 'Mosteller 5' metric which evaluates the margin between the top two candidates in the race. [5] For instance, if a poll shows the Democrat beating the Republican by 4 points, and the Republican instead beats the Democrat by 2 points, that is considered a 6-point error.

In particular the regression analysis accounts for the following variables:

(a) A series of dummy variables to correspond to each pollster in the database. The coefficients associated with these variables can be thought of as reflecting the amount of error that a particular pollster adds or subtracts, all other things being equal -- that is, it reflects the pollster's skill.

(b) Variables to indicate the recency of the poll, as measured by the number of days between the poll's median field date and the election. The functional form of this variable is determined by the square root of the number of days between the poll and the election. Separate variables are used for primary and general elections, as primary polling is considerably more time-sensitive than general election polling.

(c) A variable to indicate the sample size of the poll. It is desirable to strip out the sample variation from pollster skill, as our formula for weighting polls accounts for the sample size separately. With that said, for a variety of reasons which are explained in the footnotes, this variable is no more than marginally statistically significant. [6]

(d) A series of dummy variables to indicate the type of election, and the cycle -- for instance, "Gubernatorial Elections in 2000". [7]. In the case of elections to nominate a candidate, there is a separate variable to indicate whether the election was a primary or a caucus. (Polls of caucuses, especially outside of Iowa, are associated with significantly higher errors.)

(e) A series of dummy variables to indicate the particular race under consideration (for instance -- "Ohio Senate general election, 2000"), in cases where the polling for that election is deemed to be robust. [8]

In other words, in cases where there is a significant amount of polling in an election, we compare the performance of a poll against other polls of the same election. In cases where there is not very much polling (sometimes, in fact, only one pollster will have surveyed a race) we compare its performance against polls of "similar" elections, i.e., elections for the same type of office in the same cycle. [9]

The regression analysis is weighted based on the number of surveys that the pollster conducted for that particular election, and based on how recently the election occurred; polling from the 2008 election receives about twice as much weight as the 1998 election. [10]

The goal of this regression analysis, again, is to determine the coefficients associated with each of the 264 pollsters, which forms our measure of pollster skill. These coefficients, which are re-calibrated such that the average score is zero, are what we call a pollster's rawscore. For instance, SurveyUSA's rawscore is -0.82, meaning that a SurveyUSA poll has about eight-tenths of a point less error than average. American Research Group's raw score is +0.72, meaning that it has seven-tenths of a point more error than average.

Reversion Toward the Mean

Technically speaking, our goal is not to evaluate how accurate a pollster has been in the past -- but rather, to anticipate how accurate it will be going forward (as these ratings are principally used in conjunction with our electoral forecasting). This requires us to regress the raw scores toward the mean, since there is a very significant amount of luck involved in polling over the short-run. Consider that a poll of 600 voters will miss by an average of 3.5 points on the margin between the two candidates, accounting to sample variance alone. If the very best pollsters only contribute about 1 point of skill (as seems to be the case), and the worst ones only subtract about 1 point of skill, we would to look at a large number of elections before we can separate out the skill from the luck. This is especially so because elections are not entirely independent of one another, e.g., a pollster's skill in forecasting the outcome of the 2008 Presidential election in Pennsylvania is probably correlated with its skill in forecasting the same election in Ohio. Be very suspicious of pollsters who claim to be superior on the basis of only having called one election correctly. [11]

Although our previous version of the ratings had accounted for reversion to the mean, it had done so in a somewhat ad hoc way. For this version, I analyzed the appropriate degree of mean-reversion more robustly. Specifically, I ran a version of the rawscore calculation that accounted for polling through 2006 only, and another version that accounted for polling in 2008 only, and then ran another analysis to regress the latter rawscores on the former on the former. In other words, we were trying to see how accurately we could predict a pollster's performance in 2008 based on its performance prior to 2006.

The regression analysis found that the optimal degree of mean-reversion is proportional to the square root of the number of polls in the sample. Specifically, it dictated the following formula:

reversionparameter = 1 - (0.06 * sqrt(n))

where n is the number of previous polls in the sample from that pollster. For instance, with a pollster which had conducted 25 polls:

reversionparameter = 1 - (0.06 * sqrt(25))
reversionparameter = 1 - (0.06 * 5)
reversionparameter = 1 - (.30)
reversionparameter = .70

In other words, we would regress this pollster's rawscore 70 percent of the way toward the mean, and "keep" 30 percent of it. (In plain English, at a sample size of 25 polls, we are still mostly looking at noise rather than skill. So if, say, this pollster had compiled a rawscore of -0.50, this would be regressed back to a -0.15.

The minimum amount of mean-reversion is set at 10 percent. What this implies is that after 225 polls, the amount of weight which we attach to a pollster's previous track record is essentially maxed out. In addition, for reasons that are explained in the footnotes [12], rawscores superior to -2.22 are truncated to -2.22.

But toward which mean do we regress?

Although we have reached a fairly technical place in our discussion, the following point is essential and constitutes a major change from our previous version of the pollster ratings.

Is the only thing which is useful in forecasting a pollster's future performance his prior track record (regressed to the mean as appropriate?). Or are other, more qualitative features of the pollster also worth considering?

It turned out, when I ran my analyses, that the scores of polling firms which have made a public commitment to disclosure and transparency hold up better over time. If they were strong before, they were more likely to remain strong; if they were weak before, they were more likely to improve.

The variable I used to denote disclosure/transparency is a dummy indicating whether, as of June 1, 2010 [13], the polling firm was either a member of the National Council on Public Polls, or had committed to the AAPOR Transparency Initiative [14]. (Neither NCPP nor AAPOR endorse these ratings in any implicit or explicit way, although NCPP President Evans Witt provided me with a current membership list). This ncppaapor variable was statistically significant at approximately the 95 percent significance level in predicting performance in 2008, when placed to a regression analysis along with past pollster performance; in fact, it was almost as important in predicting future performance as past performance itself.

In a separate (although related) analysis, the ncppaapor variable is statistically significant at approximately the 95 percent level in predicting current rawscores, when it is placed into a regression analysis along with variables indicating whether the polling firm was partisan [15], and whether its polling was conducted over the Internet. [16]



Therefore, the ratings for polling firms which were members of NCPP, or which had signed onto the AAPOR Transparency Initiative, as of June 1, 2010, are regressed toward a different mean than those which hadn't. Essentially, then, polling firms are rewarded for having made a public commitment to disclosure and transparency -- but the basis for rewarding them is statistical rather than ideological. Note, however, that this will mostly effect firms with relatively few polls; it will barely make a difference in the case of a firm like Rasmussen Reports or SurveyUSA which have surveyed hundreds of elections, as their ratings are subject to only a minimal degree of mean-reversion.

(Internet-based polls are also regressed toward a different mean. We do not regress the scores of campaign pollsters toward a different mean, even though they clearly perform worse, because we only include their polls in our projections when they have not been conducted on behalf of a campaign.)

For the time being, I'm going to pass on the question of whether firms which join NPRR or the AAPOR Transparency Initiative subsequent to today will receive the "bonus"; it is a sticky issue, and one which I simply haven't decided upon. Certainly no firm is going to get rewarded if it fails to transparent in actual practice as well as in theory. What we can say is that firms which have already joined on to one of these efforts tend to be more accurate than those which haven't.

The Final Step: Pollster-Introduced Error

As mentioned above, different types of polls have their rawscores regressed toward different means. Polling firms which are members of NCPP or the AAPOR Transparency Initiative have their scores regressed toward a mean of -0.50. Those which aren't, but which conduct their polls by telephone, are regressed toward a mean of +0.26. And those which conduct polling by means of the Internet are regressed toward a mean of +1.04. [17]. We can call this variable groupmean.

A pollster's adjusted score, or adjscore, is then calculated as;

adjscore = (rawscore * (1 - reversionparameter)) + (groupmean * reversionparameter)

Adjusted scores, like rawscores, may be either positive or negative, with negative scores indicating superior performance and positive scores inferior performance. Traditionally, however, we have expressed pollster skill by means of a positive variable called Pollster-Introduced Error or PIE, which is a measure of how much avoidable error which the pollster introduces. That is, it is a measure of how much error a polling firm introduces other than sampling error, and other than temporal error (i.e., the error introduced by the necessity of having to conduct polls some number of days in advance of the election). PIE is calculated simply by taking the adjscore and adding 2 points to it.[18]

PIE = adjerror + 2

The lower the PIE, the better the pollster; the minumum possible PIE is zero.

The new pollster ratings will be posted in a separate thread in approximately an hour.

Notes

[1] Very few elections fail to meet the threshold of 'minimally competitive'. The exceptions would be a primary election after all viable candidates have dropped out, or a general election in which a major-party candidate is not opposed by a candidate from the other major party, and the third party candidates are not viable. These elections are rarely polled and only about 5-10 polls were excluded for this reason.

[2] A slight adjustment is made to the national House popular vote calculation to account for the fact that some states (Arkansas, Florida and Louisiana) do not tally ballots when a candidate runs unopposed; we assume that such candidates would have received votes equal to 75 percent of the average district-level turnout in that year's Congressional elections.

[3] One of our eight data sources includes only the last poll by each pollster in advance of the election; the other seven include all polls. However, there is significant redundancy in our coverage, so only about 5 percent of the races are impacted by this.

[4] Note, however, that the pollsters may have been a bit fortunate in this respect -- particularly in 2004 and 2008, where the polling trend was very flat in the final few weeks of the campaign. There are some other relatively recent elections, like 1980 and 1996, in which some of the national polls did quite badly.

[5] That is, the candidates who actually finished in first and second place -- not necessarily the candidates who were predicted to finish in first and second place by the consensus of pollsters.

[6] The t-value on the sample size variable in our model -- which is actually expressed as the square root of the sample size -- is only 1.30, indicating that it is statistically significant only at about the 80 percent confidence level. However, this should probably not be interpreted literally. The problem is that pollsters tend to use about the same sample sizes for the same types of elections, e.g., a Research 2000 general election poll will usually consist of 600 people. This makes it difficult to segregate out the effects of sample size, since they may be captured instead in the large series of dummy variables used to denote the pollster and the election or type of election. What we'd ideally want to have is instances where, for instance, Quinnipiac released a poll of 2,500 people in advance of an election, and another poll of just 500 people later on for the same election. But such cases rarely occur in practice. With that said, it probably is also the case that sample sizes are in fact less important than they theoretically should be, because true sample sizes and margins of error are significantly impacted by things like demographic weighting and cluster sampling, and the pollsters rarely account for this when disclosing their margins of error.

[7] Odd-year elections (e.g. 2005) are grouped with the next even-numbered year (e.g. 2006). Senate and gubernatorial primaries placed into the same grouping.

[8] Polling is deemed to be 'robust' if:

(a) At least three distinct nonpartisan pollsters have surveyed the contest, and each of those pollsters have surveyed at least three different contests throughout the database, and
(b) At least two of the three pollsters make a 'short list' of prolific pollsters, which (i) have surveyed at least three distinct races in each of at least three distinct states and (ii) are not campaign pollsters; (iii) are not Internet-based pollsters; (iv) are not Strategic Vision, since Strategic Vision's polling was probably fake.

The idea here is that performance relative to other pollsters is only meaningful to the extent that we know a fair amount about those other pollsters. If we don't know very much about them, there is nothing to anchor to, and it is probably better to evaluate error relative to similar types of elections in other states or districts.

[9] Careful readers will note that the series of variables used to designate the type of election are effectively redundant in cases where the polling for a particular election is deemed to be robust [8] and is given its own variable. Effectively, then, the election-type variables reflect the error in all elections within that cycle in which polling was not robust. The reason this distinction is important is because elections with nonrobust polling tend to be associated with larger errors (across individual polls -- not just in the polling average) than elections with robust polling. This may be because the pollsters are avoiding these types of elections for a reason -- there is some evidence, for instance, that lopsided elections are actually harder to forecast than close elections. But it may also be that it helps the pollsters to see what one another are doing and to have some 'guidance' when doing things like constructing likely voter models or demographic weightings. In any event, the way that our regression is structured, it implicitly accounts for this 'Wisdom of Crowds' effect; that is, a pollster will not be punished (over the long run) for being adventurous and releasing polls in elections that few other pollsters are evaluating.

[10] The weighting parameter contains two components, which are multiplied together. The first component is designated as 1/sqrt(n), where n is the number of polls that a firm has conducted of a particular race. The second component weights the poll based on its date, where a weight of 0.5 is given to polls conducted on 11/3/1998 (Election Day 1998), and a weight of 1.0 is given to polls conducted on 11/4/2008 (Election Day 2008), and all other dates are scaled accordingly. Neither of these weightings make an especially material difference in the results.

[11] I write: "Be very suspicious of pollsters who claim to be superior on the basis of only having called one election correctly." But the converse is less true; if a pollster is really terrible, you may not need very many elections to determine this. This is because accuracy is bounded on the upside (you can't do better than calling the margin exactly right -- having no error), but effectively unbounded on the downside (a pollster could miscall the election by 30 or more points).

[12] The reason is that the theoretical minimum average error is on the order of 3.5 points, which is the average error owing to sampling variance when conducting a poll of 600 people. On the other hand, the average error for all polls in our database is about 5.5 points. Thus, the maximum possible skill, over the long run, is probably about 2 points, and skill in excess of 2 points can reasonably be assumed to be noise or luck. The reason we set the threshold at -2.22 points rather than -2.00 points is because we separately regress rawscore to the mean by a minimum of 10 percent, and so a pollster with a rawscore of -2.22 would have it truncated to -2.00 anyway at the mean-reversion step of the analysis. This sets things up in such a way that the minimum possible Pollster-Introduced Error (PIE) is 0.00.

[13] Although it is problematic to run a regression analysis on 2008 data based on a condition which was not in existence until 2010, I also ran the analysis based on old NCPP membership lists (which would have been salient as we entered the 2008 cycle), and got very similar results.

[14] Mere membership in AAPOR is not enough, since AAPOR (unlike NCPP) is an organization of individuals rather than polling firms. However, commitments to AAPOR's Transparency Initiative were made at the firm level, so it is suitable to be used in conjunction with NCPP membership.

[15] For these purposes, a firm was designated as 'partisan' if a majority of its polls included in our database were conducted on behalf of a candidate, campaign committee, PAC or 527 group.

[16] You may note that the coefficient on the Internet variable is not actually statistically significant. Indeed, assessing the accuracy of Internet polls is problematic, as Zogby's interactive polling has been abominable, whereas Harris Interactive and YouGov have had fairly decent results. (A fourth Internet-based pollster, Knowledge Networks, has only three surveys in our database). For the time being, I nevertheless think it is wise to regress Internet-based polls toward a different mean.

[17] Note that two firms -- Harris Interactive and Knowledge Network -- both meet the NCPP/AAPOR disclosure test and conduct polling by means of the Internet. They are regressed to the average of the NCPP/AAPOR and Internet means, which is +0.28.

[18] The reason why 2 points, rather than some other number, is added to adjscore to produce PIE is explained in note [12]: this is the difference between the theoretical minimum error owing to sampling variation and the average error observed among all polls in our database.

There's More...

Contract Post

A Horrible Jobs Report

From the BLS:

Total nonfarm payroll employment grew by 431,000 in May, reflecting the hiring of 411,000 temporary employees to work on Census 2010, the U.S. Bureau of Labor Statistics reported today. Private-sector employment changed little (+41,000). Manufacturing, temporary help services, and mining added jobs, while construction employment declined. The unemployment rate edged down to 9.7 percent.

This report stinks. The majority of job creation came from Census hiring -- the private sector was non-existent.

Let's look at the data:

The civilian labor force decreased by 322,000 and the number of unemployed decreased by 287,000. This lowered the unemployment rate to 9.7%.

The labor force participation rate decreased .2% because of a drop in the number in the civilian labor force. The employment to population ratio decreased .1%. This means that fewer people participated in the labor force last month and the percentage of people employed as a percentage of the civilian labor force decreased.

Total private jobs created were 41,000. Goods producing industries increased 4,000, with an increase in manufacturing and mining offset by a drop in construction. Service employment increased by 37,000. With the exception of financial services (which decreased 12,000), all sectors saw growth. Just not much growth.

Simply put, the private sector dropped out of the equation last month. Period. Unfortunately, this report comes at the worst time possible -- when the markets are already shaky. In addition, the initial unemployment claims numbers -- which have dropped the last two weeks -- are still high. This is not a good combination.

Several weeks ago, I wrote an article titled Storm Clouds on the Economic Horizon. In that article I wrote the following:

[Going forward] the economy needs to keep up its current pace of job creation. Last month we had a great employment report. That needs to be repeated in the next report.

This is the worst possible jobs report we could have in the current environment. Now -- this is just one report. The record indicates things are moving in the right direction; this could be nothing more than a speed bump. But, this is a most ill-timed speed bump.

There's More...

Contract Post

Is a Stable Government in the Netherlands Coming?

From 538's Dan Berman

Next Wednesday, June 9th, the citizens of the Netherlands will go to the polls to elect a new government for the fourth time in eight years. The elections were called nearly a year early when the government of Jan Peter Balkenende, a coalition of the centrist Christian Democrats(CDA), Center-Left Labour party(PvdA), and the Christian Union, fell over whether to continue the Dutch military’s mission in Afghanistan.

The Dutch forces in Afghanistan have seen more combat than any national contingent, other than the American and British missions, and as a consequence the deployment has become increasingly unpopular. That, however, had less to do with the fall of the government than the fact that the unpopularity of the mission was in particular hurting the standing of Labour party, which in some polls had fallen as far as fifth, suffering the same fate that befell the Social Democrats in Germany who occupied a similar position in a grand coalition.

Having his government fall from under him is not a new experience for Balkenende. Since 2002, he has headed four different governments, each of which fell when a member party left the coalition. His first, formed after the chaotic 2002 elections, fell apart due to infighting within the Fortuynist party following the assassination of its charismatic leader. His second fell in 2006 over the decision of his Immigration Minister, Rita Verdonk, to pursue an effort to strip activist and MP Ayaan Hirisi Ali of citizenship after the latter admitted to lying on her application for asylum in the course of a television interview.

In each case Balkenende was able to hang on as Prime Minister after the subsequent election, adjusting coalition partners at will. This too is not a new development in Dutch politics. The CDA, occupying a central position in the political spectrum of proportional electoral system without a threshold, has traditionally been central to governmental formation in the Netherlands. Except for a period when an unusual alliance of the Labour Party and the right-wing Liberals(VVD) kept them out of power from 1994 to 2002, the CDA has headed every government since 1918.

This time, however, it appears that Balkenende’s luck, and that of the CDA, has finally run out. Polls after the collapse of the government showed the Prime Minister’s party taking a large degree of the blame for the fall of his fourth government. Cohen’s appointment as Labour leader appears to have given a boost to the party. Polls taken after the fall of the government showed the CDA falling to high twenties in projected seat totals, substantially below its showing in 2006 of 41 seats.

The real question instead seemed to be who would benefit. Before the fall of the government the big winners in any future election looked to be anti-immigrant politician Geert Wilders, and his Party for Freedom (PVV), which had surged into the first place in polls on the strength of the governments attempt to prosecute Wilders for inciting hatred against Islam. Some polls showed it winning as many as 29 seats out of 150.

On the other side of the spectrum, the Democrats 66, a left-leaning Liberal party that had strongly opposed both Wilders strident anti-immigrant tone, and the government’s efforts to revive blasphemy laws to combat it, saw an unprecedented rise in support, with polls showing it approaching 25 seats.

Had these results been repeated in the general election, it would have been a transformational election that likely would have forced a gutted Labour and CDA into a new grand coalition. However, Wilders, who thrived in opposition, proved much less relevant to a campaign that actually required more solid economic policies than banning the Koran. And his decision to embrace left-wing populist economic policies increasingly alienated support he had gained from traditionally Liberal voters who found the VVD’s leader, Mark Rutte, who had defeated Verdonk for the leadership in 2006, boring.

The fading of Wilders, also lead to a fading of the Democrats 66, who increasingly lost a raison d'être. The beneficiaries were the parties of the coalition of the 1990s. The Labour party selected a new leader at the start of the campaign, a left-wing MP, Job Cohen, who was recently featured in a New York Times Magazine piece on the election. Cohen reinvigorated the party, raising it to first place early on in the campaign, and creating the prospect he might be the Netherlands’ first Jewish Prime Minister.

That prospect has dimmed due to the success of the other beneficiary of Wilder’s collapse. That is the VVD or Liberals, headed by Mark Rutte. Rutte had run as a moderate alternative to hard-line former Immigration Minister Rita Verdonk for the party leadership in 2006, more or less entirely on a platform of not letting Verdonk turn the party into a clone of Wilders’ movement. Having succeeded, he promptly disappeared from the scene, with his party even dropping to sixth place for a time after Verdonk had launched her own party, a personal vehicle using the minimalist title Trots op Nederland(Proud of the Netherlands). Verdonk however, proved herself to lack Wilders’ theatrical talent, and faded to a single seat, and in the course of an extended campaign in which economic competence trumped immigration, the VVD, the only party to publish detailed plans on how to balance the national budget, surged.

This can be seen in the three most recent polls, taken between the 25th of May, and the 1st of June, showed the VVD leading with between 36 and 37 seats, followed by Labour with between 28 and 32 seats. The formerly dominant CDA trails in third with between 20 and 26 seats, followed closely by Wilders’ PVV with 18-19, the Democrats 66 and Socialists with 9-12, and everyone else trailing behind.

Such a situation presents several different possibilities for government. It is worth noting that virtually every poll over the last two weeks has shown the CDA, VVD, and PVV with a majority between them. While it is almost impossible to imagine Wilders being allowed into a government, its worth noting that far-right parties such as the Danish People’s Party and the Norwegian Progress Party have supported right-leaning minority governments without officially joining them. Both parties however, despite similar platforms, possessed institutional leaderships that were far less erratic than Wilders. In such a circumstance, it is more likely to be Wilder’s unpredictability rather than his views themselves which will eliminate him as a potential partner in government

With reliance on Wilders eliminated as a possibility, a similar fate is likely to befall the CDA, or at least its central roll. If the Liberals hold nearly 40 seats, they will almost certainly drive government formation, and with Mark Rutte having moved them away from a hard-line on immigration, they will have a number of options. There is little that divides them from the Democrats 66 or even the Greens, and they may also be tempted to recreate their 1990s alliance with Labour, only this time with roles reversed in terms of the leading party.

In any event however, the recovery of Labour and the Liberals in next weeks elections, symbolizing the revival of the mainstream left and right after several electoral cycles in which extremists seemed dominant, looks like likely to give the Netherlands its first stable government in nearly a decade. This is important given the awe-inspiring economic challenges that will face any new government in the next few years. But it does create hope that the extremist tide on both sides of spectrum is reaching its crest in Europe.

---
This article was authored by research assistant Dan Berman. Please send comments or suggestions to sexton538@gmail.com

There's More...

Contract Post

A Note On Early Voting and June 8

No matter how much we all remind ourselves that early voting has revolutionized politics in some parts of the country, it's easy to get caught up in the traditional patterns of pre-election activity, climaxing on Election Day.

But as we head into the busiest primary day of this cycle, June 8, it's important to remember that in some states Election Day has already begun to unfold--notably in California and Nevada.

Early voting--in-person or by mail, "no-excuse" or requiring an affadavit of inability to vote on Election Day--is entirely within the purview of the states, and the variations in law and practice are significant. On one end of the spectrum you find Alabama, which has no in-person early voting and requires that absentee ballots be accompanied by an affadavit and the signature of a notary or two witnesses. In 2008, only 4% of Alabama ballots were cast this way, which is why you heard nothing about early voting leading into the June 1 primary. On the other extreme is Oregon with its 100% mail-in ballot system.

The trend in recent years has been away from the Alabama model and towards Oregon's, though some states prefer early in-person voting to any sort of mail-in system.

One June 8 primary state, California, has adopted a system that allows individual voters to permanently register for an Oregon-style mail ballot, which is then automatically sent out within a few weeks of the election. As of 2009, 5.5 million--or about a third--of the state's 17 million registered voters were permanently signed up for voting by mail. And in 2008, a total of 5.9 million votes were cast by mail (including both permanent registrants and those requesting mail ballots for the individual election), or 41% of all votes. Since by-mail voters seem to be more likely to vote than those using the traditional system, the percentage could go even higher on June 8. Since the only requirement for mail voting is receipt by Election Day, it's difficult to estimate when ballots are actually cast, though there is some evidence this year that many voters are holding their ballots until the last minute.

Another June 8 primary state, Nevada, has gone heavily towards in-person early voting, with 58% of 2008 presidential ballots being cast in that manner. This year the "window" for early in-person voting in the primary ran from May 22 to today, June 4. And given estimates that early voting is heavier than in the 2008 primaries, it's entirely possible that over half the ultimate vote has already been cast in the competitive GOP gubernatorial and Senate primaries.

A third June 8 primary state, Iowa, illustrates a third variation on early voting by offering broad opportunities for in-person absentee voting beginning 40 days before every election. 36% of the 2008 general election votes were cast via absentee ballots in person or by mail.

Still another state, South Carolina, allows by-mail or in-person absentee voting, though not without an "excuse," but in practice locations for this form of early voting are often limited. (This is also the situation in Virginia, which has a couple of hot congressional primaries next Tuesday). 17% of SC's 2008 presidential votes were cast by absentee ballot. The state legislature is currently deadlocked over legislation that would match Democratic-supported efforts to expand early voting with Republican demands for photo ID. But by and large, most Palmetto State voters will be exposed to the full insanity of this primary season before making up their minds.

In every state, campaigns know the rules and how to exploit them. But the important thing to keep in mind on June 8 is that the results are not always a snapshot of Election Day public opinion, but are increasingly a compilation of decisions made over an extended period of time.

There's More...

Contract Post

Ranking States by the Liberalism/Conservatism of Their Voters

The scheduled and recent primaries in South Carolina and elsewhere remind me that a couple of years ago David Park and I made a series of graphs that revealed some interesting ways in which Democrats and Republicans differed in different states.

We estimated the political ideology of voters in each state using the 2000 Annenberg survey (which asked enough questions for us to separately estimate ideology on economic and social issues, and also had a large sample--good for getting state-by-state estimates).

Here's a graph of the 50 states (actually, I think Alaska and Hawaii are missing), showing the average economic and social ideology of adults within each state. Each of these is scaled so that negative numbers are liberal and positive are conservative; thus, people in Massachusetts are the most liberal on economic issues and people in Idaho are the most conservative:



West Virginians are on the liberal side economically but are extremely socially conservative, whereas Vermont is about the same as West Virginian on the economic dimension but is the most socially liberal of all the states. Coloradans are economically conservative (on average) but socially moderate (or, perhaps, socially divided; these are averages only). (The error bars in the graph indicate uncertainty in estimation; we can't rank the states perfectly, but we can get a pretty good picture of what's going on.)

Democrats and Republicans separately

The next step is to break these voters down into Democrats and Republicans (based on self-reported party identification and following the usual practice among political scientists of throwing the "leaners" into the regular party categories). In the graph below, each state is shown twice: the avg social and economic ideologies of Democrats in the state are shown in blue, the avgs for Republicans in red.



We made these graphs during the 2008 primary election season, and one thing we noticed was that South Carolina ("SC") is in the middle of the pack among Democrats and among Republicans, but it's one of the most conservative states overall. My take on this: South Carolina is a strongly Republican state, and the moderates in South Carolina are likely to identify as Republican. This pulls the Republican average to the left (as they includes the moderates) and also pulls the Democratic average to the left (as they are not including so many moderates).

But the big thing we see from the graph immediately above is that Democrats are much more liberal than Republicans on the economic dimension: Democrats in the most conservative states are still much more liberal than Republicans in even the most liberal states. On social issues there is more overlap (although in any given state, the average Republican is more conservative than the average Democrat).

P.S. How do these rankings fit with our usual rankings of states? Here's a plot showing average economic and social ideology for each state, plotted vs. George W. Bush's vote share in 2000:



P.P.S. More info here.

There's More...

Contract Post

OFA Learned Lessons From Coakley Loss

Following is the second half of my interview with Organizing for America's Mitch Stewart and Jeremy Bird, and DNC spokeswoman Lynda Tran. The main headline is that, after a sometimes-bumpy road during the first year, OFA seems to have a plan in place to mobilize its volunteer database and figure out ways to connect those volunteers--many of whom were drawn into national politics by Barack Obama's 2008 presidential run--with state and local Democratic candidates for the 2010 midterms.

Fivethirtyeight: Are you guys are getting a sense as you get to know your list better, are you able to—and I don’t know what the word to use is here—profile your members? This is person who will call her member of Congress, and this is a person who won’t do that but will show up for a local organizational meeting, and this is a person who will write a letter to the editor, and this is a person who will do all of those.

Mitch Stewart: That’s exactly right. We do a ton of list segmentation, both online and offline, based off of what a volunteer’s most comfortable action is. So we definitely we have folks who love to host candidates and phone banks and house parties, and we have a strong sense of who those people. We have folks who show up but maybe not host, and we have a strong sense of who they are. We know made phone calls to Congress, we know who wrote letters to the editor, we know who wrote letters to their members.

So the long answer to that question is yes. We try to tailor our communication with these volunteers in a way that will allow them to get involved and engaged in the actions we’re doing.

Lynda Tran: And Tom, that actually gets to the earlier point about what does translate from the legislative organizing to the electoral organizing. We have spent the last 14 months on this, the last year or so, identifying activists and building these team structures and having in place the largest field structure the party has ever had in an off-presidential year. So that definitely is going to translate into the work we are doing this year for the midterm.


538: In terms of the midterms, shifting more into gear on that, to me—and I may be wrong about this—the most important question is the degree to which the so-called “Obama surge” voters—first-time voters as you mentioned, younger voters, non-white voters--these people who turned in waves or for the first time in 2008. We’re not expecting all of these people to turn out, but the question is how many of those people will turn out or how few. So I’m wondering what you guys are doing to identify, target, communicate with and mobilize the so-called surge electorate.

MS: It’s true that we’ll use a lot of the same tactics that we did during in 2008 and, frankly, 2009. We know who these folks are, we know where they live, we have their phone numbers. What we continue to believe that the most powerful way to reach out and talk to these voters is to have a friend or neighbor, someone that they know from their community, reach out and engage them and talk to them about why it’s so important [to turn out in 2010].

The key is that once we explain the stakes and what election day could look like, good or bad, these folks will get motivated. They don’t follow the back-and-forth of what happens here in Washington like some other voters do in this country. So a bit part of what we’re going to have to do is get out there and have conversations. That’s through phone calls, that’s through door knocks, that’s through public events, and using our online tools.

It seems to me that the more that technology advances the more that the bread-and-butter is the backbone of any strong organization. You just can’t replace those personal, face-to-face or over the phone conversation. And that’s something we’re uniquely positioned to do.


538: Are there any new social media or technological developments since 2008 I should know about that you guys are tinkering with?

LT: We definitely have some things in the works, and you’re going to hear more about it in the coming weeks. I think it’s safe to say we’re looking at some of the newer ways that people are getting their info, like iPads.


538: Anything else? Why can’t I know now?

JB: Yeah, I think there will be some interesting things we’ll be doing, and I think we’ll be advancing some of the technology. But I think also now, with some distance from the 2008 campaign, we are having an opportunity to make those things that we already had better—to make the data integration better, to make sure we moving quicker from an online interest to an offline volunteer. I know that’s not very exciting, I know that’s not very sexy for reporting. But we’ve done a lot around those upgrades. And we will have some press-worthy things to talk about.


538: I’m wondering if, either in-house or through Ken Strasma or one of your number wizards or consultants, going back to this surge voter, have you guys been able to model what kinds of people you expect to drop off and not turn out in 2010 versus which kinds of surge or first-time voters you do think will turn out? In other words, can you figure out or are you starting to figure out where your best rate of return is, in terms of contacting certain people who are likely to show up for the midterms and other people who may be less or unlikely to?

MS: Tom, that’s a spot-on question. Ken worked with me in Iowa during the caucuses. I think there are actually three stories from 2008, one that did not get told and two others that did. The two that got told were the unprecedented volunteer effort and voter contact that happened across the country, and I think the second that got told was the online effort that the [Obama] campaign had.

But the third story that really didn’t get told was the data analysis that we did in 2008 and we’re building upon it in 2010. And the regression analysis that we do, not only for candidates but also for likely participation, is something that will be the driving force for our resource allocation both with volunteer time, as far as who we talk at the door and on the phone, but also as you look at paid communications and the best way to allocate those resources.

The second thing, though, that were focused on is not just looking at candidate support and participation and an undecided score—the folks that are most persuadable—but also looking at things like who is most likely to answer the phone and who is most likely to answer their door. We are an organization driven by data. We were in 2008, but in 2010 we will become even more so. And we’ll continue to look at ways to become more efficient and data is basically the driving force in that.


538: We talked about having a vote goal and end date. It was kind of a short time line, but you did have a deadline in January special election [in Massachusetts]. And the Politico and others reported that you were going all-out, or you’re all in or whatever. So what happened there, and what lessons did you learn from that loss in the Teddy Kennedy seat race?

MS: The biggest lesson that we took away from that is that you can’t manufacture an organization in-state behind a specific candidate in three weeks or two weeks—that it takes more time than that. Part of the reason that we started this [2010 midterm] process quite a while ago, and on June 5 will be the official kickoff and will provide us the opportunity to build that volunteer capacity.

But the one caveat I’ll add to that is that we as an organization were able to generate 2.4 million phone calls in the closing days of that [Massachusetts] election. So the work that we did we fell very good about. But each campaign will be unique; each candidate will be unique. And the biggest takeaway for us is that we need to make sure we get in early.

JS: I think the lessons that we learned from NY-23, PA-12, or even places like FL-19, were also lessons about how do we engage our volunteers, how do we connect them better with candidates in a way that they can get excited about it and motivated. You need the time to do the work and get them to do the work and get engaged earlier. Those are the lessons we learned in the last year and half.


538: Tell me, now that David Plouffe coincidentally, or maybe not coincidentally, came back a week after the Massachusetts special election, can you tell me a sense of what he's doing, what his role is on a day-to-day basis? What’s changed since he came back?

MS: We actually have to get off the phone with you because we have a meeting with him in a couple of minutes. He’s somebody that we constantly talk to. He’s somebody who is still the driving force in this organization. He’s someone that looks over emails. He’s someone that’s still heavily involved in the strategic direction that this organization is heading in, and that will only continue to deepen as we get closer to 2010. And, again, I don’t think this is super interesting for what you’re writing about, but he’s the smartest person I’ve ever dealt with.

There's More...

Contract Post

Colombia Redux: Mockus is in Big Trouble

Last Sunday we previewed the Presidential race in Colombia, an election that was expected to be neck-and-neck until the bitter end. Establishment candidate and former Defense Minister Juan Manuel Santos was running to inherit the mixed political and security legacy of outgoing President Alvaro Uribe, while upstart former Bogotá mayor Antanas Mockus had surged into contention as a centrist alternative.

With polling from Gallup and Ipsos showing Santos pulling 35 to 38 percent and Mockus earning 30 to 35 percent, the results on Sunday evening were expected to show a tight finish. Regardless of who pulled off the win -- though Santos was ahead by a few points throughout -- it looked like both candidates would emerge in a strong position for the likely Presidential run-off.

However, with 99.7 percent of precincts reporting, Sunday night's results presented a commanding victory for Santos, with the "Party of the U" standard-bearer earning 46.6 percent of the national votes. Mockus, running on the ticket of his new "Green Party" earned just 21.5 percent of the vote, less than half of Santos' total.


The Santos victory was even more impressive when viewed geographically. He won every department of Colombia save Putumayo in the south, where Mockus eked out a win with 28.9 percent to Gustavo Petro's 26.7 percent and Santos' 23.4 percent.


Even in Mockus' political homestead of urban, cosmopolitan Bogotá (the 'Athens of the Andes'), Santos won convincingly, with just over 40 percent of the vote to Mockus' 27.5 percent.*

One interesting characteristic of the race, relative to other elections we have covered, was the lack of a serious socio-economic divide in vote shares among the top candidates.

Looking at the vote share per department, the vote shares of each major candidate explained by the percentage of people living in poverty is quite low. Broadly speaking, however, Santos received better results in poorer departments, while Mockus generally better in wealthier ones.


The key question going forward, as we look to the 20 June run-off between Santos and Mockus, is how much we can trust the polling that we have seen thus far, and whether the next batches will be any better.

Before we jump into any condemnations of the pollsters -- though these numbers are really horrible -- we must give them a break on a key issue. The last poll was released on 22 May, by law the last day that election-related publications could be made, in order to comply with the national rules that ban campaign propaganda for the week before the election. The fieldwork was done from 17-19 May, meaning that the data were a full eleven days old by election day -- an eternity in election time.

That said, we have to work with what we've got, and bad numbers are bad numbers.

Of the five pollsters that were published in major Colombian news outlets, the best performers were the local Gallup outfit, Ivamer, and the Centro National de Consultoría (the National Consulting Center). Based in Bogotá and focused on Colombia and a few other Latin American markets, CNC is a market research firm that also does political polling, much like most other polling outfits.

CNC is of interest because of the two best-scoring pollsters, they managed to capture the Santos rally where no other firm did. While Ipsos also captured a drop-off in Mockus' support, they reported a drop for Santos at the same time.


Overall, the hanging question regards the 'surge' of Mockus in April, a polling rally that peaked several weeks before the election, and resulted in a quite disappointing (for supporters) final result -- a situation that was eerily reminiscent of the Liberal Democrats peak and crash in the UK. Even with some additional erosion of Mockus' numbers in the days before the election, dropping from around 35 percent to just 21.5 percent in less than 10 days (with no scandal) is highly unlikely.

Plausible explanations for the overstated Mockus numbers are many, ranging from an overreliance on city-residing respondents (easier to get to) to selection bias towards wealthier respondents, poorer than expected turnout among the young, and the old chestnut of the incumbent party cooking the results in favor of their candidate.

Of course, this suggests that the previous figures showing a 5 to 10 point lead for Mockus in run-off are highly questionable at this point. Ostensibly building from a 47 percent baseline, and likely pulling nearly all of Noemi Sanin's voters and many of German Lleras, it is Santos that now looks to be in the stronger position.

We can use the basic ideology of each candidate to allocate his/her votes to Santos and Mockus for the second round.

This allocation would result in a major second round victory for Santos, by nearly 20 points.


All told, it looks pretty bad for Mockus. His main path to victory at this point is to swing back a big number of Santos voters, while minimizing losses to Santos from the conservative candidates by perhaps their staying home rather than switching to Santos -- a long shot at this point.

---
Renard Sexton is FiveThirtyEight's international affairs columnist and is based in Geneva, Switzerland. He can be contacted at sexton538@gmail.com

* Here is a chart of the Santos-Mockus results in each of the 32 departments of Colombia, ordered by Santos margin of victory (other candidates are removed to make it less confusing). You'll want to click on this chart to see the details.

There's More...

Contract Post