Obama's Domestic Afghanistan Puzzle

The national security, regional stability and humanitarian elements of the Obama Administration's Afghanistan policy are a difficult mess of intertwined problems to disentangle. Reading the lengthy profile of Richard Holbrooke, Obama's "special representative" for Afghanistan, in last week's New Yorker, only makes one depressed. "The conflict in Afghanistan will be far more costly and much, much longer than Americans realize," predicted Holbrooke in March, 2008. "This war, already in its seventh year, will eventually become the longest in American history."

Tomorrow, in fact, Afghanistan enters its ninth year, and now stands just four months shy of the American Revolution, six shy of Vietnam. With no end in sight, Afghanistan is also the riskiest and most dangerous foreign policy issue--and thus one of the riskiest electoral gambits--of Obama's presidency. Yesterday, Obama met with relevant congressional leaders from both parties to discuss American policy in Afghanistan. Legislators on both sides of the aisle expressed reservations; Obama's vanquished 2008 presidential opponent John McCain was apparently a particularly vocal critic. The president is having top-level staff meetings later this week to discuss both Pakistan and Afghanistan, specifically Gen. Stanley McChrystal's recommendation of an additional 40,000 troops.



Putting aside the policy challenges and uncertainties, how thorny is the political thicket for Obama?

Pretty thorny. There are divisions between the parties, divisions within his own party, and divisions among the American public more broadly. A new Quinnipiac poll shows how divided and discouraged Americans are. Though a 52 percent majority think "the war...is the right thing to do," 37 percent do not and by only a 49-38 percent margin do they think the war there will successfully remove the terrorist threat to the United States. (I cannot help but pause here to note that critics of Obama's health care reform who say the public option is unpopular with Americans ought to be reminded that public support for the public option is higher than that for the war in Afghanistan.)

As for pols, McCain, who expressed support for the administration's policy back in March, reportedly warned Obama against moving at a "leisurely pace" s outburst yesterday. On her Facebook page, McCain's 2008 running mate says Obama should stand firm. "Our allies and our adversaries are watching to see if we have the staying power to protect our interests in Afghanistan," wrote Sarah Palin. "I recently joined a group of Americans in urging President Obama to devote the resources necessary in Afghanistan and pledged to support him if he made the right decision. Now is not the time for cold feet, second thoughts, or indecision -- it is the time to act as commander-in-chief and approve the troops so clearly needed in Afghanistan."

There are two fundamental domestic political truths about Afghanistan for Obama. The first is that this is a war without much of a political upside because it is difficult to ever prove that we have "won" it, for even temporary victories can be reversed. There will be no armistice, convention, or capitulation, and the asymmetries of the fight mean that the "enemy," such as it is, can always regroup or displace. We can only measure victory by the vague metric of days during which no major terrorist attack hits America or its allies, or by the withdrawal or reduction in resources invested there. The second is that, although Obama inherited the war, he owns that inheritance by virtue of his statements during the 2008 campaign. He can complain about the war's intractable nature, but he cannot complain about its burden or point as directly to the persistence of the problem as a result of his predecessor's (in)actions.

Having said that, and given the grumblings by McCain and Palin, I wonder if we are about to witness the (partisan) politicization of Afghanistan in the way we saw the "Waterloo"-style opposition to Obama's health care plan. During the Bush era, Republicans used the politics of warmaking to cudgel Democrats, and warned that criticisms of the president or his policy amounted to near-treasonous politicization of, and thus the undermining of, America's national security interests. There are no real good solutions in Afghanistan, but whatever semi-promising options are available to us need not be clouded by a new round of shameless politicking designed to batter the president at the expense of non-failure--I hesitate to use the word success--in Afghanistan.

There's More...

Contract Post

Checking in on Tim and Mitt

Given the focus on health care, the 2009 gubernatorial races and 2010 congressional cycle, it's surprising how much Tim Pawlenty and Mitt Romney have been in the news of late. Let's check in on these two 2012 Republican presidential primary frontrunners.

Starting with the Minnesota governor, last week Pawlenty launched a new political action committee called Freedom First to being amassing funds for his expected '12 bid. "T-Paw," as he refers to himself (yes, on the site), is quoted in the inaugural press release saying that the "organization is dedicated to putting freedom first again in America. By helping candidates and translating our ideas into policies that everyone can relate to and support, we can turn back the growth of Washington and renew the promise of freedom." The 285-word mission statement uses the word freedom or some variant 14 times. We got it, Guv: You're for freedom--really, really, really for it. With such generic, conservative talking-point fodder, perhaps he should start going by the nickname "T-Pap."

Despite the thin rhetorical gruel, Pawlenty boasts a promising profile for a potential nominee--a governor from the key swing region of the country, young and attractive and without a lot of the social issue baggage. He's raising eyebrows. Newt Gingrich recently called him a "terrific talent...a very attractive guy [with] a good reform record." Gingrich further told The Politico that Pawlenty will be a "player" in 2012 and that there "is every reason he should run, there is wide open field right now." Politico's Jonathan Martin earlier reported that, in addition to jetting around the country to raise dough, Pawlenty has assembled a top-notch support team boasting a variety of people with presidential-level experience, including advisers Terry Nelson, Sara Taylor, Phil Musser, Freedom First co-chairmen William Strong and former Rep. Vin Weber, and a corps of tech consultants.

As for Romney, he too is fundraiser-hopping across the country to rake in the bucks for his Free and Strong America political action committee. (I'm detecting a "freedom" theme here, aren't you?) To bolster his image as a kingmaker and party leader, he's done a fundraising event for Virginia gubernatorial candidate Bob McDonnell and will soon do one for New Jersey GOP nominee Christ Christie. "Should the former Massachusetts governor decide to run for president a second time, his team from the 2008 race would be largely intact," The Hill's Reid Wilson reports. "Veterans of the first race have stayed close, and they frequently get together when their former boss is in town."

Stacking up side-by-side, Newsweek's Andrew Romano argues persuasively that in a potential matchup against Romney in '12 Pawlenty's health care record could provide an advantage: "Assuming...Romney and Pawlenty do face off in the finals, Pawlenty has at least one distinct advantage: while Gov. Romney passed a universal health-care plan in Massachusetts that looks largely like whatever will come out of Congress, Pawlenty recently took the opposite tack, seeking to balance the budget by cutting millions of dollars in funding from MinnesotaCare, a government-supported insurance system for working-class Minnesotans that had previously slashed the state's percentage of uninsured residents to one of the lowest levels in the country."

The Wall Street Journal's Allysia Finley agrees:

[T]here's also one huge elephant in the room that Mr. Romney will have to clear if he decides to run in 2012: Massachusetts' universal health care legislation, which hasn't been popular with fellow GOPers. Indeed, it must pain Mr. Romney that so many Republican critiques of ObamaCare cite the cost overruns and other problems encountered by RomneyCare. As he geared up to run for president a few years ago, Mr. Romney notably switched his positions on abortion, gun rights, immigration and "don't ask, don't tell." He won't find it so easy to flip-flop on Romney care, and he apparently knows it. Mr. Romney is sticking to his guns by defending the Massachusetts plan.

I'll go on the record right now as saying I don't think that Romney will win the nomination. He had enough problems with the evangelicals and social conservatives, and he now has an equally bigger policy problem on health care. As for Pawlenty, I think he's more likely to be on the bottom of the ticket.

UPDATE: Pawlenty will head to Iowa soon for an event.

There's More...

Contract Post

Delaware Leaps to #1 in Senate Rankings; Is 60-Seat Majority Doomed?

Although we haven't generally been updating our senate rankings more than once a month, I am making an exception in Delaware, where it was reported today that Mike Castle, Delaware's lone U.S. Representative, will be running for the seat formerly occupied by Joe Biden.

Previously, Missouri had held the number one position, another open-seat race in which polling and fundraising numbers have been slightly more favorable to Democrat Robin Carnahan than Republican Roy Blunt. I'd estimate that the Democrats have somewhere between a 55 and 60 percent chance of taking over that seat, which is now held by Republican Kit Bond.

Castle's odds are somewhere north of that range. With that said, the race is no gimme, particularly if as expected Castle is opposed by Joe Biden's son Beau, who is currently Delaware's Attorney General. Polling released by Rasmussen and PPP had shown Castle with leads of 5 and 8 points in an as-then-hypothetical matchup against Beau Biden. Castle recently turned 70 and has had some health issues and is unlikely to be as energetic his 40-year-old opponent. Although a gifted and experienced handler of his constituents, Castle may also run into trouble trying to balance some of the more radical elements of the his party with his centrist positioning, a problem common to all moderate Republicans. And, given that the Vice President's son is running, we can expect the White House to go "all-in" on this race, although the Biden last name could be a fundraising magnet for Republicans as well.

Although there's not a lot of territory to cover in Delaware, this race could get fairly expensive, as most of the state is in the pricey Phiadelphia media market.

Nationally, I think some of the gloomier analysis for Democrats are overwrought. If I assign rough probability estimates to each of the 38 available Senate seats changing hands, I come up with an average of about 3.5 Democratic seats being won by Republicans, but 2.7 Republican seats being won by Democrats. There is no particular reason to think that Carnahan can't win in Missouri, or Lee Fisher or Jennifer Brunner in Ohio, or Paul Hodes in Ohio, or one of the Democrats in Kentucky. The Democrats also have more tenuous prospects in states like North Carolina, Texas, Louisiana, and possibly Iowa. Currently, I would guess that there is about a 55 percent chance that the Democrats will enter 2011 with fewer than 60 seats, a 20 percent chance that they'll hold exactly 60, and a 25 percent chance that they'll have more than 60. With that said, the situation has certainly deteriorated from what once looked to be an extremely fortuitous cycle for the Democrats, and since the outcome of individual races are correlated based on national conditions, the prospect of a net loss of as many as 5-7 seats for the party is distinctly possible.

It should also be born in mind, however, that the mechanics of the 60-seat majority has always been fuzzy. Several of the Republican candidates in this cycle, including Castle, Mark Kirk of Illinois, Charlie Crist of Florida, and Rob Simmons of Connecticut, have a track record as moderates and could form a working group of moderate Republicans along with Susan Collins and Olympia Snowe. That is not to suggest that President Obama should be sanguine about his prospects to advance major legislation post-2010, as he's had enough difficulty even with his large majorities. But if, hypothetically, an improving jobs picture had re-charged Obama's approval numbers into the high 50's or low 60's by mid 2011, and the Democrats found themselves with only a 57-seat majority in the Senate but 4-5 Republican moderates to work with versus the two they have now, the situation would not be obviously inferior for them as compared with today. The House, where the entire chamber is up for re-election and more seismic-type shifts are possible, arguably remains the bigger worry for them.

There's More...

Contract Post

Obama-Corzine 2009?

Democrats continue to trail both the New Jersey and Virginia governors races. Last week, I raised the question of whether Northern Virginia voters can save Democratic nominee Creigh Deeds. This week I ask, Can Barack Obama save Jon Corzine in New Jersey?

As Nate wrote a couple days ago, evidence of a Corzine comeback is mixed. And there is mounting evidence that Corzine knows that perhaps only with Obama's help can he stagger across the finish line ahead of Republican candidate Chris Christie--namely, the mounting of billboard signs around the state juxtaposing the two men's names.


But the real testament is found in the state registration data and recent poll results showing Corzine either tied or potentially ahead--if, that is, one projects an electorate next November that looks like the registered votership in the state as opposed to the likely votership in the state. This key distinction requires a bit of unpacking, so let's start with the party registration data in New Jersey today as compared with the recent past.

Here are the most current voter registration data from the NJ State Board of Elections. If we hold aside the very small fraction of voters who identify as third-party registrants--that is, those registering as something other than "DEM"ocratic, "REP"ublican, or "UNA"affiliated--the three-way splits for registration work out as follows: Democrats, 33.9%; Republicans, 20.4%; and Unaffiliateds, 45.7%. These are not the final data, but registrations in the final month will not alter these shares very much.

Registration over the past four years, however, has changed these numbers dramatically. If you think the nearly 46% "unaffiliated" share is big today, four years ago on the eve of the last statewide election, as indicated by the final, official registration figures for 2004, Garden State "unaffiliateds" were an eye-popping 53.0% of registrants. More tellingly, the 14-point net (and 62 percent two-party share) Democratic advantage today is significantly wider than the 5-point (and 56 percent two-party share) advantage Democrats enjoyed just four years ago.

So, how the heck can Corzine possibly be trailing Christie? Setting aside the myriad campaign, candidate and issue-related reasons, the short answer is two-fold:

1. The electorate expected to turn out five weeks from today contrasts starkly with the registered votership, most notably in that the former is likely to be both whiter and older. (These are not mutually-exclusive distinctions, of course.) If you look at the very bottom of the Monmouth results, you'll see that the likely-voter screen they use results in a drop of about four percent in the statewide share of non-white voters. The share of likely voters ages 18-34 drops six percent. The younger, more multiracial electorate on the state registrant rolls puts Corzine ahead, 41% to 40%; the older, whiter likely-voter one has Christie ahead, 47% to 39%. (After speaking yesterday to reps from Monmouth and Quinnipiac, I'm satisfied they are weighting the samples appropriately--but more on that below.)

2. Though some of the unaffiliateds have strong partisan inclinations, among those who do not Christie is holding down a significant enough share to compensate for the wider, two-party registration gap he faces today, as compared to the one Republican Doug Forrester faced in 2005. In the Quinnipiac poll, Christie's lead among independents has shrunk from 30 points in July to just 13 now. If registered "unaffiliateds" or self-defined "indepdents" were a much smaller share of the state, the two-party advantage Democrats enjoy statewide would be enough to swamp that margin.

Which brings us back to Obama. His election--in New Jersey and nationally--depended upon forging a new coalition. Nationally, the white vote increased just 1% between 2004 and 2008, but the non-white vote jumped 19%. The (somewhat overlapping) youth voter turnout also surged. In short, an Obama-style turnout has the potential to tilt the NJ electorate closer next November toward its registration splits than its presently estimated likely-voter splits. And Obama has proved previous turnout models to be wholly wrong before (think Iowa, January 3, 2009). Which is why an Obama-Corzine may well be Corzine's best, even only, hope.

There's More...

Contract Post

Question Order May Bias Fox News Health Care Polling

There's a Vietnamese proverb, con sâu làm sầu nồi canh. This loosely translates to: a drop of poison spoils the whole glass of wine. Here is an analogous proverb in polling: a drop of bias can spoil your whole poll. Let me explain what I mean.

Fox News yesterday came out with a poll that suggested that just 33 percent of registered voters favor the Democrats' health care reform package, versus 55 percent opposed. These are not good numbers for Democrats, as they represent a backtrack from the improvement that other pollsters had shown in their health care polling recently.

The Fox News numbers on health care, however, have consistently been worse for Democrats than those shown by other pollsters. Since the health care debate began, the average non-Fox poll has shown 43 percent of the population supporting health care and 45 percent opposed -- producing a net score of -2. By contrast, the average of four Fox polls on health care has shown 35 percent in support of health care and 49 percent opposed -- an ugly -14. The differences in the net numbers statistically significant at the 99 percent threshold.

The first instinct that most of the liberals in the audience will have simply this: well, it's a Fox poll, so of course it's biased. The reality is a little bit more complicated, however. Fox News's pollster, Opinion Dynamics, generally hasn't shown much evidence of a Republican-leaning "house effect". Take a look, for example, at their Obama approval numbers. Since the beginning of Obama's term, they have shown, on average, 58 percent of registered voters approving the President versus 32 percent disapproval. This is, if anything, generous to Obama, as the average non-Fox polls has shown 57 percent approval and 37 percent disapproval over this interval.

The next suspect would ordinarily be the question wording. But Fox News's question is perfectly fine:

Based on what you know about the health care reform legislation being considered right now, do you favor or oppose the plan?

No bias that I can detect there. The question is slightly unusual in that it mentions neither "Congress" nor "Obama" nor "the Democrats", but it's not unusual in a biased way. If anything, the question is particularly unbiased -- it looks to me that support for health care improves slightly if Obama's name is mentioned, but goes down slightly if Congress is mentioned instead. Fox escapes this problem by simply phrasing things in the passive voice and not mentioning either institution.

So how can Fox News ask a seemingly unbiased question of a seemingly unbiased sample and come up with what seems to be a biased result?

The answer may have to do with the questions Fox asks before the question on health care. This weekend, for example, Fox News put out a separate release with their health care questions -- but the health care questions weren't asked separately. Instead, they were questions #27-35 of their larger, national poll, which you can find here. And what were some of those questions? Here are a few:

3. Do you think Barack Obama's travel and speaking schedule makes him look more like he is a candidate on the campaign trail or more like he is the president of the United States?

4. Do you think President Obama apologizes too much to the rest of the world for past U.S. policies?

5. Do you think the Obama administration is proposing more government spending than American taxpayers can afford, or not?

6. Do you think the size of the national debt is so large it is hurting the future of the country?

7. Would you rather: [ROTATE OPTIONS 1 and 2]
Cut spending now so future generations don't have to pay
Keep spending at current levels and let future generations pay

20. When Barack Obama was a candidate campaigning for the presidency, he spoke of the urgent need to finish the fight in Afghanistan, which he called the central front on the war on terrorism. Do you think that, as president, Obama is doing what it takes to win in Afghanistan?

These questions run the gamut slightly leading to full-frontal Republican talking points. Some of them, such as question #3, are almost literally rhetorical questions, which are never good things to have on a poll. And no, you can rest assured that Fox News was not asking questions formed from comparably biased Democratic talking points.

A respondent who hears these questions, particularly the series of questions on the national debt, is going to be primed to react somewhat unfavorably to the mention of another big Democratic spending program like health care. And evidently, an unusually high number of them do.

The reason Fox's Presidential approval numbers are not affected is because Obama approval is question #1 -- asked before the leading series of questions on spending and foreign policy or anything else. Likewise, when Fox was conducting its Presidential polling (which also did not have any discernible house effect), the horse race questions were asked before any of the policy ones.

But when you ask biased questions first, they are infectious, potentially poisoning everything that comes below. I don't particularly care if Fox News wants to ask leading or even outrightly biased questions -- but they have to ask them after any questions they expect the policymaking community to take seriously.

Another problem is that Fox tends to ask different sets of questions before they get to the health care ones -- sometimes, questions about Afghanistan, sometimes questions about deficits, etc. What seems, then, to be "movement" in the numbers may simply be an artifact of what sort of mood they've worked the respondent into before they get to the health care stuff.

To be clear, these question order effects can arise even when pollsters have the best of intentions, and even when they are asking unbiased questions. If, for instance, back during the Presidential campaign, you had asked a series of perfectly neutrally-worded questions on the economy before asking about the horse race, they could easily have tipped the numbers slightly in Obama's direction, since the economy was perceived to be the Democrats' strength. (The LA Times poll had this exact problem at various points). Pollsters have to grapple with these sorts of considerations all the time, and some do a better job of handling them than others.

But when you ask a series of biased questions before taking the voters' temperature on health care or the horse race, you have much less excuse. Going forward, Fox News should put its health care questions closer to the top of their survey or break them out into a separate poll; take their numbers with a grain of salt until they do.

There's More...

Contract Post

Seen Through Sharper Statistical Lens, Anomalies in Strategic Vision Polling Remain

Note: the below is fairly technical, but since the discussions of Strategic Vision's polling had become quite technical in the comments, I thought it was worth giving Michael Weissman, a retired physics professor at the University of Illinois and a frequent commenter at this website, a guest column in this space. Using a robust and fairly elegant statistical technique known as Fourier analysis, Weissman has found that Strategic Vision's polls indeed contain unusual statistical artifacts that are highly unlikely to have arisen by chance alone and which differ substantially from those of comparable pollsters. I have given Weissman's words a light, non-technical edit, with his permission, from the version he originally sent to me. --Nate Silver
____

Fourier visits Strategic Vision
by Michael Weissman

Three weeks ago, a polling association censured Strategic Vision LLC ("SV") as the only pollster who refused to answer repeated requests for routine information on their methodology -- twenty other pollsters had complied. Nate Silver followed up by checking whether there was anything statistically odd about SV’s results, finding that the distribution of trailing digits in their reported percentages for the two major candidates showed much larger deviations from uniformity than would be expected by pure chance draws from a uniform distribution. Some digits, such as 8, appeared much more often than others, such as 1. A closely matched comparison group of polls from Quinnipiac also showed larger-than-expected deviations from uniformity, but not nearly as extreme. A commenter on this site, "steve", sent in the results from a comparable collection of SurveyUSA polls, showing no unusual non-uniformities at all. A discussion immediately ensued on this blog and others as to whether the strongly non-uniform SV results could easily arise by normal causes or whether they constituted evidence suggesting that the results had not been obtained by other-than-normal polling methods.

One potential source of non-random non-uniformity, pointed out here and elsewhere, could be some rounding method that systematically favored evens or odds. It turns out, however, that evens and odds appeared with nearly the same frequency in the SV result. In addition, the sample sizes that SV typically uses are divisible by 100, making rounding errors unlikely.

A more challenging objection was as follows: There is no a priori reason to expect the distribution of trailing digits to be uniform, even in a large sample. We know that the full percentage poll results are not uniformly distributed from 0 to 100. Polls, rather, are generally taken in races where the leading candidates each have some major chunk of the vote. There are usually a few undecideds as well. So you might expect the distribution of ideal poll results to have a broad peak somewhere roughly in the vicinity of perhaps 45, trailing off smoothly on either side. That’s not uniform – although, if the results were routinely spread throughout the 30s, 40s, 50s and 60s (as is the case with Strategic Vision’s polls), you would expect a pretty smooth distribution. The distribution should also be cyclic, in that 0 – such as in the number 50 -- is just as ‘close’ to 9 (49) as it is to 1 (51).

The problem is that pretty smooth isn’t definitive enough to say whether the extra variance (the average of the squared differences from uniformity) should be considered alarming or not. Nate explored tentatively some other possible non-uniform distributions, but there were some justified objections that these were arbitrary. What’s needed is a way to remove the variability due to the non-uniform distribution without pretending to know just what the distribution is. Fortunately, we have some tools – in particular, a tool called Fourier analysis -- to solve what might sound like an intractably subjective problem.

Fourier waves of different frequencies combining to form

another, seemingly complex wave.

First, regardless of the true underlying distribution of results, the actual poll results cannot show any major non-random variations between adjacent digits. The reason is that SV’s polls are taken of relatively small numbers of subjects (generally 600, 800 or 1200), leaving random uncertainties in each result of about 2 percent. If, for example, there were some (wildly implausible) real tendency of the true values to cluster on even digits as opposed to odd, the poll results wouldn’t show it very much because the random errors would smear them out too much.

Second, we have a standard mathematical tool called Fourier analysis to describe our ten digits in terms of components. These components can be manipulated such that the non-random non-uniformity is concentrated in some of the components, while leaving others random. This provides for a big advantage over the initial form, in which the non-uniformity might be distributed among all ten numbers.

One of the Fourier components is completely flat and just represents the average value. The other nine Fourier components are sinusoidal waves on our plots of occurrence rates for the ten digits. These include a range of broad and narrow waves. The wave with the most frequency is the period-2 even-odd cycle -- but I have chosen to ignore this because it might plausibly arise from rounding methods. The most slowly-moving wave has period 10. There are two such period-10 components, with different peak locations. These are the components where some plausible non-uniform distribution could show up, even after smoothing by the random sampling error. So we can remove them too, without bothering with arguments about what we think they should be.

Now comes the fortunate part: the smoothing of the distribution from random sampling effectively wipes out all the non-random shorter-period components. This reduction in the shorter-period components can be calculated with great quantitative precision using the known width and Gaussian shape of the sampling error distribution, providing for very clean random sampling variations.

Does this leave us enough statistics to work with? There were originally ten Fourier coefficients, and we’ve thrown out the irrelevant mean, the two that could reflect non-uniformity, and the one that could come from rounding. That leaves six with random amplitudes. It would be nice to have more than six, but that’s enough to catch extreme cases. We know how big the coefficients should be on the average because standard simple statistics tell us precisely how big the random variations are on average in our numbers.

Now we can ask: how big is the variation of the numbers, after all the suspect components of the variation are filtered out, compared to the statistical expectation? Remember, this filtering is done precisely in response to the serious objections, largely by defenders of SV, which were made to Nate’s original post, removing the potentially innocent components which could have made SV’s statistics look suspect. Here, then, is the filtered variance as compared to the statistically expected filtered variance:

SurveyUSA: 0.46
Quinnipiac: 0.30
Strategic Vision 4.40

How unlikely are those results? The SUSA result is pretty much typical. Quinnipiac actually has notably low variance, but random chance would result in variances that low or lower about 5 percent of the time. The Strategic Vision result, on the other hand, or something more extreme, would occur by chance with probability only 0.00019. That’s not as low a p-value as the results obtained without filtering the non-uniform components, but it’s still very low -- less than one chance in 5000 to have occurred by chance alone. For statistical sophisticates, this is a genuine relevant p-value, testing a well-specified prior hypothesis, not the sort of misleading p value obtained when one screens many data sets looking for anything unusual.

I’d like to thank Nate for getting this started and various commenters for helping keep the discussion lively: especially ecarlson, Mark Grebner, MarkinIL, steve, shma, and loner. Finally, since the core issue here is transparency, I’ve included the code by which the p value was calculated. Anybody who writes real programs will get a kick out of this, since I used a baby language (Basic) to handle a fairly basic statistical problem.

>list
10 dim d(10)
15 d(0) = 562
16 d(1) = 431
17 d(2) = 472
18 d(3) = 490
19 d(4) = 526
20 d(5) = 599
21 d(6) = 533
22 d(7) = 639
23 d(8) = 676
24 d(9) = 616
30 for i = 0 to 9
40 sum = sum+d(i)
41 sumcos = sumcos+(d(i)-554.4)*cos(0.6283*i)
42 sumsin = sumsin+(d(i)-554.4)*sin(0.6283*i)
43 sumdif = sumdif+d(i)*(-1)^i
45 sumsq = sumsq+d(i)*d(i)
50 next i
60 ave = sum/10
70 print ave
80 var = sumsq/10-ave^2
90 print var
100 lowf = 0.02*(sumcos^2+sumsin^2)+0.01*sumdif^2
110 print var-lowf
120  dev = (var-lowf)/(0.6*ave)
130 print dev
140 p = exp(-3*dev)*(1+3*dev+4.5*dev^2)
150 print p
>run
554.4
5521.44
1463.071532
4.398363
1.882962E-04

Michael Weissman is a retired physics professor (University of Illinois) whose research has focused on using random noise to characterize disordered materials. He is a Fellow at the American Physical Society, was once nominated for the Nobel Peace Prize by Barbara Boxer, and was born and raised a St. Louis Cardinals fan.

There's More...

Contract Post

In New Jersey, Two Glasses Half-Empty

Stu Rothenberg has New Jersey half-right:

Describing Corzine as closing the gap or pulling closer conveys the impression that Corzine is gathering support and increasing his standing in the contest. He is not. He hasn't moved in the Quinnipiac University poll (or in other polls, for that matter) since the beginning of the year.

Corzine's chances of winning re-election now are no better than they were a month ago. The governor continues to be stuck between 38 percent and 42 percent in the ballot test, where he has been for many months, and the fundamentals of the race continue to favor the Republican challenger.

Rothenberg is absolutely correct that it would be improper to convey the impression that Jon Corzine, the embattled Democratic governor of New Jersey, is surging in his race against Republican challenger Chris Christie. Corzine's numbers, which have been stuck around 40 percent some time, haven't really moved at all. What's happened, rather, is that Christie has been losing support:



But Rothenberg is wrong that "Corzine's chances of winning re-election now are no better than they were a month ago". Those voters that Christie is losing aren't disappearing into the ether. They're moving, rather, to independent candidate Chris Daggett, who has run on a Bloombergian platform (pro-reform, pro-environment, pro-choice) although without any of Michael Bloomberg's monetary firepower. The decline in Christine's numbers since July and the rise of Daggett's correspond nearly one-for-one.

It's hard to say what will happen to those Daggett voters on Election Day. You can perhaps draw a parallel to Minnesota's Senate race, where you had an unpopular incumbent (Norm Coleman) running against a challenger (Al Franken) that many people were uneasy with. In that race, the third-party candidate, Dean Barkley, wound up with 15 percent of the vote, almost exactly where pre-election polls had pegged him.

But Minnesota was different; there was a Presidential race on the top of the ticket and Minnesota was a swing state. People who showed up to cast their votes for Barack Obama or John McCain had to vote for someone in the Senate race. Well, not technically: it's perfectly legal to undervote a race and a few people skipped the Senate contest. But the vast majority figured, as long as they were there in the ballot booth, they might as fill out one the ovals for Senate; 15 percent went ahead and voted for Barkley.

In New Jersey, it's the gubernatorial race at the top of the ticket, and this being an odd-numbered year, in fact, there are no federal races at all on the ballot. So what will those Daggett voters do? There are two more debates remaining in New Jersey and I suppose there is an outside chance that Daggett can surge further, to the point where he actually appears he has a shot at winning. But if he doesn't (and in all probability he won't), and the Daggett vote appears to be purely symbolic, a lot of those folks might exercise their right to stay home. From the standpoint of Corzine and Christie, however, this makes no difference: a Daggett vote and a nonvote are essentially the same.

Then there's the issue of New Jersey's undecideds, which constitute about 7 percent of the electorate. In the latest Quinnipiac poll, 8 percent of Democrats are undecided as are 7 percent of independents. The Democratic undecideds, most likely, will probably be deciding between Corzine and not voting. Although it's likely that a plurality of the independent undecideds will turn to Christie, to whom they give somewhat better favorability ratings, a lot of them will probably stay home too. And there are hardly any Republican undecideds -- just 2 percent of them, according to Quinnipiac. So Christie won't get much help there.

Here's what this boils down to: there are probably a finite number of people willing to get out of bed and vote for Jon Corzine on November 3rd. And it's a number, moreover, that wouldn't ordinarily be enough to allow a candidate to carry the state. But the voters who don't want to vote for Corzine have two alternatives, other than voting for Christie: they can vote for Chris Daggett or they can sit the election out. If enough of them choose one of those options, then Corzine can still win a low-turnout election. Although Corzine remains the underdog, that possibility is indeed looking much stronger than it did a month ago, and Christie still has some work left to do to convince people that he is worth voting for.

There's More...

Contract Post

The Weekly Standard's John McCormack is a Socialist or Something

In a post entitled, "White House: Opposing Obama's Olympics Lobbying Is Unpatriotic or Something", John McCormack, the deputy editor of the Weekly Standard, writes

I was rooting for Rio. The United States has hosted the Olympics eight times before. Not a single South American country has ever hosted the games. Isn't it only fair for Brazil to get its day in the sun?

Why, yes, that does seem fair. South America is not a particularly large continent; the whole continent has only about 30 percent more people than the United States does. But for one of its great cities -- Rio, Buenos Aires, Santiago -- to have never hosted an Olympic Games is a little jarring. Cheers to Rio, which will be a fine host.

On the other hand, doesn't this sound a little, uh, redistributive? The United States pays in slightly more than half of all Olympics broadcasting rights fees, which in turn makes up about half of the Olypmics' budget. Most of the balance is paid for by corporate sponsorships -- but six of the Olympics' twelve major sponsors (Coca-Cola, McDonald's, General Electric, Kodak, Johnson & Johnson and Visa) are based in the United States. (All supporting documentation is here). Why should those hard-working United States corporations and television networks have their money redistributed to a bunch of Greeks or Brazilians or Chinese? We should host the Olympics every other year, dammit.

OK, so the preceding was sarcastic. The truth is, the Olympics are the sort of prize that you might not want to win. The financial windfall they produce for their host cities often fails to outweigh the cost, and some cities like Montreal and Athens have ended up in debt as a result. The requirements imposed by the IOC, which fully leverages its bargaining power, are fairly ridiculous: Chicago, which recently spent $600 million to renovate the 61,500-seat Soldier Field, was going to have to build a new facility just to host the Opening Ceremony because a football stadium evidently isn't big enough.

But the fact is, we probably wouldn't be hearing conservatives like Mr. McCormack "Rooting for Rio" if John McCain had been elected President, and he were lobbying for Phoenix's Olympic bid instead. And we certainly wouldn't be hearing many of them -- including McCormack and his colleagues -- erupt in cheers after the American bid had lost.

Nor do I think you'd have seen liberals reacting that way. Although there has also been some liberal criticism of the Olympic bid, and some liberal sentiment that the United States has been a Very Bad Boy and doesn't deserve the Olympics, I honestly don't think you'd have seen the Netroots Nation convention burst into cheers once the Phoenix were rejected*.

For Obama to have gone to Copenhagen to pitch the event may have been a mistake -- a few phone calls from Washington might have had 98 percent of the impact for 2 percent of the exposure. But he went in his capacity as an American President, and not as a partisan. That the conservative intelligentsia reacted giddily to news of the Americans losing is telling. It's telling of a movement that was long ago knocked off its intellectual moorings and has lost the capacity to think about what people outside the room think about. Sometimes -- certainly on the health care debate, very probably on the bailouts question -- conservatives back into something approaching mainstream American sentiment and can cause Obama and his allies a lot of problems. But any movement which also criticizes the President for giving a speech to schoolchildren, which cheers when the United States loses its Olympic bid, is mostly just engaged in the business of throwing a bunch of Kaká at the wall and seeing what sticks. I don't know whether it's unpatriotic -- but it's pretty freakin' dumb.



* Although if you change 'McCain' to 'Palin' in my counterfactual, and 'Phoenix' to 'Anchorage' -- presumably this would be a Winter Games -- I'll grant you that it might have been a little closer.

There's More...

Contract Post

Disband the IOC!

Congratulation to Rio de Janeiro, which will host the 2016 Summer Olympics, having beaten out Madrid, Tokyo, and my former hometown of Chicago for the honor. Rio will be a worthy place to hold the Games, as any of these cities would have been. Nevertheless, it's worth considering whether the IOC is as just as it could be and what role this played in the defeat of Chicago and the other cities.

Here are the home continents of the 108 current IOC members:



Europe, needless to say, has a very healthy representation. Although North America has 12 percent of the seats, the United States itself has just 2 out of 108 -- as many as Morocco, and fewer than tiny countries such as the Netherlands (3) or Switzerland (5) or the much less populous Australia (3). The United States is not the only country which is underrepresented -- China, which represents about one-fifth of the world's population, also has just 2 seats, although the number rises to 4 if you count Hong Kong and Taiwan. Japan has just 2 seats, as does Brazil; India, the second-most populous country in the world, has only 1.

Aggregating things back up at the continental level, we see that it's Asia that is especially underrepresented:



Arguably, however, this is not the right metric. We can look at something like the number of participating athletes in Beijing instead:



This is quite a bit more proportionate. Likewise with the number of medal winners:



On the other hand, it's money that makes the world go round -- especially in the Olympics -- so perhaps rights fees should be taken into account instead. And there, North America takes the cake -- the United States itself pays in about half of all broadcast rights fees for the Summer Olympics.

If we average these four metrics, we get the following:



Asia and North America can make a good case for more representation; Europe and Africa should probably have less. It's extremely bizarre, for instance, that Africa has almost as much representation on the IOC as Asia when Asia has about four times as many people, sends twice as many athletes to the games, and wins six times as many medals.

If we were using this formula to determine the number of seats for the United States in particular, it would argue for 20 seats of the 108 -- not the 2 they actually hold. And were that the case, Chicago would very possibly have won the Olympics, rather than being the first country eliminated. Here were the voting totals by round:

We see that Rio's total exploded in Round 2 after Chicago was eliminated -- almost all of the people who had voted for Chicago originally transfered their votes to Rio -- and then again in Round 3 after Tokyo was ousted (somewhat bizarrely, Tokyo actually lost two votes from Round 1 to Round 2 after Chicago was eliminated). Had Rio been the first eliminated rather than Chicago, those votes might have gone to Chicago instead. And, obviously, if there were a more proportionate number of delegates from the United States, Chicago's chances would have been very strong.

The headline aside, the IOC probably does not need to be disbanded. But we should recognize that the organization is in essence a cartel, and sets the rules as it pleases -- and has goals that appear to be pretty far removed from any sort of proportionate representation of its member countries. A number of relatively obvious reforms could be adopted:

1) Apportion IOC membership on a formula basis, in reflection of population, participation, revenues, and possibly other objective metrics;
2) Publish the votes of individual IOC members;
3) Adopt a true Instant runoff voting system, rather than let the delegates switch their votes from round to round, which increases the likelihood of gamesmanship and deal-making;
4) Adopt a rule, as FIFA uses for the World Cup, that the Summer Olympics cannot be held on the same continent on successive occasions (something which has not happened since 1948/1952 anyway, when London and Helsinki hosted the games in consecutive Olympiads).

There's More...

Contract Post

Proportional Voting? Well, Kinda

People have long criticized the process electing the US President, where a candidate that receives the most popular votes can lose, and the incentives for geographic campaigning and representation are quite skewed. While some have called the debate impractical, the 2000 results proved that the issue was far from academic. Nonetheless, the Electoral College system, while threatened by alternative state-based measures, remains the law of the land.

In Germany, where the recent Federal election ushered in a centre-right majority, proportional voting at the Federal level is protected in the country's Basic Law (effectively, the constitution). In fact, the parliamentary voting system is designed to both to ensure proportional power of parties (and therefore the chancellorship and ministries) as well as including districts where members of parliament are directly responsible to their constituents.

As such, in theory, the Chancellor should always be selected from a party or coalition that has received the majority of the votes. For example, the parties of the grand coalition that has governed until the present earned a combined 69.4 percent of the national vote in 2005. The CDU/CSU, having a slightly larger share, was allowed to name the head of government.

In 2009, however, the clash between district-based seats and the overall popular vote (done by party list) resulted in a significant shift in the allocation of Bundestag seats -- almost enough to change the result. In constituency voting, shown above (from then official, provisional returns), the CDU and CSU absolute crushed all other parties at the district level. Of the 299 district seats in the Bundestag, the Christian Democrats picked up 173, while the CSU won every seat in Bavaria, for a total of 218 of the 299, or 73 percent. The Social Democrats won just 64 seats -- a mere 21 percent. The Free Democrats, who will join the new government, won not a single seat, while the Greens and The Left won 17 seats between them.

However, in terms of national popular vote -- tallied and allocated at the State level -- the results are much less impressive for the center-right.


The CDU/CSU plus FDP partnership pulled only 48.4 percent of the national vote, which by Basic Law regulations, entitles them to just 48.4 of the total seats in the final Bundestag - which ostensibly should have 598 members.

Theoretically, at this point the 299 party list members of the Parliament should be allocated to give the appropriate proportionality in the final accounting: With the CSU/CDU due 33.8 percent of the 598 (202 member) they should get enough additional seats to get up to that margin, while the SPD needs to be "topped-up" to 137 seats.

However, CDU/CSU have already earned 218 seats from the district voting and the system, of course, does not allow for seats to be taken away from a member who won the district (or forcing him or her to change party). Therefore, additional "overhang" seats are added to the parliament, in this case raising the total number of Bundestag members to 622 (24 overhang seats).

To add an extra wrinkle, party list seats (based on the popular vote) are allocated at the State level, rather than nationally. So in Bavaria, for example, the CSU won every constituency by plurality earning 45 seats, the popular vote entitled then to just 42.6 percent of the 90 seats (38 seats), meaning 7 overhang seats are required.

Overhang seats in hand, the CSU/CDU earned 239 seats of the 622 total, or 38.4 percent of the parliament (over-performing their popular vote count by nearly 5 points), while the Free Democrats pulled their proportional 93 seats (15 percent of the total), all earned from the party lists. While the popular vote should have kept the coalition below the 50 percent majority line, except the Black and Yellow coalition walked away with more than 53 percent of the seats and a comfortable majority*.

This will likely be the last time this happens, however. In 2008, the top court in Germany ordered that the system be repaired such that proportionality is restored. The Bundestag has until 2011 (two years before the next Federal Election) to get it sorted out.

Conversely, in the US, "overhang votes" for President -- where winning constituencies (states) is rewarded rather than winning popular votes nationally -- do not seem likely to be dealt with in the near future.

In the German case, it seems certain that Angela Merkel would have retained her post as Chancellor with or without the overhang situation. However, without those extra seats she would have needed to find an additional coalition partner had a sliver thin margin - possibly moving the government towards the center. While no-one will ever accuse her of "stealing" the election, as was widely said following the 2000 results in the US, it's clear that she got a powerful advantage.

---
*Update: A small, but important element was neglected in the first drafting of this article. In the party list allocation, parties that receive less than 5 percent of the vote are disqualified from receiving any proportional seats. (6 percent of the vote fell into this category, including the NPD and Pirate party). This meant that the original article overestimated the impact of the overhang seats. In fact, they were not needed for the CDU/CSU plus FDP majority.
---
Renard Sexton is FiveThirtyEight's international columnist and is based in Geneva, Switzerland. He can be contacted at sexton538@gmail.com

There's More...

Contract Post

Analysis: Gay Marriage Ban is Underdog in Maine

Back in April, I conducted an analysis of the prospects of a gay marriage ban becoming law in each of the 50 states. The analysis found that support for gay marriage bans was strongly tied to two factors: the degree of religiosity in a state, as measured by 2008 Gallup tracking surveys, and the year that the initiative was up for vote -- marriage bans have lost support at a rate of about 2 percent per year, ceteris paribus. That analysis concluded that a Maine is one of 11 states that would probably vote to reject a ban on gay marriage if a referendum were held this year.

Mainers, in fact, will soon have a chance to test this proposition. In November, they will go to the polls to vote on Question 1; a yes vote would overturn a law passed earlier this year by the state's legislature that permits gays and lesbians to get married in the state.

I decided to re-visit my model, which consists of a relatively simple data set of all previous anti-gay marriage initiatives. 31 of 32 such initiatives have passed, the sole exception being Arizona Proposition 107, which failed in 2006, although Arizona's voters decided two years later to approve a similar measure that limited its scope to marriage rather than civil unions. I've expanded the model to include a new variable, which -- pursuant to the Arizona case -- is whether the initiative sought to ban civil unions in addition to marriage. (Although I'd given this a cursory look before, I evidently wasn't careful enough, because it turns out to be highly statistically significant). I then placed the initiatives into a regression model, which yields the following results:



Bangay is the percentage of the vote in favor of the marriage ban; this is the dependent variable. The independent variables are year, which is the number of years elapsed since 1998, relig, which is the percentage of the state's residents that consider religion "an important part of [their] daily lives", and civil, which is whether the initiative sought to ban civil unions in addition marriage.

All three variables are highly statistically significant. Support for the marriage ban rises nearly one-for-one with religiosity; it falls by about two points (actually, 1.9) for each passing year, and it falls by 5-6 points if the amendment seeks to ban civil unions in addition to marriage.

Maine is the third least-religious state in the country, according to Gallup, with only 46 percent of that state's residents saying religion is an important part of their daily lives. That bodes well for those who are hoping the initiative fails; the comparable fraction in California, which passed Prop 8 last year, is 57 percent. We're also another year down the line on a type of initiative that pretty reliably loses support with each passing election. On the other hand, Question 1 would not seek to overturn civil unions, which gives it a better chance of passing.

Throw Maine's numbers into the model, and we come up with an estimated level of support for the ban of 43.5 percent, with 56.5 percent opposed. In other words, the model's prediction is that the ban will fail. The standard error of the forecast (not the margin of error, which is larger) is 5.2 points. This implies that the marriage ban only has about an 11 percent chance of passing.

But don't start counting your (gay) chickens yet, because there are a couple of additional circumstances that are relatively unique to Maine. One is that this is a standalone initiative in an off-year election in which voters will have few other things to consider. What sort of electorate will turn out? This precedent has previously occurred twice, in Texas and Kansas, both of which voted on marriage bans in 2005. That Texas and Kansas voted to approve the marriage bans is no surprise, but the margin was somewhat wider than the model predicted -- 76 percent in Texas, rather than the prediction of 71 percent, and 70 percent in Kansas, rather than the prediction of 67 percent.

Arguably, that implies that an marriage ban will gain about another 4 points' worth of support if it occurs in an off-year election. If that is the case, the projected support for the marriage ban is 47.5 percent, which means that it has a 32 percent chance of passing -- about one in three.

It is debatable, however, whether the same ought to hold true in Maine. In Texas and Kansas, which are conservative states, the base electorate is particularly conservative, and it's the base that comes out to vote in off-year elections. But Maine is a fairly liberal state, and it's not clear where the base lies there. In 2006, a mid-term election, 26 percent of the Maine electorate identified itself as liberal, and 26 percent as conservative. In 2008, with a much larger turnout, the numbers were essentially unchanged: 27 percent liberal and 28 percent conservative. On the other hand, while 27 percent of Maine's voters identified themselves as having no religion or an "other" religion in 2008, only 20 percent did in the lower-turnout year of 2006. On balance, I suspect the off-year status of the election is slightly more likely to help Question 1 than to hurt it -- old people vote in off-year elections, for instance, and old people mostly don't like gay marriage. But the effects are not what they might be if we were talking about a state like Oklahoma.

Another factor is that this is the first time that a state's voters will be considering on a gay marriage bill that was actually affirmed by the state legislature. With the exception of Prop 8 -- which was a response to court rather than legislative action -- all of the other marriage initiatives have been preemptive in nature. In addition, unlike virtually all other initiatives, Question 1 would not seek to ban gay marriage in the state Constitution; it would merely overturn the legislature's decision. I figure that the first contingency is probably slightly unfavorable to Question 1 second is probably favorable, but this is fairly speculative.

There have also been three polls conducted on the gay marriage ban, although one is somewhat out of date:



On average, the 'No on 1' position -- which would preserve gay marriage -- appears to be about 3 points ahead. It trails slightly, however, in the only poll of likely voters, which is the one from Research 2000 / Daily Kos.

Time to play oddsmaker: I'd lay about 3 to 1 against the marriage ban passing. But it's liable to fairly close -- clearly a winnable campaign for conservatives and a losable one for liberals, especially if the sort of complacency sets in that we saw in California*.


* With that said, the model predicts that Prop 8 should have gotten 54 percent of the vote in California when it actually got 52 percent. So it's not clear if the No on 8 campaign deserves quite the flak that it's gotten.

There's More...

Contract Post