A Refinement to the Adjustment, Part I

In consideration of everyone's feedback, I am making two refinements to the timeline adjustment that I introduced yesterday.

The first refinement is to slightly dampen the effect of the timeline adjustment at the endpoints of the curve. The second is to use a state-specific timeline adjustment, rather than a one-size-fits all model. I will describe the first adjustment in this post.

Before I continue, I want to make clear what the goal of this project is. I want to provide you, at any given moment in time, with the best possible projection of what's going to happen in the November election. This is inherently a forward-looking exercise. If what you're interested in instead is simply a summation of what the polls are telling you now, there are plenty of other websites that can provide that for you. I do require that the projections be based on objective and quantifiable evidence. For example, I'm not going to say: "McCain is awful on the campaign trail, and people don't realize it yet. Let's take 5 points off his averages". Nor am I going to say "I heard from a well-connected source that the Republicans have put together a devastating attack ad on Barack Obama. We'd better cut his win percentage by 10 points". But that doesn't mean I'm going to limit myself to simply averaging the current polls.

* * *

In the long methodological discussion that we have had over the past couple days, there is one important point that hasn't been raised. Suppose you grant me that my timeline adjustment does an essentially optimal job of telling you what would happen if the election were held today? Does it necessarily follow that that the best projection of what would happen if the election were held today is also the best projection available to us of what would happen if the election were held tomorrow?

In other words, suppose that we are holding an election for the President of Hell. The candidates are Gary Condit and Mark Foley. In June, Foley leads by 2 points. In July, Foley leads by 5 points. What is our best possible projection in July of what the outcome will be in November? There are three possible answers to that question.

1. The random walk hypothesis. There is no way to guess whether the polls will move upward or downward in any given future period. Therefore, if a candidate's current lead in the polling is 5 points, our best guess at the eventual election outcome is 5 points.

2. The bounce hypothesis. Polls have some tendency to regress back to the mean established in previous periods. Therefore, if a candidate leads by 2 points in June, and by 5 points in July, our best guess is that he will probably finish somewhere between 2 points and 5 points ahead.

3. The trend hypothesis. This is sort of the opposite of the bounce hypothesis. Polling from previous periods does tell us something, but those polls are inversely related with the eventual outcome. So if Foley leads by 2 points in June and 5 points in July, that is evidence that he is trending upward, and is likely to eventually win by some number greater than 5 points.

I've tried to produce an answer to this question in several different ways, revisiting it this weekend by using Andrew Gelman's dataset. In some cases, like in 1988 or the summer of 1992, when the movement in the polls was fairly unidirectional for long periods of time, the more recent your poll was, the better off you'd be. In other cases, like in 2000 and 2004, the polls tended to oscillate, as though regressing back toward the mean; a bounce was usually just a bounce.

We can model this more formally by using different LOESS curves. The smoothness of a LOESS curve is determined by something called the smoothing parameter. A smoothing parameter of .7 or .8 will give you a very conservative curve that reacts slowly to new information (put differently, it still places some value in old information). A smoothing parameter of .3, on other hand, will give you an extremely volatile curve that gives a strong presumption to the most current information.

I went back and tried to evaluate whether there was an optimal smoothing parameter based on the weekly national polling averages from 1988, 1992, 2000 and 2004 (skipping 1996 because my dataset is scattershot for that year). I was looking for an answer in the following form: with X weeks to go until the general election, you will minimize your error by using smoothing parameter Y. If Y is a smaller number, like .3, that would be evidence for the random walk hypothesis or perhaps even the trend hypothesis. If Y is closer to .8, that would be evidence for the bounce hypothesis.

Unfortunately, there is no clear answer to this question. Different parameters performed better or worse in different elections, and at different points in those elections. All smoothing parameters from about .3 to .8 produced roughly the same average error when applied to the weekly polling data, with a possible exception of the two weeks immediately prior to the election, when a smaller parameter (e.g. a more sensitive curve) may be more desirable.

What this tells us is that it's frankly a judgment call as to how much emphasis we want to give to the most recent polling results. Neither the random walk hypothesis nor the bounce hypothesis can really be ruled out (we can probably rule out the trend hypothesis, however, as that would require low smoothing parameters to be demonstrably better than higher ones).

What I wound up doing was using a hybrid smoothing parameter, which is conservative toward the endpoints of the curve, but more aggressive in the middle of the curve.



There is a good, logical reason to do this, namely that we have less information available to us at the endpoints of the curve than we do in the middle. We can fairly clearly isolate the impact of something like Jeremiah Wright's first appearance on the scene, because we can look at polling both before and afterward: we see Obama's polls tumbling and then recovering. However, in trying to evaluate the polls right now, we only know what the polls were in the past; we do not know in which direction they'll move in the future. The hybrid curve allows us both to be fairly aggressive in isolating events that might have impacted the polls in the past, but also erring on the side of caution about the present direction of the polls.

The net effect of all of this is a somewhat more conservative estimate of Barack Obama's current strength in the polling; we know he's bouncing, but we don't know how long that bounce is going to last. If his polling remains strong into next week, that will be three weeks in a row where his numbers have shown a marked improvement, and even the most conservative estimator will start to give him credit for more or less the entirety of his bounce. If he and McCain regress back to a tie, on the other hand, we may even start to take a point or two away from polls that were conducted over the past couple of weeks. This is one thing, by the way, that I think some of the McCain supporters around here are missing. If Obama's post-nomination bounce does prove to be a temporary thing, we will be able to adjust for this more quickly, and recognize that states that were polled frequently during this period may not be as strong for him as they appear.