情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,...

2009-02-12T22:01:00.000-05:00

Obsessed,Here's a quick explanation of a random wa...

2008-06-30T11:53:46.170-04:00

Obsessed,

Here's a quick explanation of a random walk:

Imagine a bug on a string. The bug takes steps that are 1mm long. The probability that the bug takes a step to the right is p, and the probability that the bug takes a step to the left is (1-p). This bug is making a "random walk" in 1 dimension. The path of the bug along the line will look a bit like a drunk stumbling around. After t steps, we don't know exactly where the bug will be, but we do know that probability of the bug being at a given location.

In fact, we can state that after t steps, the expected (average) displacement of the bug is, in mm:

pt - (1-p)t.
Where negative numbers indicate a net leftward movement, and positive numbers indicate rightward movement.

The variance of the distribution of positions after time t is given by:

4*t*p*(1-p)

So the standard deviation of the distribution is proportional to the square root of t.

This problem can be extended to higher dimensions and more complicated rules regarding steps. The math gets more complicated, but the basic idea is the same. Random walks are used to model a wide range of processes, from financial markets to atomic diffusion.

A simple theory of voter preferences would be that it evolves over time as a random walk. That is, each day the probability that a voter will vote for Obama increases with probability p or decreases with probability (1-p). In reality the model would be a little more complex than this, but the result is that the prediction error due to evolving voter preferences should grow over time as roughly sqrt(t), which is what Nate empirically observes.

In my experience (http://election-projection.net),...

2008-06-30T11:20:09.653-04:00

In my experience (http://election-projection.net), 10K iterations gives you a different looking electoral vote distribution every time. To get a stable (and therefore accurate) curve, 500K iterations looks like a safe number.

Much, much improved IMHO. The red states look red...

2008-06-30T09:54:32.050-04:00

Much, much improved IMHO. The red states look red, the blue states look blue, and you don't contradict states where you have recent polls. Since what I mainly want is an accurate poll regression and trends, your site is perfect for me.

Now my question is, can we have top 10 list of states most affected and least affected by national trends?

So it looks like you've basically narrowed the ban...

2008-06-29T13:45:31.027-04:00

So it looks like you've basically narrowed the bandwidth in your estimator, but you don't mention how you chose either the older one (with a higher degree of smoothing) or the new one (with a lower degree of smoothing). In my field at least (machine learning), the standard way of doing that these days is with leave-one-out cross-validation. Did you just eyeball it or something?

kubla000, you're ranting against the second law of...

2008-06-29T09:34:39.404-04:00

kubla000, you're ranting against the second law of thermodynamics. Information cannot be conjured out of thin air. All Nate does is to extract as much information as possible, aiming to contaminate it, in the process, as little as possible. And he's really, really good about that.
The diminishing differences between scores are a mark of improved information, in fact, as more recent, reputable polling is available.

The IA difference for McCain/Obama is that Obama b...

2008-06-29T01:50:55.251-04:00

The IA difference for McCain/Obama is that Obama built a HUGE organization in Iowa that never stopped working because it was built so heavily in local activists and electeds, whereas McCain skipped the state and never built a structure here.

Obama still has gigantic name ID here. It is a safe blue state for him.

But do they say "soda" or "pop" or "coke" in Minne...

2008-06-29T00:05:24.635-04:00

But do they say "soda" or "pop" or "coke" in Minnesota? Check on this site, and also be sure to click on the detailed map. I grew up saying "sodapop." Must be a hybrid.

http://popvssoda.com:2998/

@ lilnev: "Ask 'em to say "soda", and if the "O" i...

2008-06-28T23:42:25.970-04:00

@ lilnev: "Ask 'em to say "soda", and if the "O" is really, really long, that's Minnesota." If that's true, then many Minnesotians (or whatever they call themselves) must have roots in Hamburg and Schleswig-Holstein. And I always thought, they all ended up in Chicago ...

Kubla,Nate pointed out on his latest thread that f...

2008-06-28T23:20:05.064-04:00

Kubla,

Nate pointed out on his latest thread that for most states the contrast between states will become starker over time (the reds become redder the blues become bluer) so that the true "toss-up" states will remain the ones that a good model may do better in projecting than one would obtain from naive assumptions or a naive model (e.g., that the 2008 electoral map will look like the 2004 election outcome).

Nate also noted yesterday that the pace of daily state poll production is picking up, and in states with a lot of polls the polling averages are going to converge to the trend adjusted figures AND to the 538 regression -- if he's doing the regression right.

You use the term "prediction." This was never a "prediction" site. To do that, you'd also have to use your intuition or a crystal ball to predict future contingent events in the campaign. Instead this site was and remains a "projection" site, attempting to systematically forecast the implications of currently available, and updated, information on the electoral college vote shares.

The latest set of improvements to the models takes those projections a step further by, in effect, dampening the implications of current popular preferences on future outcomes -- based on past evidence of "mean regression" of the leading candidate.

There was and is nothing magical or prophetic about this system. But it's extremely clever in how it has been built up step by step, so that now it's a more believable system than it was in the beginning. And it's been built up in interaction with his "panel" of experts (us -- his readers).

To my knowledge nobody else is doing it, at least not in real time on an active blog, with updating as each round of new polls becomes available. Political scientists will be doing it about the 2008 election long about 2009 or 2010 -- after the election is over. That's their usual way. One wag once described political science pejoratively as "slow journalism." Everybody knows what happened already and we have all the data to retrofit the model and "postdict" the outcome of the election. It can be a very instructive exercise, but who wants to wait until 2009 or 2010? (I know that several political scientists are reading this blog; no offense intended!)

For sure some wiseguy is going to come along and tell us that if we know the median polling estimate of the vote share in each state using the polls taken in the last week prior to election day, we can almost perfectly predict which candidate will win each state on election day.

Well BFD. Let's all wait til Mickey Mantle's birthday to pay serious attention to the polls. But that is merely a prediction exercise, and waiting til October 20th wouldn't be any fun and would provide no understanding of how the process unfolded. You don't have to be an astrophysicist to carry out such an exercise.

Here we have the adventure (IMO) of watching that little red LOESS line wriggle onward (discounted in the little yellow line), but as smart as Nate is he (and his model) doesn't know where those lines are going next week, let alone in November.

What he's doing mainly is trying to efficiently summarize and update the likely implications of the best currently available information for where each state's voters will end up in November -- and, in some cases (if you've really been paying attention the last week) anticipating what the next state polls are likely to reveal -- or, if there are no such polls, come up with a good estimate of what the polls would show if they were taken. And he's doing it in real time.

Heh, I think it's just "Iowan". And I don't know ...

2008-06-28T23:02:02.965-04:00

Heh, I think it's just "Iowan". And I don't know an Iowan accent, but I can tell Wisconsin from Minnesota. It's all in the long "O". Ask 'em to say "soda", and if the "O" is really, really long, that's Minnesota.

I'm no statistician. And what makes me so pleased...

2008-06-28T22:45:38.263-04:00

I'm no statistician. And what makes me so pleased with the gradual updates to the projection method, is that they come in little increments that I can understand (but could not replicate on my own.) It's kind of wonderful reading each day as the your projection slowly becomes refined. Thanks for all of your hard work.

As to Obama in Iowa, I wonder if his appeal there has to do with his Iowa accent. Iowanian(?) has a specific sound that I can recognize, but is much difference than the accents in other Midwestern states. Obama nails it when speaking to Iowa audiences.

at JGabriel:What you say, that a random walk model...

2008-06-28T22:45:10.580-04:00

at JGabriel:

What you say, that a random walk model predicts nothing, is true. Given that today the value of our variable is 29, our best guess about tomorrow is that it will be the same. If tomorrow it turns out to be 30, our best guess about the next day is that it'll be the same as tomorrow, 30. There's no restoring force, and there's no momentum. The probability distribution of movement on each day is zero mean, and independent of each previous day.

Contrast this to a model with momentum. If yesterday our variable was 29, and today it's 30, our best guess for tomorrow is 31.

The LOESS model that Nate is currently using is a momentum model, at short timescales. (The point of LOESS is that you find a linear or quadratic fit over each "reasonably small" local interval, and require each fit to meet up with its neighbors.) LOESS can do a lot of great things, fitting trends when you have no a priori reason to expect a particular functional form. And I'm fine with a LOESS fit for the great majority of data points that are not at the extreme outer edges of the dataset being fitted.

But in this case, the outermost point, today's point on the trendline, has a huge influence on the projection. Every single poll gets adjusted according to today's point -- the accuracy of today's point is as important as the accuracy of all other points on the trendline combined. If today's point is determined by a momentum model -- such as LOESS -- and a momentum model is inappropriate because public sentiment in fact moves as a random walk, then the projections based on today's momentum-influenced point won't be optimal. They'll likely swing, sometimes high, sometimes low, but not optimal.

I don't actually know if political races are better described by random walk or by momentum. My hunch is that random walk is the better model, but that's just a hunch. It maybe ought to be discernable from the data? Needs some thought....

Anyway. In terms of who's going to win the election it probably makes less than a half points difference. Hell, probably a lot less than a half point's difference. It's a concern for those of us who get our dopamine charge out of finding the best possible mathematical models for the observable world. Nerds unite!

peace,
lilnev

@ Kubla000: You are maybe overshooting a bit. I ge...

2008-06-28T22:21:34.184-04:00

@ Kubla000: You are maybe overshooting a bit. I get your point, and I aggree with it. I as well have been attracted by the demogrphic analysis, and would like to see some more analysis on how the race unfolds among women, "Americans", seniors etc. On the other hand, Nate's "trend adjustment" has really been excellent and spot on, and, in spite of his current technical pre-occupation, he has recently provided interesting insights like, e.g., the fact that Appalchia seems to come to some terms with Obama. Give the guy his time, and the 'benefit of the doubt' - I am sure he will continue to surprise us all!

I'm losing trust in this site. Honestly as I go d...

2008-06-28T21:57:15.796-04:00

I'm losing trust in this site. Honestly as I go down the list of the state by state, I'm seeing that in state after state, your Polling Average is essentially the same as your projection...

PA: 5.5/6.2
OR: 7.1/6.6
OH: 3.5/3.3
NV: 2.8/2
NM: 4.3/4.1
NH: 6.2/6.9
NC: 4.7/4.1
MT: 7.3/7.4
MN: 10.9/10.2
IN: 1.8/2
IA: 6.2/6.3
FL: 1.4/1.9
CO: 3.8/3

Nearly 1% Difference:

MI: 3.2/4.1
MO: 3.5/2.2
GA: 8.6/7.4
CT: 8.9/13.6

Maybe it's because I grew to respect you during the primaries when you saw things polling did not based upon demographic regressions, and since a General Election is Nation Wide all at once, you can't so same/similar to neighbors as much.

Maybe it's because in many states, polling is frequence enough and weighted heavily enough as to render the Regression models less important.

But, what I'm seeing is less interesting every time a change is made. What you were showing days ago seemed to be a prediction, and what I'm looking at today is about as valuable as Karl Rove's electorial maps which are simply based upon the current average of polling from RCP, or Kos/OpenLeft's daily updates based upon the Pollster leader in any given state.

I fail to see how this site is capturing anything unseen by polling anymore. IT seems that prediction powers have been eliminated as enhancement after enhancement is made... only lending polling more power than demographics. That's what was so interesting before, that you could look at trends in populations and see how things would shake out.

Anyhow, not to be a sour puss, but this just isn't nearly as interesting anymore. It seems that you've done a whole terrible amount of work, and at the end of the day, simply mirror what Pollster/Kos/Openleft and Karl Rove all already do...

What seemed at one point to be a break through in predictions beyond polling has now become simply a prediciton based upon polling averages... and that's sad. I've noted the comments are going down, less and less. I think readers are being lost as the novelty wears off and the findings converge with already established information.

Is there a quick lawyman's explanation of "random ...

2008-06-28T21:47:07.055-04:00

Is there a quick lawyman's explanation of "random walk"?

obsessed said..."a nice enhancement would be to ca...

2008-06-28T21:18:01.252-04:00

obsessed said..."a nice enhancement would be to calibrate the y-axis of the Electoral Vote Distribution chart."

Each line represents exactly 10 simulations. I'm basing my answer on the fact that there are 538 possible outcomes on the x-axis (round down to 500 for simplicity sake) and average outcome seems to be around two lines up the y-axis. 10,000 (number of simulations) divided by 500 = 20.

This is the best site on the internet for polling ...

2008-06-28T21:10:16.209-04:00

This is the best site on the internet for polling and election projection analysis. Thanks again Nate for all the hard work and time you've invested to make it so.

Now, get off your ass and give us those Senate projections!

linev: "... is a better assumption that his moveme...

2008-06-28T20:39:34.830-04:00

linev: "... is a better assumption that his movement in the last 3 days is independent of his movement between 6 days ago and 3 days ago? My hunch is that the latter, essentially a random walk with large sampling noise, is the better theoretical model."

Lilnev, could you explain this a little further?

In particular, I'm having trouble understanding how a random walk, even after smoothing, has any more predictive value than just following a flat line until more data becomes available. It would appear, on the surface anyway, to just be randomly noisier.

A trendline, on the other hand, would seem to have at least marginally more predictive value than simply leaving the result flat or randomly noisy until the next data point is added.

I'm probably missing something in your explanation. I get the theory behind it, I think - I'm just puzzled over where the predictive value would come in. Please elaborate a little bit?

.

Re ethanol/corn as pork, it's useful to keep in mi...

2008-06-28T20:17:05.375-04:00

Re ethanol/corn as pork, it's useful to keep in mind that Illinois is a huge corn-producing state, right up there with Iowa, so Obama wasn't just being neighborly toward Iowa when he voted for this.

I think the changes are all for the better and mos...

2008-06-28T20:08:43.932-04:00

I think the changes are all for the better and most of the results seem very reasonable. However, I still think that the model has some trouble projecting believable results in states with very lopsided support. This will not flip any state, but if you're looking for accurate predictions across the board, you should look into it.

Pointing again to Utah, the model currently predicts that Obama will give Democrats their best result in the last 40 years. This might well be possible, but as a projection it doesn't really fit with the rather conservative estimates elsewhere in the model, such as the assumption that Obama's lead in national polling will shrink.

On the opposite side of the spectrum, another example that have been raised in comments is DC, where the projection gives Obama one of the worst results in recent history, significantly below Kerry's.

At the risk of repeating what I've said in a previous comment, I think one explanation could be how the model treats undecideds. As I understand it, the assumption is that they will split evenly between the two major party candidates (with some going to third party candidates).

But with McCain at only 55.0 in the Utah snapshot, do most undecideds in the state actually ponder whether they should vote for the Republican or the Democrat? Probably not. Most of the people in Utah now telling pollsters that they are undecided will either get behind McCain or stay at home on election day. Some might vote for a third party candidate, and a few will chose Obama. But likely not enough to take his numbers to record levels in the state, if his national numbers stay at the current level. With Obama at only 82.8 in the DC snapshot, the exact opposite probably applies there.

Giving some more thought about how this could be taken into account, I would suggest that you start with assigning undecideds according to the candidates' current shares of the two-party vote. This makes more sense than the 50-50 assumption. In essence, you're saying that currently undecided voters will eventually vote like their neighbors, or not at all (both ought to have the same effect on vote shares). A more sophisticated method might be to use some kind of regression to allocate the undecideds.

Although the model is great as it is, and although you've now declared construction season over, I hope that you can see that it still produces some funny results at the edges, and that you're able to come up with some clever tweak to fix that.

It is pork. Well, I'll chip in to send a truckload...

2008-06-28T19:41:30.456-04:00

It is pork.

Well, I'll chip in to send a truckload of carnitas to Iowa. Actually, after reading the Bob Barr thread, maybe we should make that honey baked hams.

Comments on FiveThirtyEight: Politics Done Right: Construction Season Over (Technical)

cheap nike shoes nike sports shoes puma running ...

Truly a nice blog! Thanks for your great work! Wis...

酒店經紀人, 菲梵酒店經紀, 酒店經紀, 禮服酒店上班, 酒店小姐兼職, 便服酒店經紀, ...