FAQ and Statement of Methodology
FiveThirtyEight.com
Revised 8/7/2008
Site/Meta
Who are you? My name is Nate Silver and I live in Chicago. For additional background, please see here or here. The other contributor to this website, Sean Quinn, lives in Washington, DC.
What is the significance of the number 538? 538 is the number of electors in the electoral college.
What is the mission of this website? Most broadly, to accumulate and analyze polling and political data in way that is informed, accurate and attractive. Most narrowly, to give you the best possible objective assessment of the likely outcome of upcoming elections.
How is this site different from other compilations of polls like Real Clear Politics? There are several principal ways that the FiveThityEight methodology differs from other poll compilations:
Firstly, we assign each poll a weighting based on that pollster's historical track record, the poll's sample size, and the recentness of the poll. More reliable polls are weighted more heavily in our averages.
Secondly, we include a regression estimate based on the demographics in each state among our 'polls', which helps to account for outlier polls and to keep the polling in its proper context.
Thirdly, we use an inferential process to compute a rolling trendline that allows us to adjust results in states that have not been polled recently and make them ‘current’.
Fourthly, we simulate the election 10,000 times for each site update in order to provide a probabilistic assessment of electoral outcomes based on a historical analysis of polling data since 1952. The simulation further accounts for the fact that similar states are likely to move together, e.g. future polling movement in states like Michigan and Ohio, or North and South Carolina, is likely to be in the same direction.
How often is the site updated? Generally, the charts, graphs and polling averages on the site are refreshed once per day to reflect any new polls. Sometimes, there might not be any polling on a given day, and so an update will not take place. Other times, volume may be so heavy that multiple updates are necessary.
You can tell that the charts and graphs on the site have been updated any time you see the "Today's Polls" tag in the footer.
Senate polls are updated less frequently: generally once per week, on Mondays.
What is your political affiliation? My state has non-partisan registration, so I am not registered as anything. I vote for Democratic candidates the majority of the time (though by no means always). This year, I have been a supporter of Barack Obama. The other contributor to this website, Sean Quinn, has also been a supporter of Barack Obama.
Are your results biased toward your preferred candidates? I hope not, but that is for you to decide. I have tried to disclose as much about my methodology as possible.
Does this site accept advertising? FiveThirtyEight.com is a commercial site and accepts advertising. Our preferred advertiser is BlogAds. To run an ad at FiveThirtyEight.com, please click here. If you wish to purchase an ad that doesn’t fit into the template provided by BlogAds, you can contact me directly at 538dotcom@gmail.com.
Why do you run ads for [insert name of candidate you don't like]? I believe in the right of free speech. Blogging is one form of free speech, and political advertising is another. If I believe an ad is particularly misleading, I will seek to block it, but otherwise, this site takes a non-partisan position toward which advertising it accepts. Ads for John McCain, Barack Obama and Hillary Clinton have each appeared on this website at various times.
How was the site designed? FiveThirtyEight.com is based on a Blogger.com template. The graphs are designed in MS-EXCEL 2007. I also use a statistical package (STATA) for some of the more complicated number-crunching. Thanks to Robert Gauldin for his design assistance.
The site isn't showing up properly in my browser. FiveThirtyEight.com should render reasonably well in the latest versions of Firefox and Internet Explorer. Older versions of Internet Explorer have pervasive problems with Blogger.com templates and are not recommended.
How do I contact you? Nate can be reached at 538dotcom@gmail.com. Sean can be reached at pocket99s@gmail.com.
Why haven't you responded to my e-mail? Between my various jobs and projects, I receive more e-mail each day than I'm able to respond to in full. However, I read each e-mail and very much appreciate both compliments and constructive criticism. Many of the new ideas and new features on the blog are a direct result of reader feedback. I appreciate your patience. Some e-mails are answered days or even weeks after they are received.
Are you hiring? Not really, but if you think there may be an exceptionally good fit, it never hurts to get in touch.
Are you available to do media appearances? Yes. I enjoy doing media and have done a fair amount of it in the past. If your request is pressing, please include the phrase “MEDIA REQUEST” in the subject heading of your e-mail.
Are you available to do consulting or speaking engagements? Theoretically yes, but practically speaking it will be very difficult in the midst of an Presidential election cycle.
Process Overview
The basic process for computing our Presidential projections consists of six steps:
1. Polling Average: Aggregate polling data, and weight it according to our reliability scores.
2. Trend Adjustment: Adjust the polling data for current trends.
3. Regression: Analyze demographic data in each state by means of regression analysis.
4. Snapshot: Combine the polling data with the regression analysis to produce an electoral snapshot. This is our estimate of what would happen if the election were held today.
5. Projection: Translate the snapshot into a projection of what will happen in November, by allocating out undecided voters and applying a discount to current polling leads based on historical trends.
6. Simulation: Simulate our results 10,000 times based on the results of the projection to account for the uncertainty in our estimates. The end result is a robust probabilistic assessment of what will happen in each state as well as in the nation as a whole.
Step 1. Polls, the Polling Average, and the Reliability Rating.
What is the reliability rating? It is a weight assigned to each poll based on three factors: the pollster's accuracy in predicting recent election outcomes, the poll's sample size, and the recentness of the poll.
How do you determine a pollster's reliability? For a very thorough explanation, see here.
OK, so just who are the most reliable pollsters? Pollsters are rated by their long-term pollster-introduced error (PIE). This is the amount of error that a pollster introduces to its results because of methodological imperfections, rather the inherent limitations associated with limited sample sizes and conducting poll far in advance of the election.
Current pollster ratings can be found here.
How you do assess the reliability of other polling firms not included in the table above? These polls are treated as being slightly-below average and assigned a PIE of +2.11.
Are polls weighted by the number of respondents? Yes, although the methodology is a little involved. For a fuller explanation, see here.
How do you adjust for the recentness of a poll? For Presidential polling, polls are treated as having a half-life of 30 days. Specifically, the weight assigned to each poll is...
0.5^(P/30)
...where 'P' is the number of days transpired since the median date that the poll was in the field.
How did you derive this recentness formula? It is based on an analysis of 2000, 2004, and 2006 state-by-state polling data. Previously, this formula varied based on the number of days until the general election, with the half-life becoming shorter as we got closer to the general election. After further investigation into the data, I discovered that there was really no empirically valid reason for doing this. The 30-day half life did an optimal job, or very close to optimal, across a broad range of time frames, ranging from the evening before the election to 250 days before the election. Note that this is not true for Senate data, for which a different formula is applied.
Well, I still think you're making a mistake by using 'old' polls. The recentness formula is just one of the mechanisms we use to keep the data fresh. All polls are also adjusted based on a trendline adjustment (see Step 2).
What do you do when you have multiple polls from the same polling firm? When a specific polling agency comes out with a new poll, we do not drop their previous poll. Instead, its sample sizes are aggregated for purposes of calculating the weight assigned to the poll, which has the effect of penalizing redundant polling data from the same firm. See the bottom one-third of this post for further discussion.
Are national polls accounted for? Yes, but only insofar as they are used to inform the trendline adjustment. See Step 2.
How do you handle tracking polls? Tracking polls are treated as any other poll, except that the number of respondents is taken to be the number of interviews conducted per day. So a tracking poll that consists of a rolling three-day sample of 900 voters will be counted as a separate data point each day, but as a data point at 300 voters per day.
Does a poll ever become so old that you drop it entirely? Yes. Once a poll's weight falls below 0.05, it is dropped from the model for the sake of simplification and aesthetics. Exception: the highest-rated poll (not necessarily the most recent) in any given state is guaranteed a minimum weight of 0.25. For further discussion, please see here.
How do you find the polls you include in the analysis? I periodically scan the links you see on the left-hand side of the page. If you've come across a poll that is not included in the analysis, please give it a shout-out in the comments in the daily polling thread, and we will get it included in the next update. Occasionally, pollsters also e-mail me their results directly. This is very helpful.
Are there any polls you don't include? All scientifically-conducted polls are included provided that they meet our reporting requirements and the internal poll rule (see below).
What are the reporting requirements for a poll? At a minimum, the poll must list (1) the percentage of the vote for each major candidate -- not simply the margin; (2) the sample size; and (3) the dates that the poll was in the field. We may temporarily list a "BREAKING" poll that is missing some of this information, but if it does not become available promptly, it will be de-listed.
Do you list internal polls that are leaked by the campaigns? This site has a ban on listing internal polls. The logic behind this is that when an interested party conducts a poll, it is only liable to leak its results to the public only if it contains good news for their candidate, thereby encouraging donors, press persons, etc. This does not mean per se that the poll is "biased" -- many pollsters do very good and thorough work on behalf of campaigns and affiliated interest groups. But it does mean that there may be a bias in which information becomes part of the public record: we learn about a poll that has a candidate ahead by 10 points in a state, but not one where he is down by 2.
For this reason, such polls are excluded. More specifically, a poll is excluded if it was conducted by any current candidate for office, a registered campaign committee, a Political Action Committee, or a 527 group, unless (i) the poll has a bipartisan partner (partisan polling groups will sometimes pair with one another to reduce the perception of bias), or (ii) the organization has a long and demonstrable track record of releasing all its data to the public.
Polls are not excluded simply because the pollster has conducted work on behalf of Republican or Democratic candidates, provided that the particular poll in question was intended for public consumption.
What precisely is indicated by the 'date' reported in association with the poll? It will indicate the median date of interviewing for that poll -- not when that poll was reported or posted to the site. For example, a poll which conducted interviews on July 1, July 2 and July 3, and was reported to the media on July 5, would be listed with a date of July 2.
What if a pollster provides multiple versions of their poll -- e.g. with or without third party candidates included, or different versions for registered and likely voters? When these situations arise:
(i) I will use the registered voter version until the first Presidential debate. After that, I will use the likely voter version;
(ii) I use the version with third-party candidates included if (i) they have officially announced their candidacy, and (ii) they are on the ballot in that state.
(iii) If a pollster lists separate results with and without ‘leaners’ (people who are initially uncommitted but pick a candidate after prompting), I use the version with leaners.
Step 2. The Trendline Adjustment.
What is the purpose of the trendline adjustment? Polling data comes out in different increments in different states. Some states are polled frequently, while others are only polled only occasionally. The trendline adjustment is an effort to correct for this problem by using polling movement in states that have been polled recently to adjust the data in states that have not been.
In addition, the trendline adjustment can account for what I refer to as ‘timing bias’. If a particular state is polled in the midst of a bounced cause by something like the conventions, such pollig may reflect only a temporary, near-term fluctuation rather than the longer-term demographic reality.
To take a more concrete example, suppose that Virginia was last polled in the weekend prior to the Democratic convention, and that poll showed John McCain ahead by 2 points. Suppose also that North Carolina was last polled in the weekend following the Democratic convention, and that poll showed Barack Obama ahead by 4 points. Looking at these two polls might give the impression that North Carolina is a better state for Barack Obama than Virginia. But depending on the size of Obama’s convention bounce, this could entirely be an artifact of when the respective polls were conducted. The trendline adjustment attempts to correct for this.
How does the trendline adjustment work? In plain English, we look at movement in the polling in recently-polled states and in national polls to predict movement in other states. For example, if there are new polls conducted in Massachusetts and Connecticut showing the Democratic candidate gaining 5 points, we can probably also infer that the candidate’s numbers have improved by about 5 points in Rhode Island.
For the original methodology behind the trendline adjustment, please see here. For subsequent refinements to the methodology, please see here, here, here and here.
Does the trendline adjustment assume that polling movement is uniform between different states? No, it does not. The adjustment attempts to account for which particular demographic groups are responsible for the polling movement, and those groups may produce differing results in different states. See here and here for discussion.
Does the trendline adjustment account for the convention bounce? We will have a special set of procedures in place on and around the time of the conventions to account for the convention bounce, but they have not yet been fully developed.
Are the polls weighted for purposes of calculating the trendline adjustment? Yes. More reliable polls have more influence in the computation of the trendline.
Step 3. The 538 Regression Estimate
What is the regression estimate? It is an analysis of what the polling data “should” be in each state based on its underlying demographics. Put differently, it is a way not to be held hostage by the results of individual polls that might defy common sense, particularly where polling data in a state is sparse.
Polls are an imperfect measure of voter sentiment, subject to the vagaries of small sample size, poor methodology, and transient blips and trends in the numbers. For example, the late February SurveyUSA polls had Barack Obama four points ahead of John McCain in North Dakota, but behind by four points in South Dakota. Since North Dakota and South Dakota are very similar, it is unlikely that there is a true eight-point differential in the polling in these states. The regression estimate is able to sniff out such discrepancies.
For general background on the process of regression analysis, see here.
What is the dependent variable in the regression analysis? Technically speaking, there are two regressions that are computed in each state. The first regression is a regression on the share of the two-way (Democrat + Republican) vote held by the Democratic candidate in that state based on our current polling averages after adjustment for present trendlines. The second is a regression on the total committed vote held by either of the major-party candidates.
What independent variables are included in the regression estimate? The regression models evaluate a total of 16 candidate variables. Variables are dropped via a stepwise process, until such time as each remaining variable is statistically significant at the 85% level or higher.
The 16 variables presently considered by the model are as follows:
Political
1. Kerry. John Kerry's vote share in 2004. Note that an adjustment is made in Massachusetts and Texas, the home states of Kerry and George W. Bush respectively, based on Al Gore's results in Massachusetts in 2000, and Bob Dole's results in Texas in 1996.
2. Fundraising Share. The total share of funds raised in that state by each candidate (expressed specifically as the percentage of all funds raised that were raised by the Democratic candidate).
3. Clinton. The percentage of the two-way (Obama + Clinton) Democratic primary vote received by Hillary Clinton in that state. An adjustment is made to caucus states to account for their higher proclivity to vote for Barack Obama. In Michigan, the variable is based on the results of exit polling, which indicated who voters would have selected if all candidates were on the ballot.
4. Liberal-Conservative (Likert) Score. Per 2004 exit polls, a state’s liberal-conservative orientation, wherein each liberal voter is given a score of 10, each moderate a score of 5, and each conservative a score of 0. The most liberal state, Massachusetts, has a Likert score of 5.65. The most conservative, Utah, has a score of 3.30.
Religious Identity
5. Evangelical. The proportion of white evangelical protestants in each state.
6. Catholic. The proportion of Catholics in each state.
7. Mormon. The proportion of LDS voters in each state.
Ethnic and Racial Identity
8. African-American. The proportion of African-Americans in each state.
9. Hispanic. The number of Latino voters in each state as a proportion of overall voter turnout in 2004, as estimated by the Census Bureau. The reason I use data based on turnout rather than data based on the underlying population of Latinos is because Latino registration and turnout varies significantly from state to state. It is much higher in New Mexico, for instance, which has many Hispanics who have been in the country for generations, than it is in Nevada, where many Hispanics are new migrants and are not yet registered.
10. "American". The proportion of residents who report their ancestry as "American" in each state, which tends to be highest in the Appalachians. See discussion here.
Economic
11. PCI. Per capita income in each state.
12. Manufacturing. The proportion of jobs in each state that are in the manufacturing sector.
Demographic
13. Senior. The proportion of the white population aged 65 or older in each state. Because life expectancy varies significantly among different ethnic groups, this version has more explanatory significance than when looking at the entire (white and non-white) population.
14. Twenty. The proportion of residents aged 18-29 in each state, as a fraction of the overall adult population..
15. Education. Average number of years of schooling completed for adults aged 25 and older in each state.
16. Suburban. The proportion of voters in each state that live in suburban environments, per 2004 exit polls.
How often is the regression updated? The regression updates automatically based on the latest polling data. Periodically, I will also test out new variables for potential inclusion in the model.
Step 4. The Snapshot.
What is the snapshot? It is (i) the combination of the trend-adjusted polling average (Step 2) with our regression estimate (Step 3). This represents our best estimate of what would happen if the election were held today.
How much weight is given to the regression estimate? The regression results are treated as a single, recent poll of average reliability (see here for how I define 'average' in this context). Therefore, the regression estimate will have comparatively substantial weight in states with little polling data, but very little weight in states with robust polling data.
Step 5. The Projection.
What is the projection? It is our best estimate of what will happen when the election is actually held in November.
How does the projection differ from the snapshot? It differs in two important ways. Firstly, current polling leads are mean-reverted. Secondly, undecided voters are allocated to the two major-party candidates.
How does the mean-reversion adjustment work? There has been an extremely robust tendency in Presidential elections for national polling numbers to revert to the mean as the election approaches – that is, for the trailing candidate to gain ground. The further we are out from the election, the more tightening in the polls we can expect. For example, a 20-point national lead held 200 days before the election projects, on average, to only about an 8-pont victory on Election Day, whereas a 5-point lead held 60 days before the election projects to about a 4-point victory. This adjustment is described in much more detail here.
Is the mean-reversion adjustment applied uniformly across all states? No. The mean-reversion adjustment is based on the notion that national polling data will tighten as the election nears. This does not necessarily imply that polling in any particular state will tighten. Instead, we first calculate the overall degree of mean-reversion expected in the national popular vote, and then imprint it on individual states through the process described here. States that have been more sensitive to movement in the national numbers will receive a larger degree of mean-reversion.
How are undecided voters allocated? This process may seem to work slightly backward. Firstly, we determine how much of the vote is likely to go to third-party candidates in each state based on a regression of the current undecided and ‘other’ vote in each state against historical trends. Then, having created an allocation for third-party candidates, we allocate the remaining undecided vote 50:50 between the major-party candidates.
Are you sure that allocating the undecided vote 50:50 is the best approach? I am fairly certain that the most obvious alternative – allocating the undecided vote based on each candidate’s proportion of the vote in each state – is not superior to this approach when evaluating presidential election data. Such an approach would imply that most of the undecided voters should be given to the leading candidate, but under certain circumstances – such as when there are a high number of undecideds a long way before the election – there is some tendency for undecided voters to break for the trailing candidate.
Step 6. Simulations and Win Probabilities
What is Win % or Win Probability? Simply, the number of times that a candidate wins a given state, or wins the general election, based on 10,000 daily simulation runs.
How is Win Probability determined? By simulating the election 10,000 times each day by means of a Monte Carlo analysis, based on the current Projection in each state. The simulation accounts for the following properties:
(i) That the true margin of error of a poll is much higher than the sampling error, especially when the poll is taken long before the election.
(ii) That polling movement between different states tends to be correlated based on the demographics in those states.
What is the purpose of the simulation runs? To account for three types of uncertainty in interpreting polling data: sampling error, state-specific movement, and national movement. Please see my discussion here.
The most important concept is that the error in predicting electoral outcomes is much larger than would be implied by the margins of errors from the polls alone, especially early in the election cycle. That is, the election may 'break' in any number of different and unpredictable directions, both at a state-by-state and at a national level.
As we get closer to November 4, the potentiality for these trends will become lesser, and therefore the error assumed by the simulation will become progressively less. However, even on election eve, the errors in predicting electoral outcomes are larger than those implied by each pollster's reported margin of error calculation. Combining different polls together may reduce the problem, but it will not eliminate it, as polling errors tend to be correlated (that is, many pollsters miss in the same direction).
How reliable are polls conducted X days before the election? If ‘X’ is a number larger than about 30, the answer is ‘not very reliable’. Many voters do not begin paying attention the campaign until mere days or weeks before election day. As such, polling conducted before this period is tenuous. The specific amount of variance we apply to each state is determined based on an analysis of historical polling data since 1952 and is described here.
Is the polling in some states more volatile than in others? There is good reason to think that it is. Some states contain more true swing voters than other states. For instance, in 2008, the amount of volatility in the polling data in a given state has been positively correlated with the number of independent voters in that state, but inversely correlated with the number of African-American voters. Our process accounts for these tendencies, as described here.
What is the interrelationship between polling movement in different states? In reality, there is no such thing as national polling movement. Instead, you have millions of individual voters making up their minds in 50 individual states and the District of Columbia.
Like-minded voters, however, can be presumed to change their candidate preferences in similar ways. For instance, relative to national trends, election results in Massachusetts have historically been 90 percent correlated with election results in Rhode Island.
Our simulation accounts for this tendency by applying a similarity matrix, which evaluates the demographic relationships between different states by of a nearest-neighbor analysis as described here. Our process recognizes, for instance, that as the polling in Ohio moves, the polling in a similar state like Michigan is liable to move in the same direction. On the other hand, there may be little relationship between the polling movement in Ohio and that in a dissimilar state like New Mexico.
In our simulation runs, the state-by-state polling movement is architected so as to preserve (i) the interstate correlations described above; (ii) the historical relationship between the degree of national polling movement and that in different states – the more the polls move in the aggregate, the more volatile the polling movement in different states, and (iii) the empirical degree of volatility in the polling numbers within any one particular state (see question above).
Is there an empirical basis for this adjustment? Not as much of one as I’d like. State-by-state polling data is hard to come by in years before 2000, and the 2000 and 2004 cycles may not be representative as they were unusually stable elections. Therefore, there is a little bit of guesswork involved in calibrating the model and determining the appropriate amount of interdependence in polling movement between different states. But I am convinced that we have a substantially better model with this adjustment than without it.
What is the margin of error in the simulation runs? In terms of predicting the winner of the national electoral vote, there appears to be margin of error of somewhere around +/- 1 percentage point over our 10,000 daily simulation runs.
Charts and Graphs
National Summary Chart
How can there be fractional numbers in the electoral vote counts? For example, Obama winning 293.4 electoral votes? We are not predicting any one particular outcome in the election – Obama winning states A, B, and C, and McCain states X, Y, and Z. Rather, we are predicting a probability distribution – the relative likelihood of different outcomes occurring. The electoral vote counts represent an average of thousands of individual simulations, and the average may produce a fractional number of electoral votes.
State Summary Chart
What do the percentages mean next to each individual state? They are our estimate of the chances that Barack Obama and John McCain will win that state, respectively.
What is the significance of the color of the state (red-blue-purple) in the state summary chart? They reflect the results from that state in 2004. States are rendered in purple if the Bush-Kerry margin in those states was within 7.5 points.
What is the significance of the 'regions' as defined on the state-by-state summary charts? There isn't any, other than as a way to present and organize the data. For additional discussion, see here.
Electoral Projection Map
How many colors are used in the electoral projection map? There is no specific limit. Rather, states are colored on a red-white-blue gradient based on the current win percentage in each state.
Electoral Vote Distribution
What do the individual spikes / data points represent? The number of simulations, out of 10,000, in which Barack Obama finishes with some precise number of electoral votes (such as 290). Simulations that result in a McCain electoral win are colored red, and an Obama win colored blue.
Is the distribution normal (e.g. a bell curve?) Not necessarily. Because the polling movement between different states is assumed to be correlated, the distribution can take on a variety of different shapes, with multiple peaks and so on. The distribution will also clearly not be normal in the event that one candidate is headed for a landslide, as there is an upper bound in his number of electoral votes.
Super Tracker
What do the individual, blue data points represent in the Super Tracker chart? They represent the inferred popular vote outcome based on all polling (state and national) conducted on that particular day, as determined by analyzing the degree of movement between previous iterations of that poll. This is not the same as simply averaging the polls, although the Super Tracker usually resembles the Pollster.com and RealClearPolitics.com national averages closely. While the individual data points can be interesting to look at, we advise against overinterpreting them – there is a lot of noise in any one particular day’s data. Instead, the red trendline curve represents our best estimate of the current state of the election. For further background on the Super Tracker chart, please see Step 2 above.
Tipping Point States and Return on Investment Index
What are Tipping Point States? A Tipping Point State is defined as a state that would alter the outcome of a close election if it were decided differently. For a thorough discussion, see here.
What is the Return on Investment Index? The ratio of a state’s Tipping Point percentage to the number of eligible voters in each state, calibrated such that an average state has a Return on Investment Index of 1.0. This is intended to represent the marginal return from spending one additional dollar (or other type of campaign resource) in that state. For further discussion, see here.
Poll Detail
How are states classified as ‘Lean’, ‘Likely’ and ‘Safe’? States are classified as follows, based on the Win Probability of the Democratic candidate in each state:
Win % Classification
0%-5% Safe GOP
5%-20% Likely GOP
20%-40% Lean GOP
40%-60% Toss-Up
60%-80% Lean Democrat
80%-95% Likely Democrat
95%-100% Safe Democrat
What does it mean when a polling result is highlighted in yellow? It means that the poll was conducted within the past 10 days.
Senate Polls
Do you assume that senate races move independently of one another? Or is the movement correlated, as in the presidential simulations? We assume a small amount of national (correlated) movement in senate races, as determined from an analysis of historical trends in the Generic Congressional Ballot. However, relative to the Presidential contest, the movement of individual senate races are relatively independent from one another.
What variables are included in the regression analysis for senate races? Five variables are included:
(1) A dummy variable to indicate the presence of an incumbent in the race;
(2) Where there is an incumbent, the approval ratings for that incumbent;
(3) The share of fundraising obtained by each candidate;
(4) The highest elected office held by each candidate, expressed on a proportional basis;
(5) The partisan ID index of each state (the number of self-identified Democrats less the number of self-identified Republicans), based on 2004 exit polling data.
For a more complete discussion, see here.
Is senate polling less reliable than presidential polling? This is debatable. However, senate races tend to break later than presidential races. Therefore, the degree of uncertainty tends to be higher at a given date before the election; a 10-point lead in the presidential polls in a state tends to be more meaningful than a 10-point lead in the senate polls. The win percentages for senate races are determined based on a historical analysis of senate race data and senate race data only, and apply different parameters than are used in the presidential estimates.
What other methodological differences are there between the senate numbers and the presidential numbers? There are several differences:
(1) For senate races, the half-life assigned to each poll shortens as we get closer to the election. That is, we place progressively more of a premium on the recentness of a poll as we near the presidential election. This is not true for our presidential numbers.
(2) However, there is no timeline adjustment applied to senate races, as there is in the presidential contest.
(3) In senate races, our allocation of undecided voters depends in part on the number of undecided voters. Higher numbers of undecideds indicate more uncertainty in the race and a greater likelihood of these undecideds breaking to the trailing candidate (usually the incumbent). We do not directly evaluate the number of undecided voters in our presidential polling.
Why do you count Joe Lieberman as a Democrat? Good question.
Miscellaneous Wonkery
How are ties broken? Ties (269 electors for both the Republican and Democratic candidates) are assigned to the Democrat based on the assumption that the Democrat would likely carry the day in the incoming House of Representatives. For additional discussion, see here.
Do you account for the potential for split electoral votes in Nebraska and Maine? Nebraska and Maine assign some of their electors based on the election results in individual congressional districts. The win probability and electoral vote averages do in fact account for these contingencies. This is somewhat relevant in this election, as Barack Obama looks to be competitive in both NE-1 and NE-2, while he will probably lose NE-3 (Western Nebraska) badly.
Do you account for home state effects, like in Arizona and Illinois? Directly, no, but indirectly yes. There is a very strong relationship between a candidate's home state and the amount of fundraising that they've received from that state. Since fundraising is one of the variables in our regression model, these effects will in turn show up in our weighted average for that state.
What if any assumptions do you make about turnout? I don't make any assumptions about turnout. The pollsters make various sorts of assumptions about turnout, and I rely on the pollsters. The only exception is in calculating the popular vote percentage shares for each candidate. For this purpose, I assume that the same proportion of the electorate will turn out in each state as turned out in 2004. However, the turnout figures are adjusted based on changes in the eligible voter population in each state since 2004.
So is this your prediction about what will happen in the election? Not necessarily. The goal of the model is to do absolutely as much as it can with current state-by-state polling data. That is not exactly the same thing as accounting for external contingencies that might move the polling data (and, more importantly, the actual election result) in the future.
Do you have any plans to introduce polling averages for House and Governor’s races? Unlikely in this cycle, but almost certainly in 2010.
What will you do after the election is over? Sleep. But FiveThirtyEight will continue to exist. There is alll sorts of political data to sort through even when an election is not going on, particularly as it concerns the legislative process. We hope to continue presenting this data to you in new and exciting ways.
87 comments
3) In senate races, our allocation of undecided voters depends in part on the number of undecided voters. Higher numbers of undecideds indicate more uncertainty in the race and a greater likelihood of these undecideds breaking to the trailing candidate (usually the incumbent).
Is this a typo? I would think the challenger would usually be trailing, but more likely to pick up undecideds....
Thank you for the FAQ update! Much improved.
FYI, Sam Wang at Princeton University has a similar (but different) electoral vote projection. It also includes a blog that allows users to submit comments.
Mr. Wang had one blog post criticizing 538 for having a predicted electoral vote distribution that was not Gaussian. I posted a comment pointing out that you only get a Gaussian distribution if you assume independence between the state outcomes, and 538 includes correlations which breaks that assumption.
In response to another comment from Mr. Wang, I posted a comment further explaining how not including state-to-state correlations can give you a incorrect level confidence in your results. (Mr. Wang's results do not include correlations, and for that reason, show a level of certainty higher than most people would intuitively believe.)
It turns out however that Mr. Wang moderates the comments on his blog. He deleted the comments I made, and also deleted an earlier comment I made pointing out that the number of electoral votes on one of his maps did not add up to 538.
Because he is moderating thoughtful comments that point out potential problems in his assumptions or methodology, I believe both his site and his blog should be ignored.
Hey, now you can answer my question in comments, if you don't get to my mail. :-)
For the electoral vote pie-chart you say you use an average over the sim runs. OK, but wouldn't the median value be more meaningful? At least that would represent an actual result. (Also interesting would be a 25%-tile and 75%-tile values, or some spread that would indicate how volatile the results are.)
In any case, I'd like to see some sort of measure of volatility (standard dev if you stay with mean electoral votes).
Nate,
Can you please explain why some states are colored RED while others are BLUE and some are PURPLE.
There doesn't seem to be an obvious correlation based on win percentage probability. For example, VA is RED (Obama has 46% chance of winning there) while FL is purple (Obama only has 27% chance to win there. Shouldn't VA be PURPLE and FL be RED?
How do you deal with the underreporting of racist sentiments in surveys? I suspect this may create an overestimation of Barak Obama's numbers.
Thanks!
Sometimes, i wonder what time you really posted messages, and poll updates... is everything based on eastern time? could you put an ET next to the times, if that's the case?
This is the first primary in which I voted. I would like to see more discussion on voter turnout. Given the large voter turnout for the 2008 primaries, if the numbers had a strong correlation(?) to November election day numbers, the trend would be dramatically affected suggesting a landslide democratic victory.
Selected Primary Turnouts
(based on CNN)
...... FL....NC....OH....MO....TX
Dem 1.7M 1.6M 2.3M 8.1M 2.8M
Rep 1.9M 0.5M 1.0M 4.1M 1.3M
I would be very interested in your comments, Nate and Sean.
How does your methodology factor in 3rd parties? Today's popular vote numbers show only 2.1% not voting for the Big 2 and from memory, that seems quite low vs. other times I've seen these numbers from you. Have you looked at historical results on third party voting levels and factored that into your model as a cross-check, so that various lazy polls, that don't do a good job here, don't create results in your formulas that lead to very unlikely results? Given that the race could be very close, an reasonably accurate tally of third party voting affects seems to be warranted.
I agree with Steve above--could you also post the median EVs? It seems that when Obama wins, he wins big, whereas McCain wins by smaller margins. This puts the mean EVs higher--which is useful information, but giving the median EVs (and a 25th and 75th percentile, like Steve suggests)--would be super helpful in interpreting your results. Otherwise, this is such a fantastic resource.
How is it possible for your simulations to come up with the Democrat winning the electoral vote (270 - 268), but the Win Percentage showing the Republican winning with 52% ?
The Republicans have taught us how to win an election even after losing the popular vote, but can even they really win an election after losing the electoral vote?
Nevermind my question about electoral votes vs. Win Percentage. A friend noted that the distribution is not normal, and it all clicked. Sorry for the noise.
Nate, How sensitive are your results to the assumption that undecided respondents will split evenly between Obama and McCain (which, as you say, favors the trailing candidate in each state)? I'd like to see the results of simulations rerun after assigning less than half (maybe 1/3rd) to Obama, as this is a well-known scenario based plausibly on racism. Better if this were a permanent "interactive" feature of your site (i.e., letting us choose 1/2 vs 1/3 for example).
It took a while and a lot of puzzling over what republican-related events are associated with senate-controlling seat counts 51 and 60 in your bar chart, before I realized they simply represented majority and super-majority. Since red carries so much meaning everywhere else, would it make sense to color those bars something else, like green?
Regarding the cell phone data: I understand statistical analysis about as well as I electronics. I have no idea how it works, but I trust that it does. So that said, forgive me if my question is stupid. But is it fair to simply add the 2.2 points across the board, or should you weight state by state for cell phone demographics?
BTW a simple yes or no is sufficient. If you try to explain why I won't understand it. I trust you. :-)
My question is if polls are "scientific" then why are so many of them so wrong and a site like 538.com is needed to sort out the mess? Seems to me, that polls can be influenced by any number of factors not least of which the political affliation of those being polled and how each poll is weighted for a particular political party's expected voter turnout. If polls were truly scientific, then there really wouldn't be so much disagreement and so many of them wouldn't be so wrong so many times.
What's your take on the guilty racist, who says he'll vote for the black guy but doesn't? I've heard that Obama needs an extra 7% polling lead to account for this.
There are sometimes retroactive changes in the Supertracker trendline. I can see how later events can let you interpret an upward tick as either a momentary spike or as the beginning of a larger trend, and I can guess how you might be fitting the curve to the data points (though a technical account of the algorithm would be appreciated). BUT - this type of thing is inconsistent with your description of the supertracker trendline as representing, on any given day, your best estimate of where the election is at--because your best estimate of last week's situation with hinsight is not the same as your best estimate of the situation as it occurred. Anyway a little description of how you fit the trendline to the datapoints and what kind of situation can result in historical revision of the trenline evolution would be greatly appreciated. Thanks for the great resource.
Do you have a method to estimate the effect of an October surprise, such as a large terrorist attack in the U.S., or Israel bombing Iran, or V.P. Cheney resigning/dieing and McCain being nominated to replace him?
awesome site; thanks for all your work.
as part of a FAQ however:
how long have you been running your predictions?
what is your record so far?
(i gather the predictions are self-tweaking, but that just means that we should expect the 'record' to improve with time)
i do like a couple of the previous comments also, that suggest that you should include race/gender and primary turnout in your regression (if you don't already)
I know you're a poker player, so maybe you don't want to tip your hand yet... but have you considered piecing the daily pie charts, maps, and simulation distributions together into a time-lapse YouTube to do some post-election visualizations?
Any notion of packaging the data into a Google Spreadsheet for post-mortem analysis?
People much smarter than me have written extensively about the problems with stepwise regression (results that are not replicable, overfitted models, lousy estimates, etc.). I'd suggest for next time you reconsider your regression methodology. Remember that there's no "correct" model, simply useful ones. You seem to know this area pretty well so you probably know what a good model should be.
Nate (or anyone else still paying attention to these comments):
Looking at the Electoral Vote Distribution Graph, it would seem that, for instance, in yesterday's run, every integer of Obama EV's between about 180 and 480 occurred in at least one simulation. And between around 240 and 400 every integer occurs at least 20-25 times! Though the range varies, this seems to have been a constant over the whole life of the graph (hundreds of runs by now, encompassing millions of simulations).
That seems, mathematically, extremely unlikely. Is it true? Or am I taking the graph too literally, or misinterpreting an image-scale issue?
How accurate is this site as a predictor? How many states were correctly predicted in 2004 as of the last update before the election and how close was 538 to the actual margins of victory?
Adam, the site didn't exist in 2004. Nate just started it this year.
Now that McCain is trying to win 1 EV in Maine, I would be interested in seeing the odds of the scenario where he wins this vote, and by doing so, gets exactly 270 EVs (e.g. Obama takes NV, but not CO, IN, VA, OH, etc).
While I really hope this doesn't happen, this is really the only reason McCain would be spending any time/money at all in Maine.
Assuming that 2004 or 2000 polling data is still available, you could in some sense validate your methodology by running it against those older polls. If you have tried something like that, what kind of results did you get?
Or perhaps that is how you developed your regression model?
Nate, Thank you! I love the site and have been glued to it since I first saw you on one of the late night talk shows a few weeks ago (my mind is mush, I can't remember which one).
I have searched for, but haven't found, any previous requests for a sequential animation of the Projection Map tied to the Super Tracker Graph. If this is easily done, it would be really neat to see changes in the map as a function of time. I often find myself looking at the map and thinking, "now what did that map look like yesterday?"
If this is something that you don't have the time for or the facility to do, then thanks anyway and please keep up the great work on the site. Maybe someone reading this, with an idea and the time to get it done will volunteer.
I'm trying to match 538's Win % Monte Carlo simulations with my own, but cant figure out how the site estimates standard deviation from MOE.
I'm modeling the vote shares of each candidate as normally distributed stochastic variables with the means equal to the projected shares and the standard deviation equal to some fraction of the MOE, but I find that to match the site's Win %, this fraction varies from state to state.
Any ideas?!
Hi there. Site is impressive.
Wondering if/how you take into account any of this
http://www.truthout.org/article/rfk-jr-and-mike-papantonio-is-your-vote-safe
This site is visual crack!
I'm more addicted to this than Porn! ;)
Hi,
Great site, especially for someone who was a former statistical wonk. It would have saved me a bunch of time if you could have a link that would give a simple listing of the meaning of the various colors, indexes, etc. For example: "Polls in yellow were conducted within the past ten days." Etc. etc. If there is actually such a link, maybe it needs to be more prominent. Anyway, thanks for all the information!
Hi, just found the site after your appearance on Colbert last night. When I heard your methods could be "sabremetric" I figured there was likely good statistics involved.
My question reiterates Ann's question of 9/25 - how, if at all, does your analysis account for what she called the 'guilty racist' or has been referred to as the 'Bradley Effect' (Tom Bradley leading big in polls but losing the election). How is the lying and/or racism of pollsters taken into account - as latent racism would not have been a factor in Kerry-Bush or Gore-Bush.
Allen B (added the B since I'm not the same as the other 'Allen' who posted earlier)
in the "Obama vs McCain Projection" map why are there 3 squares in Nebraska and 2 in Maine? I have never seen this before.
dark -
Nebraska and Maine allocate electors by congressional district. So in theory those votes could be split. I think it's rare that it happens, but Obama is making a play for a mostly urban district in Nebraska this year.
I am concerned about the effect of electronic voting corruption as evidenced by what seemed to be at times the poll-defying results of the last two elections, among numerous other points of evidence. Have you applied your model to the previous two elections in order to validate the prediction implied by the 2008 data? I suspect (though I do not know) the vote fraud effect is statistically significant, at least over the 538 electoral votes. I would also suspect this vote fraud factor is a rising factor. Anyway I think Sam Wang's poll analysis in 2004 predicted a 311/227 split in favor of Kerry and we all know how that turned out. That reflects a 120 EV swing given a decent analysis of polls. Of course other estimators using polls were closer to the final official tally.
I know the FAQ says this but the pie chart with the blue Pac Man eating the remaining red Pac Man isn't titled so I just want to make sure. Nate is projecting that based on his analysis that Mccain has a 5.9% chance of winning on Nov 4, correct? It's not winning a hypothetical election if it were held today?
Two questions: 1) How good is the fit in the regression model? I'm wondering how much the differences among states are accounted for by demographics.
2)I gather that no weight is given to the factors that drive most forecasting models--e.g., the president's approval rating, the condition of the economy, the generic popularity of the congressional parties. Most of the effects of these factors should be incorporated in polls by now; but I suspect not all of them. If this is correct, we should expect less regression to the mean than normal, as Obama should still benefit from these underlying advantages. Is all this right?
mc9cain:
Yes that's correct and his post today comments on it.
What is the meaning of the pinkish color for the PA Morning Call poll? What does the date T5 mean?
Any chance of seeing what your methodology would have predicted on a given day in 2004? Using the same formulas to predict the outcome, how would they have performed one week from the 2004 election, two weeks, and three weeks.
Electoral-vote.com has the "This day in 2004" link and if you change the url on the IEM pages you can get the 2004 stats.
I just think it would be interesting to see.
Bob: This is stupendously complex analysis and he wasn't doing it in 2004.
PJQ49: Right, but Bob was asking whether these tools could be applied to 2004 data, to test the resulting projections against actual (known) outcomes, so we can have a sense of their reliability.
Is there any way to see a graph of the history of the "Electoral Vote" and "Win Percentage" projections that show up as daily pie charts? If there isn't, I suggest that it be added.
The Super Tracker is nice, and I think pretty much does the same thing for the "Popular Vote" pie chart, but it's very hard to see the trends in the Win Percentage and Electoral Vote projections over time without a historical graph.
For example, if I want to know when was the most recent date that the Win Percentage for McCain went over 50% (if ever), and how long it stayed there, how do I do that?
@pjq49 - Ah, didn't know he wasn't doing this in 2004; that does make a big difference.
I do realize that this is an incredible amount of data to parse, which was why was suggest just a once a week snapshot if the data was there. But it isn't, so no worries. :)
I guess it ultimately comes down to wanting some assurance that the analysis is correct because I want Obama to win. In 2004, polls switched back and forth quite quickly leading up to the actual election. If you look at electoral-vote.com from 2004, you can see that in the days right before the election, the lead flipped between Kerry and Bush several times. I want some reassurance that that won't happen this time with 2 1/2 weeks left.
Also, if you look at this from a genetic algorithm perspective, what is often done is to take half of a data set, let your programs work with that and find the best predictor, then set it loose on the whole data set. In 2012, he'll be able to do that. :)
How do you determine the MoE?
Just curious.
Is there any way to determine what the tipping point is, in terms of 270 EVs, assuming a uniform national shift in the polls? If Obama support dropped 5%, is he still ahead?
"What is the significance of the color of the state (red-blue-purple) in the state summary chart? They reflect the results from that state in 2004. States are rendered in purple if the Bush-Kerry margin in those states was within 7.5 points."
No, they appear to reflect the current projection. For example, you've got both VA and IA marked blue.
In the state polling column, in Pennsylvania, one row is salmon-colored. What does that mean? I'm sure you explain it somewhere, but I can't find it. Maybe next year you can put a "Search" function in? (I take that back -- this site is superb as it is.) Thank you for doing all this work for your loyal fans.
Bonny
Late to your party, but i like your site - think it was Nate i saw on TV.
In reading your definition of people who define themselves as American, rather than a hyphen(?) - not all of us are Applachian rednecks. My wife (Mexican ancestry) and me (irish ancestry) do not use hyphens and report ourselves as Americans. How many generations born in the USA before American becomes your ancestry and whats wrong with American being my ancestry - I don't relate to any other country? (I know you're just looking at data.)
We are Americans pure and simple - born here, proud to be American, allegiances to the USA, registered independents (i voted both ways in past 30 years, wife mostly democrat), and both of us will vote Obama tomorrow.
Last!
Nate - you guys are doing great work here. Been following for a while now. Have to say, I love that the CA Prop 8 graphic looks like a Jesus Fish.
Have you or could you attempt to account for the effects of the Help America Vote Act (HAVA) which, apparently, enables any number of voters or their ballots to be rejected. Estimates are that millions of potential votes could be lost in the general election due to partisan malfeasance. All of these analyses may be for naught given these realities.
I appreciate the effort you've been putting into this, and the fact that you are reasonably open about your support of Barrack Obama. But the second graphic I checked upon arriving on your site was the Super Tracker. Which presents McCain below the origin. In red.
For all of time, below the origin has been the place for "negative," and "up-and-to-the-right" has meant "good." And red ink, when used vs. black ink, as you do in the Super Tracker, has meant "negative."
That's a clearly partisan way to present tracking data.
When you present the data in so obviously a partisan manner, why should anyone take your efforts seriously? It makes it far too easy to dismiss the work you're doing.
Why is there a McCain/Palin contribution link on your header?
Question: do you calculate/consider turnout to be a % of registered voters or of eligible (citizens age 18 and older, less felons not eligible to vote)?
davea writes, “the Super Tracker. Which presents McCain below the origin. In red. . . . . That's a clearly partisan way to present tracking data.”
This complaint would make more sense if there were any clearly NONpartisan way to present this data, i.e., one whose graphic arrangement put neither candidate in a subtly negative position. For a horizontal graph of this type, one candidate or the other _must_ be below the horizontal axis. Rotating the graph wouldn’t help, because “left” is as culturally disadvantaged as “negative.” Is there any way of graphically displaying this data without placing one candidate in an inferior (lower) or sinister (left) position? I can’t think of one offhand. Davea doesn’t suggest one. If there is no way out, then the best one can do is probably to alternate the subtle disadvantage: i.e., next time around, display the Democrat below the horizontal axis. It is interesting to consider what easily-legible graphic scheme might represent this kind of data if three or more candidates were contending seriously for the office (someday, maybe! -- we can only hope).
As for the color complaint, Red for Republican is as established in the visual code of US politics as Red for Communist is in China, and I think any complaint that it acts here as code for warning, danger, poison, or other negatives is pretty weak. For consistency, however, it might be as well to make Obama’s Super Tracker axis labels blue.
Davea’s complaint is not completely void, but if anybody would simply turn their back on these painstakingly assembled numbers because of such concerns their brain is probably too partisan to function properly no matter what.
Please excuse my ignorance, but can someone be so kind as to explain the statistics regarding Ohio:
McCain loses but wins election: .04%
Obama loses but wins election: 73.97%
Give me the stastics primer, or point me to a brief one: shouldn't Obama's number be higher?
Thanks!
mamab - look at the numbers to the right of the percentages you cite. 73.97 (1765 of 2386) and.04 (3 of 7614)
This indicates that Obama loses OH 2386/10000 times (24%) and McCain loses OH 7614/10000 (76%). This is based on the 10000 Monte Carlo simulation described.
The 73.97% and the 0.04% are based on additional Monte Carlo simulations indicating what happens beyond just OH. So, of the 7614 times McCain loses OH, he only wins the election 3 of those times. Of the 2386 times Obama loses OH, he still wins the election 1765 times. You are starting with a sub-set of the data (# of times candidate loses OH) and going from there.
Hope this helps!
d-
it really does, thanks so much ;)
@davea: If I have understood this site correctly, this is a pooling of resources that analyzes many sources and gets a median of the lot of them as well as it's own stats. Super Tracker, from my understanding, is from an outside source and quoted here to give that sites/resources opinion.
Hi Nate,
Congratulations on getting all the mentions in press about the great work you are doing on fivethirtyeight.com!
I've been following fivethirtyeight for a while now, and it has given me a great deal of insight into the elections.
I don't want to give you a big head or anything, but has anyone ever mentioned to you that your predictive techniques sorta kinda look a little bit like the beginnings os "Psychohistory" as described in Isaac Asimov's Foundation Trilogy?
http://en.wikipedia.org/wiki/Prime_Radiant
It has been ages since I read the Trilogy, but there had been something bugging me about the idea of trendlines and statistics predicting the future elections, and how you developed the ideas with your PECOTA model (as I understand it... I'm not a baseball guy), and it popped into my mind this evening about the Asimov connection.
Anyway... if no one has mentioned this, I thought I'd (try to) be the first. :-)
Good luck on CBS on election night!
Stick to the data, and you'll be much louder and smarter than the talking headsFlash Drive| Flash Card| Memory Module| Memory Card|
Smt Electronic Manufacturing|Electronics Contract Manufacturing| Electronic Contract Manufacturing Services| Subcontract Pcb Assembly
I feel better than I have ever felt about being an American. For the first time in my life, I feel hopeful, focused, and satisfied with the change toward a more progressive mindset. We are moving toward a more aware and thoughtful culture. We are embracing the best of critical thought and leaving behind the worn-out and now worthless dogma that the Bush Years have offered. This change, this newness, this hope will be nothing less than the defining moment we have been longing for. It may even promise to give the next generation something other than a title of generation "X", "Y", or "Z".
Congratulations Nate and Sean!
If you believe in the accuracy of Nate's project, then I direct you to the most recent posting at issuesthatmatter.ning.com.
History should not only be written by the winners; it should be re-written by those with the skills to place the imprimatur of factual accuracy. Go Nate.
Love your blog! congratilations
Great blog, great statistics, great analysis. Looks like a lot of work.
So, you have "A dummy variable to indicate the presence of an incumbent in the race;"
How appropriate. ;-)
when are you going to post the final regression tables!
transparency!
Either you're not going to post the final regression tables (which is rather unscientific unless you're embargoed because you're waiting publication in a journal) or I don't know where to look.
It's kind or like when they asked Maddoff to explain his system: "its proprietary."
Nate & Sean,
Have you ever noticed how much your final map looks like the current map at www.GasBuddy.com? It's kind of eerie. Maybe we don't need to do all these expensive polls and regressions? Maybe we just need to keep track of gas prices?
Keep up the good work!
情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,美國aneros,rudeboy,英國rudeboy,英國Rocksoff,德國Fun Factory,Fun Factory,英國甜筒造型按摩座,甜筒造型按摩座,英國Rock Chic ,瑞典 Lelo ,英國Emotional Bliss,英國 E.B,荷蘭 Natural Contours,荷蘭 N C,美國 OhMiBod,美國 OMB,Naughti Nano ,音樂按摩棒,ipod按摩棒,美國 The Screaming O,美國TSO,美國TOPCO,美國Doc Johnson,美國CA Exotic,美國CEN,美國Nasstoy,美國Tonguejoy,英國Je Joue,美國Pipe Dream,美國California Exotic,美國NassToys,美國Vibropod,美國Penthouse,仿真按摩棒,矽膠按摩棒,猛男倒模,真人倒模,仿真倒模,PJUR,Zestra,適趣液,穿戴套具,日本NPG,雙頭龍,FANCARNAL,日本NIPPORI,日本GEL,日本Aqua Style,美國WET,費洛蒙,費洛蒙香水,仿真名器,av女優,打炮,做愛,性愛,口交,吹喇叭,肛交,魔女訓練大師,無線跳蛋,有線跳蛋,震動棒,震動保險套,震動套,TOY-情趣用品,情趣用品網,情趣購物網,成人用品網,情趣用品討論,成人購物網,鎖精套,鎖精環,持久環,持久套,拉珠,逼真按摩棒,名器,超名器,逼真老二,電動自慰,自慰,打手槍,仿真女郎,SM道具,SM,性感內褲,仿真按摩棒,pornograph,hunter系列,h動畫,成人動畫,成人卡通,情色動畫,情色卡通,色情動畫,色情卡通,無修正,禁斷,人妻,極悪調教,姦淫,近親相姦,顏射,盜攝,偷拍,本土自拍,素人自拍,公園露出,街道露出,野外露出,誘姦,迷姦,輪姦,凌辱,痴漢,痴女,素人娘,中出,巨乳,調教,潮吹,av,a片,成人影片,成人影音,線上影片,成人光碟,成人無碼,成人dvd,情色影音,情色影片,情色dvd,情色光碟,航空版,薄碼,色情dvd,色情影音,色情光碟,線上A片,免費A片,A片下載,成人電影,色情電影,TOKYO HOT,SKY ANGEL,一本道,SOD,S1,ALICE JAPAN,皇冠系列,老虎系列,東京熱,亞熱,武士系列,新潮館,情趣用品,情趣,情趣商品,情趣網站,跳蛋,按摩棒,充氣娃娃,自慰套,G點,性感內衣,情趣內衣,角色扮演,生日禮物,生日精品,自慰,打手槍,潮吹,高潮,後庭,情色論譠,影片下載,遊戲下載,手機鈴聲,音樂下載,開獎號碼,統一發票號碼,夜市,統一發票對獎,保險套,做愛,減肥,美容,瘦身,當舖,軟體下載,汽車,機車,手機,來電答鈴,週年慶,美食,徵信社,網頁設計,網站設計,室內設計,靈異照片,同志,聊天室,運動彩券,大樂透,威力彩,搬家公司,除蟲,偷拍,自拍,無名破解,av女優,小說,民宿,大樂透開獎號碼,大樂透中獎號碼,威力彩開獎號碼,討論區,痴漢,懷孕,美女交友,交友,日本av,日本,機票,香水,股市,股市行情, 股市分析,租房子,成人影片,免費影片,醫學美容,免費算命,算命,姓名配對,姓名學,姓名學免費,遊戲,好玩遊戲,好玩遊戲區,線上遊戲,新遊戲,漫畫,線上漫畫,動畫,成人圖片,桌布,桌布下載,電視節目表,線上電視,線上a片,線上掃毒,線上翻譯,購物車,身分證製造機,身分證產生器,手機,二手車,中古車,法拍屋,歌詞,音樂,音樂網,火車,房屋,情趣用品,情趣,情趣商品,情趣網站,跳蛋,按摩棒,充氣娃娃,自慰套, G點,性感內衣,情趣內衣,角色扮演,生日禮物,精品,禮品,自慰,打手槍,潮吹,高潮,後庭,情色論譠,影片下載,遊戲下載,手機鈴聲,音樂下載,開獎號碼,統一發票,夜市,保險套,做愛,減肥,美容,瘦身,當舖,軟體下載,汽車,機車,手機,來電答鈴,週年慶,美食,徵信社,網頁設計,網站設計,室內設計,靈異照片,同志,聊天室,運動彩券,,大樂透,威力彩,搬家公司,除蟲,偷拍,自拍,無名破解, av女優,小說,民宿,大樂透開獎號碼,大樂透中獎號碼,威力彩開獎號碼,討論區,痴漢,懷孕,美女交友,交友,日本av ,日本,機票,香水,股市,股市行情,股市分析,租房子,成人影片,免費影片,醫學美容,免費算命,算命,姓名配對,姓名學,姓名學免費,遊戲,好玩遊戲,好玩遊戲區,線上遊戲,新遊戲,漫畫,線上漫畫,動畫,成人圖片,桌布,桌布下載,電視節目表,線上電視,線上a片,線上a片,線上翻譯,購物車,身分證製造機,身分證產生器,手機,二手車,中古車,法拍屋,歌詞,音樂,音樂網,借錢,房屋,街頭籃球,找工作,旅行社,六合彩,婚友,單身聯誼,未婚聯誼,婚友,交友,交友,婚友社,婚友社,婚友社,大陸新娘,大陸新娘,越南新娘,越南新娘,外籍新娘,外籍新娘,台中坐月子中心,搬家公司,搬家公司,中和搬家,台北搬家,板橋搬家,新店搬家,線上客服,網頁設計,線上客服,網頁設計,植牙,關鍵字,關鍵字,seo,seo,網路排名,自然排序,網路排名軟體,交友,越南新娘,婚友社,外籍新娘,大陸新娘,越南新娘,交友,外籍新娘,視訊聊天,大陸新娘,婚友社,婚友,越南新娘,大陸新娘,越南新娘,視訊交友,外籍新娘,網路排名,網路排名軟體,網站排名優化大師,關鍵字排名大師,網站排名seo大師,關鍵字行銷專家,關鍵字,seo,關鍵字行銷,網頁排序,網頁排名,關鍵字大師,seo大,自然排名,網站排序,網路行銷創業,汽車借款,汽車借錢,汽車貸款,汽車貸款,拉皮,抽脂,近視雷射,隆乳,隆鼻,變性,雙眼皮,眼袋,牙齒,下巴,植牙,人工植牙,植髮,雷射美容,膠原蛋白,皮膚科,醫學美容,玻尿酸,肉毒桿菌,微晶瓷,電波拉皮,脈衝光,關鍵字,關鍵字,seo,seo,網路排名,自然排序,網路排名軟體,英語演講,托福,Toastmaster,
^^ nice blog!! ^@^
徵信, 徵信, 徵信, 徵信社, 徵信社, 徵信社, 感情挽回, 婚姻挽回, 挽回婚姻, 挽回感情, 徵信, 徵信社, 徵信, 徵信, 捉姦, 徵信公司, 通姦, 通姦罪, 抓姦, 抓猴, 捉猴, 捉姦, 監聽, 調查跟蹤, 反跟蹤, 外遇問題, 徵信, 捉姦, 女人徵信, 女子徵信, 外遇問題, 女子徵信, 徵信社, 外遇, 徵信公司, 徵信網, 外遇蒐證, 抓姦, 抓猴, 捉猴, 調查跟蹤, 反跟蹤, 感情挽回, 挽回感情, 婚姻挽回, 挽回婚姻, 外遇沖開, 抓姦, 女子徵信, 外遇蒐證, 外遇, 通姦, 通姦罪, 贍養費, 徵信, 徵信社, 抓姦, 徵信社, 徵信, 徵信公司, 徵信社, 徵信, 徵信公司, 徵信社, 徵信公司, 女人徵信, 外遇
徵信, 徵信網, 徵信社, 徵信網, 外遇, 徵信, 徵信社, 抓姦, 徵信, 女人徵信, 徵信社, 女人徵信社, 外遇, 抓姦, 徵信公司, 徵信社, 徵信社, 徵信社, 徵信社, 徵信社, 徵信社, 女人徵信社, 徵信社, 徵信, 徵信社, 徵信, 女子徵信社, 女子徵信社, 女子徵信社, 女子徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社,
^^ nice blog!! thanks a lot! ^^
徵信, 徵信社,徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 外遇, 抓姦, 離婚, 外遇,離婚,
徵信, 外遇, 離婚, 徵信社, 徵信, 外遇, 抓姦, 徵信社, 徵信, 徵信社, 徵信, 外遇, 徵信社, 徵信, 外遇, 抓姦, 徵信社, 征信, 征信, 徵信, 徵信社, 徵信, 徵信社, 征信, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信社, 徵信社, 徵信, 外遇, 抓姦, 徵信, 徵信社, 徵信, 徵信社,
輕 輕 吹 奏 著 那 消 失 記 憶 緬 懷 過 去 那 段 已 逝 的 愛 情
The core dilemma of Wall Street compensation is the unexamined belief that without a certain level of pay, there would be a lack of talent. I just don't buy that at all. Every year colleges turn out thousands of economics majors, MBA's, etc. Let alone all the unemployed and underemployed credentialed and very qualified people that would love to fill in the vacuum. The media make it seem like this kind of work is as an exclusive club as nuclear science or experimental cancer research. It is not. I would love Nate to put up graphs and stats on this issue.
Nate, I thought you moved to NYC?
Hi Nate:
Your prediction of the Minnesota Senate race between Democrat Al Franken and Republican Norm Coleman was a classic. Rasmussen and Gallup had Coleman winning a close race but 538.com was dead certain that Franken was winning by over 200 votes and Nate Silver wasn't going to join the conservative bandwagon in a close recount and that uncounted absentee voters tend to lean republican. As it turned out, Franken increased his lead and eventually won the Senate seat except for some court challenges by Coleman. With that election practically settled, I can only say that you, Nate Silver is the most accurate pollster in America and probably in the world.
免費視訊聊天 辣妹視訊 視訊交友網 美女視訊 視訊交友 視訊交友90739 成人聊天室 視訊聊天室 視訊聊天 視訊聊天室 情色視訊 情人視訊網 視訊美女
一葉情貼圖片區 免費視訊聊天室 免費視訊 ut聊天室 聊天室 豆豆聊天室 尋夢園聊天室 聊天室尋夢園 影音視訊聊天室
辣妹視訊 美女視訊 視訊交友網 視訊聊天室 視訊交友 視訊美女 免費視訊 免費視訊聊天 視訊交友90739 免費視訊聊天室 成人聊天室 視訊聊天 視訊交友aooyy
哈啦聊天室 辣妺視訊 A片 色情A片 視訊 080視訊聊天室 視訊美女34c 視訊情人高雄網 視訊交友高雄網 0204貼圖區 sex520免費影片 情色貼圖 視訊ukiss 視訊ggoo 視訊美女ggoo
080苗栗人聊天室 080中部人聊天室ut ut影音視訊聊天室13077 視訊做愛 kk777視訊俱樂部 上班族聊天室 聊天室找一夜 情色交友 情色貼片 小瓢蟲情色論壇 aio交友愛情館
哈拉聊天室 洪爺影城
情趣用品 情趣用品 情趣用品 情趣 情趣用品 情趣
ダンボール段ボール箱、板)・プチプチ（エアキャップ）・梱包資材（用品）の激安通販(販売)専門店です。小ロットからのオーダー注文が可能です
チューリッヒではインターネットでお見積りいただくと自動車保険料が最大で7000円の割引！ ほんの数分お時間をいただくだけで、あなたの自動車保険料がわかります。 直接申込みができて、手続きも簡単。まずは今すぐチューリッヒ自動車保険のお見積りを！
と言う事は 当時、日本団体生命 と呼ばれていた保険会社でしょうか、
その後 → ニチダン生命 → アクサグループライフ生命 → アクサ生命 となり、契約は引き継がれていると思います。
レーザー脱毛、Vライン脱毛の銀クリの医療脱毛は安心納得価格で愛されボディに。各コース先着50名様30%OFFキャンペーン中！
レーザー脱毛、Vライン脱毛の銀クリの医療脱毛は安心納得価格で愛されボディに。各コース先着50名様30%OFFキャンペーン中！
幼児教室の春季講習
幼児教室の講習の掛け持ちについて意見を聞かせてください。
留学なら、DEOW留学センター！DEOW留学センターは手数料無料で貴方の留学を完全サ
ポート!
使い捨てコンタクトレンズ・カラーコンタクト・黒カラコン・ハードコンタクトなど取り扱いコンタクトレンズが豊富です。
艾葳酒店經紀公司提供專業的酒店經紀, 飯局小姐,領檯人員,領台,傳播妹,或者想要到台北酒店、林森北路酒店,私人招待所,或者八大行業當酒店PT,酒店公關,酒店兼職,想去酒店上班, 日式酒店,制服酒店,ktv酒店,禮服店,整天穿得水水漂漂的禮服酒店,鋼琴酒吧當酒店領檯,酒店小姐,公關小姐??,還是想去制服店當上班小姐,水水們如果想要擁有打工工作、晚上兼差工作、兼差打工、假日兼職、兼職工作、學生兼差、兼差、打工兼差、日領工作、晚上兼差工作、酒店工作、酒店上班、酒店打工、兼職、兼差、兼差工作、酒店上班等,想了解酒店相關工作和特種行業內容,想找打工、假日兼職、兼差打工、或晚班兼職想擁有快速賺錢又有保障的工作嗎???又可以現領請找專業又有保障的艾葳酒店經紀公司!
艾葳酒店經紀是合法的公司工作環境高雅時尚，無業績壓力，無脫秀無喝酒壓力，高層次會員制客源，工作輕鬆，可日領、現領。
一般的酒店經紀只會在水水們第一次上班和領薪水時出現而已，對水水們的上班安全一點保障都沒有！艾葳酒店經紀公司的水水們上班時全程媽咪作陪，不需擔心！只提供最優質的酒店打工,酒店上班,酒店打工環境、上班條件給水水們。心動嗎!? 趕快來填寫你的酒店上班履歷表
水水們妳有缺現金、有卡債、缺錢卡奴的煩腦嗎?想到日本留學日本打工嗎?妳是工讀生找工作??想要擁有高時薪又輕鬆的夜間兼職工作,打工機會和,假日打工,兼職工作日領假日打工的機會嗎??想實現夢想卻又缺錢沒錢嗎!??整天還在煩腦如何賺錢有什麼賺錢方法,和賺錢最快方法!?,想要打工,日領工作,短期打工,兼差工作,打工兼差工作嗎!?,
請加入我們艾葳酒店經紀公司工作單純輕鬆”高時薪”又可日領徵想要當傳播妹,上班小姐,酒店兼差,酒店兼職,歡迎學生打工,!!!
加入我們實現夢想就從現在開始^__^
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店經紀,
酒店打工經紀,
制服酒店工作,
專業酒店經紀,
合法酒店經紀,
酒店暑假打工,
酒店寒假打工,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店工作,
酒店打工經紀,
制服酒店經紀,
專業酒店經紀,
合法酒店經紀,
酒店暑假打工,
酒店寒假打工,
酒店經紀人,
菲梵酒店經紀,
酒店經紀,
禮服酒店上班,
酒店小姐兼職,
便服酒店工作,
酒店打工經紀,
制服酒店經紀,
酒店經紀,
菲
梵,
Two questions:
Why don't you get rid of the Chinese-language spams on this page?
Why are there pages that do not permit comments? http://www.fivethirtyeight.com/2009/09/bad-news-for-public-option.html, for example.
By comparing the traditional Internet users, Internet users to iResearch found that the traditional white-collar-based, cell phones wholesale, corporate general staff accounted for 18.9%, higher than the 5.6% of the wholesale cell phones users accounting; and discount cell phones users in the years students and blue-collar workers accounted for significantly more than the traditional Internet users, respectively, accounting for 19.5% and 18.9%, higher than the traditional Internet users Students and blue-collar workers accounted for 7.8% and 5.1% respectively.
From cell phones users to see the specific situation of occupational segmentation in 2009, accounting for 19.5% of students dropped 21.2 percent over last year, other types of occupations than those last year, the proportion of Internet users cheap cell phones increase. White collar crowd from last year's 29.2% increase to 38.9% this year, accounting for 9.7 percentage points up to replace the student groups cellphone users as one of the biggest occupational hierarchy; blue-collar crowd from last year's 13.9% to 18.9% this year, accounting for rose by 5.0 percentage points, showing that mobile phones users by a group of students to the occupational groups a significant trend in the development. Ereli advice that, cheap cell phones and mobile phone users Internet users monthly income distribution of age, education, occupational distribution has strong correlation with high spending capacity of white-collar workers and some students in the crowd will be a huge cell phone china online potential consumer groups.
The survey found that consumer 3G wholesale china from the crowd of view, the buyer 25 to 40 years old mainly white-collar workers, accounting for about 40%, followed by consumer groups of students, accounting for about three into. According to statistics, 3G wholesale products in sales, compared with a 2G mobile phone sales are still a wide gap between, but since June has been, 3G mobile phones increase in the average monthly buy products for more than 50%, "11" period due to holiday business, the increase of more than 150%. Pk that the "11" after the peak sales of 3G handsets likely to usher in more stable growth.
Post a Comment