tag:blogger.com,1999:blog-4257917002416684161.post3264167795203385881..comments2008-04-07T03:17:54.957-05:00Natehttp://www.blogger.com/profile/08334852368748204318noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-4257917002416684161.post-68579778900827390102008-04-07T03:17:00.000-05:002008-04-07T03:17:00.000-05:00Hardheaded,<BR/><BR/>These are excellent points and deserve a thorough read, but, let me acknowledge that I was trying to simplify things slightly as this post was already running quite long.538/poblanonoreply@blogger.comtag:blogger.com,1999:blog-4257917002416684161.post-57221164273323843172008-04-07T01:59:00.000-05:002008-04-07T01:59:00.000-05:00"So in a given week, we're probably looking at somewhere between 75 and 100 polling results. (By my count, there have been 86 different Presidential election-related polls released publicly since last Monday). Not only might some of these polls be "wrong" (that is, fall outside their margin of error). Some of them will be wrong. It's a mathematical certainty. We should expect to see four or five "bad" polls each week. And that's assuming that the pollsters are doing everything perfectly, which they aren't (not by a long shot)."<BR/><BR/>Poblano, you have a technical error here that you may want to address.<BR/><BR/>Just because 5 of 100 polls are going to be way outside the margin of error over an INFINITE # of trials, that does not mean that 4 or 5 polls in any particular week will be taken from samples that do not reflect the "true" rate of the response in the population sampled.<BR/><BR/>The statistical principles at issue actually suggest that we are likely to see few or no polling reports with large errors for two, three, or four weeks at a time, and there are likely to be some weeks when as many as seven or more polling reports are far off the mark from the "true" rate of responses in the target population.<BR/><BR/>In other words, the errors do not occur regularly over time, but tend to bunch up in certain time periods which will have a higher frequency of errors than predicted. This leaves many time periods when there are fewer errors than predicted.<BR/><BR/>ALSO REMEMBER that the 5% level of significance is only ONE potential source of error in the reported poll numbers. This % applies only to the random chance that an unrepresentative sample of responders has been selected from a source that starts out as being perfectly representative.<BR/><BR/>Error also can occur because the sources from which the responders are drawn does not accurately represent the target population. This is the type of error that can theoretically occur from calling only people with land lines and assuming that people with land lines are representative of people who have only cell phones.<BR/><BR/>Even greater errors are probably introduced from the "likely voter" models that each of the polling organizations has constructed.<BR/><BR/>Additional large errors can be introduced by "correcting" the raw poll response rates in an effort to make the responses more representative of the "true" response rates in the target population. Rasmussen, for example, has been "adjusting" its raw response numbers to "correct" for the fact that there are two few Republican party identifiers in its polls. The problem is that the polls probably need no correction at all - the party identifier rates Rasmussen uses show many more Republicans than show up in polls by Pew Research or other national pollsters.hardheaded liberalnoreply@blogger.com