The Associated Press has put out a very interesting piece explaining (in limited detail) how political polls are done and why they can produce wildly differing results. In this election, the variation is pretty impressive. In the last week, a Pew Research poll showed Senator Barack Obama (D-IL) ahead of Senator John McCain (R-AZ) by 14 points. An AP poll showed the two candidates tied—a statistically insignificant 1 point lead for Obama.
The article, however, misses some important bits. For a long time pollsters did not call cell phones, which excluded those of us (like me) who don’t have land-lines entirely. Some polling organizations have begun including cellular customers, but there are still classes of people—introverts (like me) who don’t answer calls from numbers we don’t recognize, for example—that get excluded. Do these kinds of oversights affect the outcomes of the polls? Maybe. Maybe not. It’s hard to say whether introverts tend toward a particular candidate or not, especially since we don’t have much poll data on that ;-).
A million things can affect these polls. What time of day did they call people? People of different financial means tend to work different hours. Calls at dinner time might be ignored by families who eat dinner together. How were questions worded? They also (as the article explains) have to perform statistical adjustments for who is ‘likely’ to vote, but that can shift every year. For example, some pollsters are skewing the weighting for the African American vote higher this year on expectations that black turnout will be higher than in past years. In reality, these weightings are an educated guess that could easily be either right-on, underestimated, or overestimated.
All-in-all, one poll matters: the vote in November. These other ones are interesting, but they can be wrong. Don’t forget, everybody thought Governor Howard Dean (D-VT) would win the Democratic nomination in 2004 based on polls, but Senator John Kerry (D-MA) trounced him.