Pollsters' Schizophrenia and the 'Convention Bounce'

In the next couple of weeks, pollsters warn us, we may well see the media polls on the presidential election take some wild turns. ABC's Gary Langer predicts the "convention bounce" could cause candidate support to "spike"—"sometimes in an election-changing way." Gallup's Frank Newport seems downright giddy about the next two weeks: "What a fantastic time we have ahead of us! The presidential election process is about to move from stagnating/static to exciting and extremely fast moving."

Yet, when we look at all of the polls, especially those of Gallup and ABC, we see reports that suggest virtually all voters have already made up their minds. How can such a decided electorate be the source of what is expected to be major fluctuations in the polling results?

The answer is that media pollsters approach pre-election polling from what might best be termed a schizophrenic position. On the one hand, they want early poll results that are "meaningful" enough to be published in the news. On the other hand, they know that many people have not yet begun to think about the presidential race, and any views that such voters might express at this stage of the campaign, about what they would do in November, would not be reliable predictors of how they will actually vote.

Indeed, if pollsters told us the truth about the electorate, they would have to report that early on in the campaign, a large segment of the electorate, perhaps as much as 40 percent or so, had not yet begun to pay attention to the presidential contest. But that's not interesting news. Who cares if Obama leads McCain by 5 percentage points, when eight times that number of voters are still undecided?

Pollsters get around this technical detail by asking a forced-choice hypothetical question: which candidates people would vote for "if the election were held today." Should any respondents have the temerity to admit they don't know, pollsters pressure respondents to say whom they "lean" toward, thus producing what appears to be a fully decided electorate. Using that format, for example, Gallup routinely reports 95 percent of voters already decided, ABC says 98 percent, while CNN is the extreme at 100 percent.

The problem is that this hypothetical situation does not represent the actual electorate. Some respondents in poll samples, of course, have made their decisions, and their responses are presumably an accurate picture of the voters at large who have also made up their minds. But the undecided voters in the sample are pressured into giving a response, and once that happens, they no longer represent the undecided voters at large, who have not been pressured.

This contamination of samples cannot be stressed too much. As we all know, in each poll, interviewers speak with only a small sample of voters nationwide, typically about 1,000 or so, and these voters are supposed to represent the larger population of some 200 million voters. In principle, despite the amazement of many nonstatisticians, such a small sample, if chosen correctly, is capable of representing the larger population within a fairly small margin of error. But if we give information to these samples of voters that we do not give to the general electorate or subject these samples of voters to pressure that the general electorate does not experience, then the samples no longer represent the larger population.

And that's the situation with most media poll samples these days. The samples themselves are typically chosen with due scientific precision. However, pollsters pressure all respondents to come up with a presidential choice, even though many have not given the election much thought. Typically, respondents will play the polling game and give an answer about whom they would support "if the election were held today," but for undecided voters, that doesn't mean they actually intend to vote for that candidate. Indeed, if pollsters would allow the respondents their true choice, many would admit they simply don't know yet. Instead, they agree to the pollsters' demand and come up with a meaningless answer.

Pollsters, in turn, treat the meaningless answer as though it represents what voters at large are thinking. But the undecided voters at large have not made a meaningless choice, because no one has forced them to do so. The net result is that in the months leading up to the election, polls misrepresent what the voters at large are really thinking.

Early on in the campaign, how do the undecided voters in the poll samples come up with a choice? Typically, the name they choose is the one they have heard of most frequently, even though they are not necessarily paying close attention to the candidates and are not necessarily committed to the choice they give the pollsters. Indeed, pollsters will readily admit that early polls do not reflect voters' real choices as much as these polls reflect the higher name recognition or greater media coverage of one of the candidates.

And that gets back to the convention bounce. During media coverage of the Democratic convention, it may well be that undecided voters in poll samples, when pressed for a candidate choice, will choose Obama, because of the positive news about him during that time (and probably negative news about McCain). Similarly, during media coverage of the Republican convention a few days later, the undecided voters in brand-new poll samples are likely to mention McCain when pressed for a choice. In neither case do the polls tell us the truth about the undecided voters, perhaps most of whom may not make up their minds on the basis of the conventions at all.

If pollsters want to accurately measure the effect of the conventions, they should allow undecided voters to admit their indecision, thus allowing us to see if people have—of their own free will—moved from one candidate to another. But pollsters are so locked in to the standard, forced-choice vote question, they will never do that. Instead, they will "discover" an apparent fickle electorate, flitting from one candidate to the other.

Indeed, Gallup's Jeff Jones has documented the history of past bounces, ranging from -1 percentage point (for John Kerry in 2004) to +16 (for Bill Clinton in 1992), with an average of 5 points for Republicans and 6 points for Democrats. This suggests an average swing in the difference between the two candidates' support of 10 to 12 percentage points. But don't believe it. The bounce is more likely a reflection of the polls' samples than the electorate at large.

Ultimately, as Election Day nears, the question about which candidates voters would support if the election were held "today" becomes more and more appropriate, because the election is almost "today." The poll samples will slowly converge with what voters at large are really thinking, the size of the real undecided vote at large will decline to resemble what the polls say, and in the end, polls will usually end up with final pre-election polls fairly close to the election results. Pollsters will then trumpet their "success" in measuring the will of the voters, ignoring how wrong and contradictory their polls have been throughout the campaign season.

David W. Moore is a former vice president of the Gallup Organization and managing editor of the Gallup Poll. He is the author of The Opinion Makers: An Insider Exposes the Truth Behind the Polls (Beacon Press).