Beware the Outlier Poll

Five weeks ago, as both public and private polling showed independents shifting strongly against Republicans, this column quoted one of the most experienced campaign consultants in the GOP remarking that it felt more like a September or October than a June or July.

That came to mind Sunday night when a friend forwarded an email from a very passionate and partisan Democrat who had grown alarmed by a just-released CNN poll indicating that the presidential race was tightening. The survey showed Joe Biden’s lead over President Trump narrowing to just 4 points among registered voters, 50 to 46 percent. Among the 72 percent in the sample who said they were either extremely or very interested in voting this fall, one method for discerning more-likely voters, Biden’s lead was a bit wider at 7 points. Yet among those who lived in one of 15 states that CNN pollsters deemed “battlegrounds,” Biden’s lead was down to a single point, 49 to 48 percent.

As an election nears, many partisans start heading for the nearest window ledge every time they see a poll bearing unwelcome news. Sometimes they have good reason to be concerned, but often the poll is an outlier, one not particularly representative of other polling conducted contemporaneously. There is an old saying that if something looks too good to be true, it probably is. In polling, a good rule of thumb is that if the results of a poll look very different from others, it is more likely than not an outlier.

Though I frequently cite poll averages, I have my own issues with overreliance on them. After all, they can only be as good as the quality of the most recent batch of polls in the time period that the average was calculated. The good thing about averages, however, is they tend to wean people away from the practice of cherry-picking polls or revealing their own confirmation bias.

In the last two weeks, there have been seven national public polls on the presidential race using live telephone interviewing, the methodology I consider to be the gold standard. The results of those seven ranged from the previously mentioned 4-point Biden lead all the way up to another putting the former vice president up by 13 points. In all, they averaged out to a 9-point lead for Biden. The former vice president had support among the seven polls of between 49 and 54 percent, averaging 51 percent. Trump’s lowest level of support was 41 percent, his highest 44 percent, and his average 42 percent.

Though I am not a big fan of online polling, one of the best such outfits is the highly regarded Pew Research Center. Their survey released last week reflected an 8-point Biden lead, 53 to 45 percent, close to the 51-to-42-percent average of the seven conducted by phone.

Not all polls are equal and should be given equal credence, but in analyzing the numbers of several reputable polls, I like to use a variation on the Olympic scoring method: Throw out the highest and lowest marks and look at the rest. You will usually have a pretty good sense of where things stand.

I’d also advise taking with a grain of salt the opinions of voters in battleground states that are extracted from a national poll. Polling in individual battleground states is important to watch, but oversampling battleground voters from a national poll is highly problematic.

A quality poll taken in a single state is carefully examined, with weights applied to make sure that some important variables as race, gender, age, and educational attainment are controlled for. With college-educated whites far less likely to be supportive of Trump while white voters with less education tend to back him, such weighting is a big deal.

As a practical matter, it is not feasible to put weights in a national poll that result in representative samples of individual states; it really cannot be done. Attempting to project electoral votes based on a national poll is beyond the pale.

The live telephone interview polls we have seen in the last week don’t look much different from the flurry of solid polls, mostly from the same pollsters, that we saw in mid-July. Things have not gotten worse for Trump in the last month, but they sure aren’t better either.

So what can happen that could close the gap and make the race competitive once again? Virtual conventions may well not yield the traditional 4- or 5-point bounce that have been routine in past cycles. The strong third-quarter economic rebound that Trump backers hoped would refocus attention on economic strength looks less likely now that the coronavirus is surging in so many places. Public perception of the president’s handling of the coronavirus is showing no signs of improvement. The speculation that Biden would pick a wild-card running mate with a problematic background didn't come to pass. Trump backers seem convinced that Biden will come across like your goofy great uncle, something that I think is highly unlikely.

Trump pulled an inside straight at the poker table in 2016. He may have to get even luckier this time around.

This story was originally published on nationaljournal.com on August 18, 2020