The likely voter models used by many pollsters to ascertain which respondents are probably going to turn out for an election seemed to conk out in 2012. A new study by one of the industry’s most respected professionals, to be accompanied by R&D on a new likely voter model, finds that the mid-20th century vehicle was no match for 21st-century demographics and targeting.
As they have for decades, pollsters in 2012 sought out voters who expressed great interest in the election or likelihood of turning out. This pool of “high-interest” voters was used to create likely voter models for pre-election surveys that subsequently showed a very winnable race for Governor Mitt Romney:
- Of the seven final pre-election public polls, four showed President Barack Obama winning by one to four points, two showed the race tied, and Gallup’s showed Romney winning by a point.
- On Election Day, the Real Clear Politics moving average of national polls showed Obama up by one point.
- Republicans and whites overall expressed high interest in voting; core Democratic constituencies like young adults and non-whites, not so much.
- Lower overall interest in the election than in 2004 and 2008 suggested that lower turnout would jack up the influence of high-interest—i.e., Republican—voters.
You know what they say about a miss and a mile. On top of an average Obama national lead of one point that ultimately clocked in at four, the Real Clear Politics moving averages for state polls in eight of 11 battlegrounds also came up short by at least three points. Obama’s unexpectedly wide margin stunned a Romney campaign that had expected to win and, some believe, goosed the break-up of longtime polling partners Gallup and USA Today.
Why did the likely voter model sputter? Bill McInturff had reasons to find out—as co-pollster for one of the most respected national public surveys, the NBC News/Wall Street Journal poll; as a partner at Public Opinion Strategies, which polled for the Romney campaign; and as a member of a small but intellectually highly competitive industry. “If we’re the first ones to get it right and it works better, it’s good business.”
The upshot of McInturff’s findings (you can read his study here): A likely voter model based solely on self-described interest in the election failed to capture the true interest level and the strength of Democratic turnout efforts among voters age 18-29 and non-whites, especially Latinos. These groups are core Democratic groups, heavily dependent on cellphones and thus tougher to poll.
“Likely voter models have generally been accurate in the past, but the sheer intensity of the scope and reach of today’s presidential campaigns” through advertising and voter contact “is unlike anything witnessed before,” McInturff finds:
- The vast majority of more than $1 billion in TV advertising was targeted at just eight states.
- A previously unheard-of 70 percent of swing-state residents interviewed in the NBC/WSJ mid-October poll said they had been contacted by a presidential campaign.
- NBC/WSJ polling also showed Obama with a 21-point edge over Romney in contacting voters age 18-34 by email or social media, which may have had more powerful effects.
- Even with lower self-described interest in voting and a drop in turnout from 2004 and 2008, turnout increased among young adults and Latinos to a degree not reflected in any final pre-election public poll—even those polls that accurately predicted the result.
Some pollsters may call this a Republican problem, or even a more isolated one. Yet even one Democratic public poll that nailed the final margin got there with a sample that was too white, too old and not sufficiently Latino to match the exit poll demographics.
Democratic and Republican pollsters made some differing assumptions going into Election Day. Republicans believed that 2008 was a onetime event—that the first African-American major-party presidential nominee drove up turnout among minority and younger voters to a degree they believed would never happen again. Democratic pollsters saw 2008 as the “new norm,” the makeup of the electorate forever changed. Only one side could be proven right.
But that doesn't resolve the question of why a historically tried-and-true method of screening likely voters didn't work. Independent pollsters with no partisan axe to grind found that it didn’t.
Given the pace of growth in our Latino population, in development of targeting techniques, and in the popularity of early voting and voting by mail, which makes casting a ballot easier than ever, the likely voter model probably could stand a makeover before the 2016 open-seat showdown.
Here are McInturff’s proposed steps for how to go about it:
- Survey samples must keep pace with the percentage of US households (34 percent in 2012) that are cellphone-only.
- The base of voters who qualify for a likely voter model in a presidential year should be roughly 80 percent of registered voters; this 80 percent should not be further refined through additional filters.
- In addition to self-described interest, other polling indicators such as past voting behavior, recall of contact by a campaign, and intensity of feeling toward a candidate should be factored into the model.
- Turnout models need to be more generous in their assumptions for certain target populations. Even then, additional weighting probably will be needed to help compensate for “missing” likely voters.
- In presidential years, the model should use a default gender breakdown of 47 percent men, 53 percent women.
McInturff’s final suggestion, that pollsters need large-scale, “living” labs to test new models, comes with some planned follow-through. In the coming weeks, Public Opinion Strategies plans to match its trove of survey data with state voter files to identify past poll respondents who actually voted in recent elections; “highlight those ‘likely voters’ who did not vote and those ‘non-likely voters’ who did;” and examine “what, if anything, better predicted their actually voting or not voting.”
After subsequent interviews and studies of target respondents to learn more about how to construct a new likely voter model, the firm will build and test-drive one in the 2013 elections. Their forthcoming update will be an interesting addition to the report from Gallup on its internal review of its 2012 tracking.
When we’re at the point where campaign spending and tactics can trump self-described interest in voting, McInturff says, it’s “time to think differently about what we do.”
Elizabeth Wilner, VP Kantar Media Campaign Media Analysis Group (CMAG), is Contributing Editor at the Cook Political Report
February 12, 2013The State of Public Affairs Polling Is… 12 Predictions for How Polling May Adapt for the Future
February 5, 2013The 113th Congress: Brought to You by 800+ Advertisers
January 29, 2013DC TV’s Challenge: Moving Beyond Alphabet Advertising