Proceed with caution on early poll results

Pollsters may need to change how they handle poll numbers for third-party candidates if they want to accurately measure voter attitudes in North Carolina's U.S. Senate race.

Posted 9:11 p.m. Jul 28, 2014 — Updated 2:56 p.m. Aug 5, 2014

RALEIGH, N.C. — The most recent survey released by Public Policy Polling finds Democratic U.S. Sen. Kay Hagan with her largest lead yet over Republican challenger Thom Tillis, 41 percent to 34 percent. Meanwhile, Sean Haugh, the Libertarian candidate, continues to poll near double-digits, with 8 percent support. Another 16 percent of likely voters said they were undecided. As I’ve been trying to make clear in past posts, we should be cautious in assuming that Hagan is really ahead.

First, I can’t stress this point strongly enough: Polling conducted further away from the actual election is less predictive than surveys taken closer to the election. In fact, PPP finds 16 percent of past voters are still unwilling to choose a candidate.

Second, when pollsters give respondents a third-party candidate alternative, it can significantly affect the results. I’ve been compiling all polls for the Senate race to demonstrate how this matters for an upcoming post. For now, I’ll just link to other recent stories that support my belief that Tillis will benefit from Haugh supporters abandoning him in November.

It matters whether pollsters ask about a third-party candidate because, as these excellent Washington Post stories find, third-party candidates usually poll better than they perform on election day. This means a non-trivial number of voters defect to one of the two main party candidates, and some won’t vote at all. Which way these defectors break can decide the outcome.

What caught my attention, though, were results PPP obtained but did not trumpet.

Respondents who said they would vote for Haugh were also asked which of the other two candidates they leaned towards. When leaners’ preferences are added to the totals, Hagan’s lead over Tillis shrinks to 42 percent to 39 percent, well within the margin of sampling error of +/-3 percentage points.

As a reminder, the margin of sampling error means that each candidates’ estimated support could be as much as 3 percent less or as much as 3 percent more due to sampling alone. In short, Hagan’s true support is somewhere between 39 and 45 percent, while Tillis’ is somewhere between 36 and 42 percent. Since the ranges of their estimated support overlap, we aren’t as confident that one candidate is truly ahead.

Interestingly, I discovered the 42 percent to 39 percent distribution by looking at all of the polls in this race aggregated by Real Clear Politics.

This raises an incredibly important question: How do polling aggregators such as RCP and Pollster at Huffington Post deal with surveys that ask about third-party candidates? Since these candidates poll better than they will fare come November, perhaps their support should be discounted or ignored when constructing the main two-party candidate averages.

Deciding how to treat third-party candidates has consequences. RCP and Pollster have very different estimates for North Carolina's Senate race. RCP has Hagan barely leading, 45 percent to 43 percent. Pollster, meanwhile, has Hagan leading 42 percent to 37 percent, a statistically distinguishable advantage. The latter seems more compatible with the New York Times' The Upshot election forecast model, which gives Hagan a 60 percent chance of winning as of today.

Yet, if RCP is using more reliable data, based on removing third-party influences, then averages generated at Pollster and forecasts like the Upshot’s are missing their mark. My sense is these models are indeed too optimistic about Hagan’s chances. I think this is why predictions that rely on more qualitative assessments of the races, such as the Cook Report and Larry Sabato’s Crystal Ball, both see the North Carolina race as a "toss-up."

Suggest A Correction