Margin Of Error: Breaking down the polls

Margin Of Error: Breaking down the polls

Poll of polls: Hagan likely to win re-election

Posted November 3, 2014

Logo for the Margin of Error blog.

— Voting in this year's midterm election ends Tuesday night, and pollsters have finished their work on North Carolina's U.S. Senate campaign. So, it's finally time to take stock and predict a winner. Of course, the only thing that really matters is actual votes, and if a candidate’s supporters don’t show up to vote, it doesn’t matter how well they polled.

That said: Democratic Sen. Kay Hagan is poised to win, according to recent polls.

In the final WRAL News and last Elon University polls, for example, Hagan had a 4 percentage point lead over Republican state House Speaker Thom Tillis among likely voters. Both polls estimated Hagan was supported by 45 percent  of likely voters, while Tillis had 41 percent of their votes. Also, looking at just registered voters, Hagan’s lead over Tillis expands to 45 percent  to 38 percent, confirming conventional wisdom that her chances of winning increase with higher turnout.

Another recent poll gives a slightly different picture. High Point University found the race to be a dead heat, 44 percent to 44 percent. So, when polls seem to conflict, which one should we trust? In this case, the simple answer is that all three polls have margins of sampling error that are greater than the differences between their results. They are all finding roughly the same thing.

Nevertheless, this situation offers two important lessons.

First, it is better to rely on averages of polls than to put too much stock in any single survey. A lot of polling data has been collected about this race, so we should care more about what, on average, they have found.

Second, sometimes one poll really is inferior. It is important that pollsters are transparent and make their survey instrument available. High Point disclosed their information, and I used it to decide their results were less reliable.

You’ve probably heard someone say that surveys can be made to find anything. That’s an exaggeration, but opinions are influenced by question order and wording, even if there is no intent to manipulate the findings. Among pollsters, it is well known that asking narrow questions about a topic before asking a more general question about it influences the results for the latter. For example, asking about President Obama's handling of the economy before asking about his overall handling of the presidency leads to greater disapproval if most people are unhappy about the economy. On the other hand, first asking about Obama’s overall job approval won't affect later opinions about his handling of specific issues, such as the economy, ISIS or Ebola.

I think High Point, which like WRAL News used SurveyUSA to make the phone calls over nearly identical dates, might have biased their election question. Respondents were asked about the direction of the country and job approval for Obama before being asked about their vote choice. That’s not conventional practice for election polling. In fact, it’s typically seen as bad practice. I am concerned that their question order primed voters to base their voting decision more about Obama than they would otherwise, and doing so increases Tillis' support.

Looking at the polls of polls

HuffPo: N.C. Senate AggregationLooking at polling averages – the results from many polls averaged together – is more reliable than worrying about any single poll. Some people do this just by averaging polls, while others combine polling results with other data to predict who will win. In my opinion, the best polling averages are Nate Silver’s "FiveThirtyEight," Huffington Post’s Pollster, Real Clear Politics, the New York Times' "The Upshot," and the Washington Post's "Election Lab."

All five agree that Hagan leads Tillis, although her estimated lead is incredibly small. The NYT estimate, for example, gives Hagan just a 1.7 percentage point  lead, and that’s the largest lead any of these sites have for her. On the low end, Real Clear Politics and Pollster estimate Hagan’s lead is 0.7 percentage point.

Prediction models that take these numbers into account give Hagan a much more comfortable advantage. They estimate Hagan has between a 70 percent and 80 percent chance of winning. While it might seem like her chances of winning are too high given the slim lead in the polls, other data about North Carolina point to her victory. Also, while the lead is small, it is based on tens of thousands of respondents, which is the main virtue of aggregating polling results.

What are some additional signs that Hagan will win?

African American early vote turnout 2014One is that Democrats are doing a better job at early voting. In contrast to 2010, when more Republicans voted early, this year more Democrats voted early. Some of these data could have changed, but the trends appear stable.

Second, Hagan needs black voters to show up. So far, they are, although it isn't clear their increased participation in early voting in 2014 compared with 2012 will be substantial enough. Tuesday will be a bigger test for Democrats’ efforts to turn out the vote of groups less likely to show up when the president is not being decided.

So, while anything can happen, it looks like Hagan will hold onto her seat. I’ll have more to say about why in subsequent posts when exit-polling data become available. That said, Republicans can take solace that, even in the event Hagan wins, Republicans are just as likely to gain majority control over the U.S. Senate.


Please with your account to comment on this story. You also will need a Facebook account to comment.

Oldest First
View all
  • LuvsThePack Nov 5, 2014

    View quoted thread

    And how did that work out for ya?

  • LuvsThePack Nov 5, 2014

    Hagan likely to win re-election?

    How'd that work out for ya, WRAL?

    But hey, be content knowing you did your part to help her.

  • Glen Page Nov 5, 2014
    user avatar

    Wow WRAL, couldn't believe you would post this article on the morning of an election to try and sway people from going to vote against your candidate Hagan, but it didn't work because myself and a lot of voters, when we saw this we went to vote and we voted for Thom Tillis, WRAL biased reporting as usual, how pathetic.

  • a sports fan Nov 5, 2014

    High Point 1
    WRAL/Elon 0

    I would like to think it was just a polling error but your left leaning posts makes me conclude that your poll had bias.

  • Stan Holcomb Nov 5, 2014
    user avatar

    Hardy har har.  When will you people at WRAL learn that you can't suppress our vote with your lies and misrepresentations?

  • jwpnc Nov 4, 2014

    LMAO how do you like them apples left leaning WRAL.

  • theliberadicator Nov 4, 2014

    Grimes got smashed to bits and Mitch is the winner in KY.

    First loss of the night for the lying Clintons and the other untruthful Dems.

  • Atheistinafoxhole Nov 4, 2014

    View quoted thread

    "I am sure that you have never received on as your posts seem to always agree with their liberal, nonsensical, agenda."Who were you talking to here?

  • Atheistinafoxhole Nov 4, 2014

    View quoted thread

    "I think I hear the teacher calling, time to leave the playground."

    Aw, we were having so much fun.

  • Atheistinafoxhole Nov 4, 2014

    View quoted thread

    Haha! Did I intrude on your fragile, private conversation IN THE PUBLIC COMMENT SECTION?