Margin Of Error: Breaking down the polls

Margin Of Error: Breaking down the polls

Trumped up poll numbers for Carson

Posted October 28, 2015 1:44 p.m. EDT
Updated July 13, 2018 2:55 p.m. EDT

Republican presidential candidate Donald Trump speaks as  from left, Chris Christie, Marco Rubio, Ben Carson, Scott Walker listen during the first Republican presidential debate at the Quicken Loans Arena Thursday, Aug. 6, 2015, in Cleveland. (AP Photo/Andrew Harnik)

— On Tuesday, CNN declared, "Carson knocks Donald Trump from the top spot nationally." The story that Dr. Ben Carson, a neurosurgeon-turned-conservative candidate, was now leading the Republican primary field was repeated just about everywhere.

I just listened to it on NPR, and if you read The New York Times, Politico, or even ABC, you’d have just been told Carson was now "winning" the GOP primary.

That’s just not true.

Just about everything wrong with how news media tend to cover election polling was encapsulated in those headlines.

Carson doesn’t lead Trump in any other poll except the one generating those headlines. According to the average of all polls, Trump leads Carson by about 10 percent. That CBS poll is the only one in the month of October to find Carson leading the field. As the graph below shows, Carson's support is increasing, but he remains second to Trump.

Huff Po: Carson vs Trump

So why is that one poll getting so much attention?

National news organizations like CBS partner with other media and one or two pollsters. When the media commission a poll, they are going to report about it, and they usually ignore other media’s polls. CBS and The New York Times can breathlessly declare that Carson leads by conveniently ignoring every other survey to date.

To make matters worse, the CBS survey really doesn't definitively show that Carson leads Trump. The margin of sampling error in the CBS survey is a whopping plus or minus 6 percentage points. That means the range of Carson's support is anywhere from 20 percent to 32 percent. In other words, if that same CBS poll were repeated over and over using the same sample size and methodology, it would sometimes find Trump ahead of Carson, much like every other poll.

Latest GOP Polls Table

Pollsters know that election polls conducted this early are not overly predictive of eventual outcomes. Most people have yet to make up their minds. NPR recently produced a great story about this fact.

Oddly, you will often hear reporters say, "Now, early polls aren't that meaningful," and then proceed to provide you with a story about an early poll. To its credit, the Times article also noted that most respondents in the CBS poll admitted they were undecided. Most, they acknowledge, means that more than 70 percent of respondents said they could change their minds.

Americans don’t vote for presidential nominees in a national primary. National polls are for navel-gazing, not for serious consumption. Asking a nationally representative sample of registered or likely voters is simply not comparable to how candidates win or lose the nomination.

In the grand scheme of things, it really doesn’t matter if Trump or Carson has a lead in a national poll. As they know, it is imperative to win delegates in states such as Iowa and New Hampshire first and foremost. Focusing on national polls ignores how an election for the president of the United States of American actually is conducted.