Be skeptical of Rasmussen's recent poll result

The most recent NC Senate poll by Rasmussen has Tillis leading Hagan, 45 percent to 40 percent. I have significant doubts about the validity of this poll, and you should too.

Posted 8:49 p.m. Aug 11, 2014 — Updated 9:39 p.m. Aug 11, 2014

RALEIGH, N.C. — The most recent U.S. Senate poll by Rasmussen has Republican Thom Tillis leading Democratic Sen. Kay Hagan, 45 percent to 40 percent. I have significant doubts about the validity of this poll, and you should, too.

First, Rasmussen has been one of the more inaccurate pollsters for the last two general election cycles. As Nate Silver found in 2010, Rasmussen was uniquely bad:

“The 105 polls released in Senate and gubernatorial races by Rasmussen Reports and its subsidiary, Pulse Opinion Research, missed the final margin between the candidates by 5.8 points, a considerably higher figure than that achieved by most other pollsters. Some 13 of its polls missed by 10 or more points, including one in the Hawaii Senate race that missed the final margin between the candidates by 40 points, the largest error ever recorded in a general election in FiveThirtyEight’s database, which includes all polls conducted since 1998.”

In 2012, Rasmussen presidential polls missed their mark on average by over 4 percent. Compounding the problem, their errors were not random. Rasmussen results were biased in favor of overstating Republican Mitt Romney’s support by almost 4 percent. Their poor performance is probably what led to the departure of its founder in 2013.

There are several reasons why Rasmussen has been more inaccurate than almost any other pollster. The two most obvious ones are that they do not call cellphones since they are a “robo-poll" – one with automated dialing and pre-recorded scripts – and that they have been known to weight their data by respondents’ partisanship. The first explanation isn’t sufficient on its own, though, since Public Policy Polling is also a robo-poll that doesn’t reach cellphones but is more accurate. Weighing the results by respondents’ party ID is probably the more important factor.

The dominant opinion in the survey industry is that weighting responses by party ID is a mistake. Unlike known demographics about a population, such as gender and age, which we know from census data, party ID is not one of them. We don’t know the true distribution of Democrats and Republicans (and other affiliations) because it is an attitude that can change, not a fixed trait. In fact, it was the false assumptions made by folks in the Romney camp about the distribution of partisans in the electorate that led to their false belief that Romney was going to win.

Another reason I think this poll is questionable is because it has other odd results.

According to the same poll, President Barack Obama has just a -1 percentage point net approval rating in the state, with 48 percent approving and 49 percent disapproving. No other poll I’ve seen finds anything remotely like it. Just one week earlier, PPP had Obama -12 point net (41 percent approve and 53 percent disapprove), while a Gravis Marketing & Human Events poll similarly had Obama -11 point net. Curiously, that higher-than-expected approval rating for Obama would suggest that the Rasmussen sample has too many Democrats in it, which would presumably bias Hagan’s support upwards, and yet they give Tillis his largest lead in any poll. That makes no sense.

Another oddity is that Rasmussen says Gov. Pat McCrory has a +10 point net approval rating, 53 to 43 percent. Again, this is far out of line with every other poll I’ve seen. As American Insights shows, almost every prior poll finds McCrory in negative territory.

Of course, the Rasmussen poll has a margin of sampling error of +/-4 percentage points. The lower end of that range for Tillis’ support is much more compatible with other recent polls that tend to give Hagan a narrow lead or find Tillis up by 1 point.

It is entirely possible that Rasmussen is picking up on an emerging trend where Tillis’ support is truly increasing. Given their track record is poor and other results from that survey defy common sense, skepticism is clearly warranted.

Suggest A Correction