Polls show Tillis poised for win despite baffling name ID data

The polls also agree that Tillis’ lead is at least double his closest rival. Both polls find Dr. Greg Brannon running second with 20 percent support, with Rev. Mark Harris languishing in third place.

Seventy-three percent of the Elon sample of Republicans said they recognized Tillis, but 80 percent and 91 percent , respectively, of the PPP and Civitas samples said they did. Likewise, 28 percent of the Elon sample of Republicans said they recognized Brannon, but 62 percent and 73 percent of the PPP and Civitas samples, respectively, said they did.

Unsurprisingly, candidate favorability ratings also varied wildly across the three polls.

As you can see, the gap in the percentage viewing Tillis as favorable varies depending on the poll by as much as 14 percentage points, while the gap is as large as 11 percentage points for viewing him as unfavorable. Most striking, the percentage of respondents who would not or could not rate Tillis varies from as little as 12 percent in the Civitas poll to as much as 44 percent in the Elon poll.

There are two things, I think, driving these inconsistent results. One, the survey questions were asked differently across the three polls. Second, the sampling methods and modes of administration for all three polls varied as well. Let me tackle each in turn.

PPP asks, "Do you have a favorable or unfavorable opinion of [candidate X]?" Respondents are allowed to answer that they are unsure, although that is not part of the question wording. This wording encourages respondents to report having an opinion.

Elon asks, "Do you recognize the name [of candidate X]?" In contrast to PPP, this wording does not encourage people to answer affirmatively, even if they lack knowledge of the candidate.

Finally, Civitas asks, "Now, I would like to read you a list of names and have you tell me, for each one, whether you’ve heard of that person, and if so, whether you have a favorable or unfavorable opinion of them." Civitas' question ended with,"or have you never heard of [candidate X]?'"

It's not immediately obvious why the Civitas poll generates more reported feeling about all three candidates than the PPP poll. My hunch is that Civitas pollster pushed respondents who initially reported not knowing to provide an answer. That practice is criticized as manufacturing opinions by some or praised by others for getting respondents with limited interest but real, albeit weakly held attitudes, to reveal them.

The second issue, mode of administration, is that PPP used only pre-recorded interviews that were automatically dialed to phone lines by computers, a practice known as interactive voice response, or IVR, polling. This means that, for this survey, people who have only cellphones had no chance of being included in the sample, thus making the final results more dependent on judgments about how to weight the responses of respondents. Importantly, people who can be reached only by cellphone tend to have different characteristics than those who can still be reached over a land line or over the Internet.

The Elon poll and the Civitas poll both included a cellphone-only sub-sample, but their sampling frame was inconsistent. Elon called a representative sample of adults in North Carolina, and then for this analysis examined how self-identified Republicans within the Elon sample felt about the candidates. Civitas, like PPP, called only people they believed to be registered Republicans. Civitas’ pollster, though, asked more questions to make sure the respondent was a likely voter in the Republican primary.

Polls are useful. But when only a handful of polls take place, and they have different sampling methods, different modes of administering the survey and vary how they ask the same basic question, it is next to impossible to say which poll is more accurate.

What we hope to find is some level of consistency in the answers to the questions we care most about. In this case, I am confident Tillis will win the plurality of the vote in the Republican primary, and recent polling indicates he likely will win a large enough percentage of the vote to avoid a runoff. But for the geeks like me out there, questions about the importance of name recognition and how it affects candidate support remain because they are very dependent on the poll in question.