Did Minnesota’s 8th District Really Swing by Almost 20 Points?

Minnesota’s 8th District is one of a handful of Democratic-held House seats where Republicans have a realistic shot to win in the midterms in three weeks. The incumbent, Rick Nolan, is retiring, and he won by only 1 percentage point in 2016 in a district President Donald Trump carried by 15.

When we polled this district in September, we found the Democratic candidate, Joe Radinovich, up by 1 point. Now, we have the Republican, Pete Stauber, up by 15.

The underlying numbers have changed a lot, too. Last time, voters disapproved of Trump by 1 point. Now they approve by 18. Last time, Democrats outnumbered Republicans by 2 percentage points; now Republicans outnumber Democrats by 10.

Is the shift real? Probably not entirely. But there’s probably more truth to it than a lot of Democrats criticizing the poll want to admit.

It’s a case that highlights the challenge of polling in general, and the particular challenges of polling in some states.

Minnesota is a tough state for pollsters because its voter file, a data set of every registered voter in the state, doesn’t contain information on partisanship, like party registration or whether people voted in Democratic or Republican primaries.

In contrast with most states, we can’t adjust to make sure we have the right number of registered Democrats or Republicans.

Based on all the other polls we’ve done, we can say with some confidence that the ability to control the number of registered Democrats and Republicans in a poll is a very important factor in results.

Response rates are extremely low nowadays, and our samples, at 500 per poll, are pretty small. Some of our poll results would have been 10 points different without the ability to weight by party registration or primary vote history, and occasionally even more than 10 points different. (Weighting means giving more weight to respondents from an underrepresented group to ensure the sample reflects the demographic profile of likely voters.) In almost all of these cases, it’s Democrats who have been overrepresented, not Republicans.

Based on that, and as we wrote at the time, we decided in September to largely avoid districts without party registration or primary vote history, including some places we’d really like to poll, like Montana or Minnesota’s 1st and 7th districts. For the same reason, we also considered not re-polling Minnesota’s 8th.

It would be foolish to rule out the possibility that this poll result would have been 10 points different if we could have weighted by party registration, given that we know it has had that kind of effect in other districts. One could find additional evidence for this case by looking at Trump’s approval rating and the party identification of the poll, two measures that lurched far to the right even though we don’t have much reason to believe that either ought to have moved so far.

It should be noted that this problem isn’t limited to us. A lot of pollsters going without party registration will occasionally get weird results like this. To compensate, some try to weight to party identification — whether people consider themselves Democrats or Republicans.

The challenge of weighting by party identification is that it’s hard to know the “real” party identification. That’s especially true in a congressional district where we’ve done only one poll before (typically, a firm weighting by party identification will choose to weight to the average result over several previous polls of the same area).

If we had weighted to the party identification from our September poll (in which we had Democrats outnumbering Republicans by 2 percentage points) or the average of the two polls (R+4), the results in this survey would indeed have moved to the left.

Stauber would have led by 9 points if we had weighted to the average party identification of the two polls together. He would have led by 4 points if we had weighted to the party identification from September.

Either of those results could be a more accurate reflection of the race. But I would note that Stauber leads in all of these hypothetical situations. There are some reasons that shouldn’t be too surprising.

I would guess that response bias — the possibility that Republicans were likelier to respond—plays a pretty meaningful role in moving this result. But there are at least three reasons to think there’s more driving the shift than that.

One factor is that we are now naming third-party candidates in the final stretch, and the Independence candidate, Skip Sandman, has 4 percent of the vote. Sandman, who has previously attracted 4 percent of the vote here as a Green Party candidate, almost exclusively wins voters who disapprove of Trump, and voters who live in the Duluth area, which leans heavily Democratic.

Another factor: There has been a lot of campaigning since early September, and this is one of the few districts where Republicans are airing more advertisements than Democrats. Republicans are believed to be favored in basically all the other contests where they are broadcasting more advertisements.

Third, there’s evidence of improved Republican standing in conservative areas since early September, including in the nearby North Dakota Senate race. If there’s a broad trend toward greater polarization of the electorate along the lines of the presidential election, that might be particularly helpful to Republicans in a conservative area where Trump won by 15 points. And while Democrats do often win here, it is worth noting that this is a socially conservative area that opposed same-sex marriage by a wide margin in 2012. The Kavanaugh Supreme Court nomination fight may be helping Republicans here, too.

Put it all together, and the change from one poll to another is probably a combination of a real shift and of the challenge of polling in a state like Minnesota, without party registration or primary vote history.