Help me keep The Crosstab ad free. Subscribe to premium content and join the community on Patreon.
Become a Patron!

Be Careful With Polls of Unnamed "Generic" Candidates

Be Careful With Polls of Unnamed "Generic" Candidates

An organization called Patriot Majority USA released an array of public polls yesterday that contain great numbers for Democrats. The readings, fielded by Public Policy Polling in October and November, give Democrats the leg up in 20 key districts currently held by Republicans. If they really are up by that much (and if the lead holds until November 6, 2018), Democrats would have good odds of taking back the House majority in the 2018 midterms.

While the polls show great news for Democrats — they are beating their benchmarks by an average of 14 points — there is one big reason why we should take these polls with a grain of salt: they don’t all include names for Democratic candidates.

Instead, the polling firms will ask respondents to say whether they’ll vote either for a Republican incumbent, perhaps “Marth McSally” or “Darrell Issa,” or “a generic Democrat.” Of course, this is not a decision that pollsters make in a nefarious manner. In many cases (like those in McSally’s AZ-02 and Issa’s CA-49) there are multiple Democratic candidates running for their party’s nomination for the general election campaign. Pollsters cannot, however, predict the future. So the “generic” candidate is really the best bet at measuring candidate performance given that the eventual candidate is unknown.

At first glance, this may not seem like a problem. After all, people keep their political party in mind when contemplating politics. Why then would it matter if the pollster didn’t name names? Don’t Democrats just vote for Democrats? Don’t Republicans just vote for Republicans? The answer: of course not.

According to data from the 2016 Cooperative Congressional Election Study, a survey that matches Americans with their validated vote record, 93% of Democrats voted for Hilary Clinton in the 2016 presidential election and 92% of Republicans voted for Trump. That’s a lot of the vote explained simply by partisanship, but the 7% of Americans who don’t vote in line with their party identification is notable. Though other factors (like ideology, feelings towards different groups, etc.) can help explain this atypical behavior, let’s circle back the original question. Do candidates matter?

As it turns out there is a treasure trove of research that finds that differences in candidates characteristics do matter. At a national level, this is easily contextualizable: Joe Biden is, after all, very different than Hillary Clinton even though they hold many of the same issue positions. Biden has his own baggage that Clinton doesn’t, and vice versa. This affects how voters evaluate them and, ultimately, what would happen in an election between the two. At the most basic level, we can ask what the purpose of primary elections would be if party identification determined all of our voting behavior — we would simply default to any candidate because their personal differences don’t matter. Obviously, this is not the case.

All told, the literature says that party identification obviously matters to voters but that other factors matter as well. As we have seen, one of those factors is candidate characteristics. What’s the most obvious way that voters receive cues about candidate characteristics? The answer is obvious: their names.

That, in a nutshell, is our theory for why the absence of candidate names in polling would matter. Specifically, the evidence prompts me to post that the inclusion of a candidate name will decrease support for any given candidate. I turn to the polling from Public Policy Polling (PPP) and Patriot Majority USA to investigate.

District PVI Dem. Lead Poll Uses "Generic Dem."
AZ-02 14 0 No
CA-25 16 12 D
CA-48 3 10 D
CA-49 15 10 D
CO-06 18 -7 No
FL-26 13 14 D
IA-01 12 2 No
IA-03 3 -1 D
IL-06 14 10 D
ME-02 7 1 D
MI-06 8 -1 D
MI-11 6 0 No
MN-02 1 -1 No
MN-03 17 4 No
NE-02 5 9 No
NJ-02 1 5 Both
NJ-07 1 1 D
NJ-11 3 2 D
NY-19 5 6 D
NY-22 2 6 No
PA-06 11 -8 Both
PA-15 7 1 No
TX-07 5 10 D
TX-32 4 5 D
VA-10 17 9 D
WA-08 14 1 D
WI-01 9 -7 No
Source: Patriot Majority USA & Public Policy Polling

There are answers in the table above. The average Democratic margin in PPP’s polls where they specified two candidate names is 0 percentage points. That’s right — dead even. But where they used the “generic” label the Democratic candidate was up by 3% on average. Coincidence? We can consult a statistical test of the polling to test for the impact of the nameless specification for Democrats’ poll numbers while controlling for other things that could be helping/hurting their chances in a district (like the partisanship there, whether an open candidate is running, etc).

What we find is that the effect of polling House elections with a generic Democratic candidate increases the expected Democratic lead there by roughly 5%. In a midterm cycle where many elections will likely be decided by 5% margins, this difference can have a huge effect.

Suffice to say that we can consider two remedies here. First, we could subtract 5% from the Democratic margin in any poll where the candidate doesn’t have a name. That’s a little ad-hoc, though, and the small number of polls we currently have means that our estimate of the effect could be as little as half a percentage point or as large as 9% — the error in our estimate is simply too large to be useful in this manner. It’s possible that we could employ some sort of survey experiment (whereby we poll voters and randomly decide if they get the name of their Democratic candidate, then look at the difference) to get a much better estimate of the effect. But for now, we have 28 polls, and they tell us that something is up but not exactly how important that “something” is.

Rather than adjust all the numbers, it’s better to not use the generic candidate polls at all, or at least until we get closer to campaign season. If I were really careful, I wouldn’t include them in tables at all. Perhaps they will remain there if for nothing besides descriptiveness alone.

G. Elliott Morris avatar
About G. Elliott Morris
Elliott is a ("big") data journalist and data scientist who specializes in American politics, public opinion, and predictive analytics. He is a government, history, and computer science student at The University of Texas at Austin and worked previously for the Pew Research Center and Decision Desk HQ.
comments powered by Disqus