“Uncertainty is the only sure thing… perhaps,” Eric Maskin, a winner of the Nobel Prize in economics, once said. This is perhaps the most important concept we can understand for analyzing the especially special election for Alabama’s Senate seat previously held by Attorney Jeff Sessions. Will Republican Roy Moore — who is accused of sexual misconduct with teenage girls beat Democrat Doug Jones in the Alabama Senate race? As I wrote last week, almost anything could happen on Tuesday.
There are a few sources of clues as to what will happen in Alabama. This post will go through the public opinion polling in the race, the “fundamentals” indicators that may be telling, and the county-by-county expectations for Roy Moore and Doug Jones. Traditionally, our best be at benchmarking would be with public opinion polls. So what do they say?
What the Polls Say:
One of the best tools we have for measuring where Roy Moore and Doug Jones stand is public opinion polling. Applying the normal approach to Alabama’s Senate election (averaging the polls over the last 10 days and weighting by sample size) we see that Roy Moore is ahead by nearly three percentage points. Notably, that’s roughly where things have stood since Moore’s sexual assault scandal settled in late November.
My average of polls has Roy Moore leading by 2%. Though there are enough polls to say for sure that Roy Moore is ahead in the average, it is “almost impossible” to say whether those polls will actually match the winner of Tuesday’s contest. This is, of course, not the first time that would happen (see: 2016 election). It is also entirely possible that Roy Moore exceeds his average and wins by more than we expect. November’s general election contests in Virginia and New Jersey had nearly 10 points of combined polling error when all the votes were counted. And historically that is not unprecedented.
Using polls of all Senate elections since 1998, I find that the average error in projected Democratic win margin is plus-or-minus 4-5%. That means that even half of a normal polling error would be large enough to see Doug Jones elected to the U.S. Senate. Here’s what that error has looked like for every year:
We can use that error to construct a margin of error for the polling average. Based on my estimates (which combine the root-mean-square error of the polls with the corresponding confidence interval — 95%), the margin of error for an average of trial-heat polls of Senate elections is plus-or-minus 12%. That means that anything from a 14% Moore win to a 10% Jones win is possible 95% of the time. Of course, outcomes closer to our 2% average are much more likely: a 5% Moore win is much more likely than a 10-point triumph, for example.
And there is at least some evidence to believe that an extreme error is possible. Because polls have to reach a representative sample of voters, not just Alabamians, there is one huge hurdle pollsters have to jump over: guessing who will vote. The tool with which pollsters adjust their numbers to match those estimates is called a turnout model, and in Alabama, nobody really knows what the turnout is going to be.
That’s because pollsters don’t really have a good benchmark for turnout in Alabama. There were no exit polls for the state during the 2016 election, for example — and even if there were there would be no guarantee that voters will turn out like they did in 2016. Again recall the Virginia gubernatorial election earlier this year: educated white votes exceeded expectations of turnout by enough to give a 5% boost to Democrat Ralph Northam that voters weren’t really expecting. Turnout models are messy, in summary, and that could have significant consequences for who wins in Alabama.
And since “many people” have asked for it, here is a trend estimate that includes ALL TEN SurveyMonkey scenarios and ALL THREE Monmouth turnout models.— Charles Franklin (@PollsAndVotes) December 11, 2017
I DO NOT recommend using this as it gives equal and large weight to 13 non-independent estimates. But FOR FUN ONLY: /end pic.twitter.com/p28SpE7DBO
Political Scientist Charles Franklin has gamed out what this might mean for the range of outcomes possible for Moore and Jones. Depending on who the polling firms SurveyMonkey and Monmouth University estimate are going to turn out, they find anything between a 10 point Moore win and a 9 point Jones win. Notably, Mark Blumenthal of SurveyMonkey followed up on his firm’s experiment and believes we should probably expect a slightly bluer election than the average of polls may indicate. Placing these different polls individually into the average I find that the polling average would not change much, however, and even including the bluest hypothetical polls only move the average to Moore +0.2%. Still, this would change our expectations from a race where Moore has a small, though important, edge to one where our best guess would be an almost exact tie between the candidates.
Sadly though, there is no way for us to know which turnout model is the best; ultimately there is no comparable election to the 2017 Alabama special on (voter)file. We just have to accept the uncertainty…
…or do we?
There are other ways to estimate who is going to win an election, after all. Otherwise, we would have been walking around aimlessly before pollsters started asking about Senate races (though, to be fair, our direction is a tad aimless right now). Let’s consult the “fundamentals.”
What the “Fundamentals” Say:
What I mean when I say “fundamentals” is a mixture of state partisanship (measured by votes in presidential elections), national mood (measured by the generic ballot), and past Senate vote shares. These can be combined together to create estimates of party vote share in elections since 1998 (the maximum time span for our polling dataset).
The predictions are, well, not great. The average error for these predictions is more than 10% and the r-squared (a measure of how well we can predict an outcome given some subset of information — in this case, partisanship, the generic ballot, past senate vote, and the party/presence of an incumbent) is just 78%. Recall that the average error for Senate polls is less than 5% (with an r-squared of 93%), so we’re certainly not getting a “better” prediction here.
However, that is true as long as you define “better” as more precise, or better performing over time. But I can think of one reason you may think a fundamentals projection is “better” in the Alabama Senate race: you don’t believe the polls. As discussed, there are some serious methodologically difficulties in polling the Moore vs Jones matchup. If you don’t believe the polls are right, you can use this projection. Notably, though there is error, the projection of a 14-point Moore win is more in line with what we would expect given the very, very red environment in Alabama (the state voted for Donald Trump by 28%, for example.)
That being said, the fundamentals are very suspect in this election which is, at the very best, “unique.” Roy Moore’s sex scandal, combined with the very pro-Democratic national environment, say that the usual rules don’t apply in Alabama. At the very least we should correct the fundamentals projection with the usual impact of a sex scandal. Political scientists have determined the cost a Moore-like wrongdoing to be, on average, five percentage points. Based on this alone one might think that the fundamentals model would now predict a 9% Moore victory, not a 14% win.
One could also make the case that, since the impact of Moore’s scandal is captured in polls of the race, we should combine that information with the fundamentals of the Senate seat to get a better estimate of the outcome. If you told me that, you’d be on track: a combination of fundamentals data and polls, on average, has 4 points of error on average but explains 94% of the variation in outcomes of Senate elections. That’s almost as high as we can home to achieve without making a model that is far too confident about the outcome (a phenomenon called “overfitting”).
This method produces an estimate of the Alabama Senate race that says Moore should win by 6% or so. Of course, there is a margin of error for this estimate too: 12%, in fact — the same as using polls alone! Below, I collect all three different forecasts we’ve made for Alabama Senate:
Notably, all three forecasts for Alabama Senate point to a Moore victory, though with different degrees of certainty and with differing margins of error. The moral of the story: we don’t know who is going to win the race for Alabama Senate, though Roy Moore seems to be positioned better to win the seat (with anywhere between a 60 and 80% chance of victory).
A fun note here, since we’re talking about probabilities now: There is a widely used theory in probability and statistics, called Bayes’ Theorom, that helps us to asess the chance an event happens when we don’t have a good handle on what the true probability will be. Bayes Theorom posits that we combine a set of prior expectations (uniform priors) with the observed data we have now. The formula in the case of Alabama is rather simple. We simply divide the number of times we think Roy Moore would win out of ten elections, plus 1, by those ten elections plus 2. I think a reasonable estimate is our poll-based probability (60% Moore), so (6+1)/(10+2) = 58%. Bayes’ Theorem has the effect of shifting our expectations towards the more uncertain side. Take with this what you will in determining who may win Tuesday evening.
But if you’re like me, that’s not enough… you want to know who is going to win the election as soon as possible. We have a pretty good technique for figuring that out as the votes roll in.
What to Watch For
Per usual, one of the best thing we can do to gauge how the race is unfolding on election night is to compare Moore/Jones’s performance against a set of benchmarks. These benchmarks are created by taking the vote margin that various Republicans earned at the county-level, minus the share that they earned statewide. For example, if Roy Moore wins Mobile County by 10% but wins the state by 2% the Republican lean of Mobile County will be 8%. Moore’s benchmark is simply an average of partisan performance in state-level elections since 2012 (with extra weight given to the 2016 presidential election).
I have created both a graph and scrollable table for these benchmarks (below).
|County||Moore 2012 (AL Supr. Court)||Romney 2012 (Pres.)||Shelby 2016 (Sen.)||Trump 2016 (Pres.)||Moore's Benchmark|
What we see is that there are a few counties in Alabama that consistently vote more Republican than the rest, some that lean more Democratic, and others that are almost always right in the middle. We call these counties “bellwether” counties, and they indicate roughly how the election is going to go. If Doug Jones is consistently beating his marks in Colbert, Coosa, Lee, and Talladega counties, we might be in for an upset election. Below, I have zoomed in on Moore’s benchmark for the contest:
Throughout the night, I’ll be updating our estimates of Moore and Jones’s county-by-county overperformance via the DecisionDeskHQ Twitter feed. Early on, we expect to get returns mostly from small, pro-Moore counties. Though the margins won’t likely be extremely surprising, if Moore is consistently beating/missing his benchmarks we’ll have a clue about what to look for when the big counties start reporting. I’ll also have a similar model running for our live precinct-level results. We hope you’ll tune in!