R for Political Data Science Week 9: The “Strongest” Democrats and Republicans (That Ran for Office in 2018)

Compared to their states’ partisanships, these popular Democrats did better/worse than expected

By G. Elliott Morris

On Mar. 01, 2019  in R for Political Data US Politics R-Posts



This is part of a series of short posts about politics that seeks to show how we use data science to learn more about the real world. Follow along here.


I have repeatedly raised the point over the past few months that some ideas of which popular Democrats are “better” than others are based on evaluations that are (a) wrong or (b) hard to quantify. I want to define a measurement for this today.

Let’s use Beto O’Rourke, the 2018 candidate for US Senator from Texas, as a prime example. The former Congressman faced off against Ted Cruz last year and lost by about 2.3 percentage points — better than any statewide Democrat has done for a long, long time (since Ann Richards’ 1990 governor election, I think). He has been repeatedly been talked about as a possible candidate for the presidency in 2020. Some party officials want him to run for Senate again. Either way, he’s a strong Democrat. Or so we think.

One (imperfect) way we can assess O’Rourke and other candidates’ strengths is by comparing their 2018 win margins to some calculated benchmark in the state. One reasonable calculation is to take the official’s margin in 2018 election and subtract the partisan baseline of the state — a combined index of the past two presidential election margins — adjusted for the national environment. This is a useful heuristic for measure a candidate’s strength if they have a recent electoral record. Of course, it’s not the final answer; this method doesn’t account for things like incumbency advantages, statewide (rather than national) partisan lean, or challenger quality. But it is a generalizable measure, and in political analysis, those are typically good enough for making comparisons (our ultimate goal).

According to this formula, O’Rourke performed roughly two percentage points better than he would have expected. More importantly, he did 4 points better than the average statewide Democrats’ performance last November. We would infer from this that O’Rourke is slightly better than average, though I’d add that a four point difference certainly doesn’t qualify his as exemplary or anything — and again, there are caveats to this simple comparison. You can scroll through this table to see every Democrats’ over/under-performance of their benchmark:

source("~/setup_elliott.R")

# read in 2018 data
results_2018 <- read_csv("../../data_no_export/post/2019_03_01-2018-vs-partisanship/2019_03_01_2018_vs_partisanship.csv")


# compute partisan lean for each state
pres <- politicaldata::pres_results

pres <- pres %>% 
  filter(year %in% 2012:2016)

pres <- pres %>%
  select(state, year,dem,rep)

pres <- pres %>%
  gather(party,vote,3:4) %>%
  mutate(party = paste0(party,'.',year)) %>%
  select(-year) %>%
  spread(party,vote)

pres <- pres %>%
  mutate(partisan_lean = ((dem.2016 - rep.2016) - 0.021)*0.75 + ((dem.2012 - rep.2012) - 0.039)*0.25)


# benchmark in 2018 is partisan lean + 8.6 points for dems, so add that
pres <- pres %>%
  mutate(benchmark = partisan_lean + 0.086)


# join that with the 2018 results
results_benchmarked <-  results_2018 %>%
  left_join(pres,by='state')

# compute over/under-performance
results_benchmarked <- results_benchmarked %>%
  mutate(margin = dem-rep,
         performance = margin - benchmark) %>%
  arrange(desc(performance))


# look
medians <- results_benchmarked %>%
  group_by(office) %>%
  summarise(performance = median(performance))



# kable table
results_benchmarked %>%
  select(state, office, partisan_lean, performance) %>%
  mutate(partisan_lean = round(partisan_lean*100),
         performance = round(performance*100)) %>%
  knitr::kable()
state office partisan_lean performance
WV Senate -41 36
KS Governor -23 19
OK Governor -38 18
SD Governor -29 17
MT Senate -21 16
ND Senate -34 16
MN Senate 1 15
PA Governor -2 10
NM Senate 6 8
TN Senate -27 8
ME Senate 3 7
PA Senate -2 6
VT Senate 26 6
OH Senate -8 5
VA Senate 2 5
NY Senate 21 4
MO Senate -19 4
ID Governor -34 4
IN Senate -19 3
WI Senate -1 3
DE Senate 11 3
MN Governor 1 2
MN Senate 1 2
HI Senate 32 2
TX Senate -13 2
MI Governor 0 2
AK Governor -17 1
WY Senate -47 1
AL Governor -29 1
MD Senate 24 1
AZ Senate -7 1
NE Governor -27 0
NM Governor 6 0
CO Governor 2 0
SC Governor -16 -1
WY Governor -47 -1
CT Senate 12 -2
GA Governor -8 -2
NE Senate -27 -2
MI Senate 0 -2
TN Governor -27 -2
RI Senate 16 -3
IA Governor -8 -3
WA Senate 13 -4
ME Governor 3 -4
NV Senate 1 -5
OH Governor -8 -5
NV Governor 1 -5
FL Senate -3 -6
FL Governor -3 -6
WI Governor -1 -6
NY Governor 21 -6
IL Governor 14 -7
MA Senate 23 -8
TX Governor -13 -9
RI Governor 16 -9
MS Senate -19 -10
UT Senate -28 -10
CA Governor 26 -11
OR Governor 9 -11
NJ Senate 12 -11
HI Governor 32 -11
AR Governor -29 -13
NH Governor -1 -15
AZ Governor -7 -16
CT Governor 12 -18
MD Governor 24 -44
VT Governor 26 -49
MA Governor 23 -65

We can summarize this table as a density plot, annotating it to show Mr O’Rourke’s strength compared to the rest of the 2018 candidates:

# a histogram
gg <- ggplot(results_benchmarked, aes(x=performance, col=office, fill=office)) +
  geom_vline(xintercept=0,linetype=2,col='gray40') +
  geom_density(alpha=0.7) +
  geom_vline(data=medians, aes(xintercept=performance,group=office, color=office)) +
  geom_vline(aes(xintercept=0.04)) +
  annotate('text',x=0.043,y=2,label="O'Rourke",hjust=0) +
  scale_color_pander() + scale_fill_pander() +
  labs(title="Statewide Democrats Underperfomed Expectations in 2018",
       subtitle="Across America, Democratic candidates — especially for governor — lagged behind\nstates' environment-adjusted partisan leans",
       x="Democratic Margin Relative to Benchmark\n(Partisan Lean + National Margin)",
       y="Density",
       caption="Sourcs: Secretaries of State") +
  scale_x_continuous(labels = function(x){x*100}) +
  guides('fill' = guide_legend(title="Office"), color = FALSE)


preview(gg, themearg = theme(legend.position = 'top'))

As I said, O’Rourke can compete about two points ahead of the average Democrat, but the large variance in over-performances means that’s likely not significantly different. We cannot reliably characterize him as above-average. But even the average Democratic candidate for the US Senate did well last year. Maybe that’s the thing to take away from this.

As always, explore the code for this post if you’re into learning more about R, and if you’re just here for politics there’s a whole lot more to read on this blog.







comments powered by Disqus