This is part of a series of short posts about politics that seeks to show how we use data science to learn more about the real world. Follow along here.
The 115th Congress came to a close today as members elected in the November 2018 midterms were finally sworn in. Was the recent congress the most polarized yet? If trends in the phenomena are any indication, it should be. Let’s load up data from VoteView.com to find out. As a note, we’re only going to analyze the ideology of House members, but you could easily replicate the analysis with the Senate.
The VoteView data is a set of scores (called “DW-Nominate”) calculated for each legislator that measures how closely they vote to other members, scaling them along a spectrum from those who vote most with what we think of as liberals and those we think of as conservatives. The result is a measure of their ideology positioned as a comparison to other members.
library(tidyverse) library(knitr) library(kableExtra) # read in the data house_ideo <- read_csv("https://voteview.com/static/data/out/members/Hall_members.csv")
What we’re interested in is how far apart the median House member for either party has drifted over time. We can compute the median distance between the parties for each Congress in the dataset:
# filter the data just for Democrats (100) and Republicans (200), and recode # also, only keep for congresses after 1960 (the 89th) house_ideo <- house_ideo %>% filter(party_code %in% c(100,200),congress>89) %>% mutate(party = case_when(party_code == 100 ~ "Democratic", party_code == 200 ~ "Republican")) # compute median ideology for each year ideo <- house_ideo %>% group_by(congress,party) %>% summarise(ideo = median(nominate_dim1,na.rm=T)) # create a measure of distance between the median member distance <- ideo %>% spread(party, ideo) %>% mutate(distance = abs(Democratic - Republican))
Our initial answer to the 2017-19 had the largest difference between the parties since at least 1900. The table of all differences is shown here.
# make a table distance %>% arrange(desc(distance)) %>% kable(caption = "Distance Between Median Dem. and Rep. House Member Ideology") %>% kable_styling(bootstrap_options = "hover")
And here’s what that distance looks like over time:
# make the plot gg <- ggplot(distance, aes(x=congress,y=distance)) + geom_line() + labs(title="Polarization Between House Members, 1960-2018", x="Congress", y="Distance between Median Member's Ideology") + theme_minimal() # add my theme preview(gg)
And here’s another look the data, using what we call a “ridge plot.”
library(ggridges) gg <- ggplot(house_ideo,aes(x=nominate_dim1,y=as.factor(congress),fill=party)) + ggridges::geom_density_ridges2(rel_min_height = 0.05,col=NA,scale=4,alpha=0.5) + scale_fill_manual(values=c("Democratic"="blue","Republican"="red")) + labs(title="Polarization Between House Members, 1960-2018", x="Ideology (DW-NOMINATE)", y="Congress") + theme_minimal() preview(gg)
From the ridge plot, we can come to two initial conclusions. One, that Republicans started the shift right before Democrats began theirs left, and two, that Democrats in 2017-19 were more centralized along the spectrum than Republicans.
There is one question of what exactly we’re measuring, though. Scholars have raised the point that analyses of roll call voting behavior, like DW-Nominate, are measuring more party loyalty than ideology. Perhaps partisan polarization is the real phenomena we’re looking at — members who are further to the left or right are simply those most averse to voting with the other party, without that being automatically liberal or conservative.
Either way, the level of polarization in Congress continued to rise from 2017 to 2019. Will it get any better from here?
- R for Political Data Science Week 11: Beto O'Rourke is a Media Sweetheart
- R for Political Data Science Week 10: What If Each State Allocated Their Electoral College Votes Proportionally?
- R for Political Data Science Week 9: The "Strongest" Democrats and Republicans (That Ran for Office in 2018)