R for Political Data Science Week 1: Polarization in the 115th Congress

On Jan. 04, 2019  in R for Political DataUS PoliticsR-Posts

This is part of a series of short posts about politics that seeks to show how we use data science to learn more about the real world. Follow along here.

The 115th Congress came to a close today as members elected in the November 2018 midterms were finally sworn in. Was the recent congress the most polarized yet? If trends in the phenomena are any indication, it should be. Let’s load up data from VoteView.com to find out. As a note, we’re only going to analyze the ideology of House members, but you could easily replicate the analysis with the Senate.

The VoteView data is a set of scores (called “DW-Nominate”) calculated for each legislator that measures how closely they vote to other members, scaling them along a spectrum from those who vote most with what we think of as liberals and those we think of as conservatives. The result is a measure of their ideology positioned as a comparison to other members.

library(tidyverse)
library(knitr)
library(kableExtra)

house_ideo <- read_csv("https://voteview.com/static/data/out/members/Hall_members.csv")

What we’re interested in is how far apart the median House member for either party has drifted over time. We can compute the median distance between the parties for each Congress in the dataset:

# filter the data just for Democrats (100) and Republicans (200), and recode
# also, only keep for congresses after 1960 (the 89th)
house_ideo <- house_ideo %>%
filter(party_code %in% c(100,200),congress>89) %>%
mutate(party = case_when(party_code == 100 ~ "Democratic",
party_code == 200 ~ "Republican"))

# compute median ideology for each year
ideo <- house_ideo %>%
group_by(congress,party) %>%
summarise(ideo = median(nominate_dim1,na.rm=T))

# create a measure of distance between the median member
distance <- ideo %>%
mutate(distance = abs(Democratic - Republican))

Our initial answer to the 2017-19 had the largest difference between the parties since at least 1900. The table of all differences is shown here.

# make a table
distance %>%
arrange(desc(distance)) %>%
kable(caption = "Distance Between Median Dem. and Rep. House Member Ideology") %>%
kable_styling(bootstrap_options =  "hover")
Table 1: Distance Between Median Dem. and Rep. House Member Ideology
congress Democratic Republican distance
116 -0.3950 0.5180 0.9130
115 -0.3960 0.5030 0.8990
114 -0.3970 0.4920 0.8890
113 -0.3920 0.4915 0.8835
112 -0.3980 0.4670 0.8650
110 -0.3795 0.4260 0.8055
109 -0.3920 0.4100 0.8020
107 -0.3845 0.4020 0.7865
108 -0.3825 0.4020 0.7845
111 -0.3495 0.4350 0.7845
106 -0.3845 0.3910 0.7755
105 -0.3840 0.3900 0.7740
104 -0.3810 0.3850 0.7660
103 -0.3385 0.3770 0.7155
102 -0.3200 0.3510 0.6710
100 -0.3205 0.3465 0.6670
101 -0.3210 0.3450 0.6660
99 -0.3200 0.3450 0.6650
98 -0.3170 0.3290 0.6460
97 -0.3200 0.3080 0.6280
96 -0.3225 0.2810 0.6035
93 -0.3505 0.2500 0.6005
94 -0.3385 0.2515 0.5900
92 -0.3440 0.2460 0.5900
91 -0.3445 0.2440 0.5885
95 -0.3240 0.2620 0.5860
90 -0.3420 0.2410 0.5830

And here’s what that distance looks like over time:

# make the plot
gg <- ggplot(distance, aes(x=congress,y=distance)) +
geom_line() +
labs(title="Polarization Between House Members, 1960-2018",
x="Congress",
y="Distance between Median Member's Ideology") +
theme_minimal()

preview(gg)

And here’s another look the data, using what we call a “ridge plot.”

library(ggridges)

gg <- ggplot(house_ideo,aes(x=nominate_dim1,y=as.factor(congress),fill=party)) +
ggridges::geom_density_ridges2(rel_min_height = 0.05,col=NA,scale=4,alpha=0.5) +
scale_fill_manual(values=c("Democratic"="blue","Republican"="red")) +
labs(title="Polarization Between House Members, 1960-2018",
x="Ideology (DW-NOMINATE)",
y="Congress") +
theme_minimal()

preview(gg)

From the ridge plot, we can come to two initial conclusions. One, that Republicans started the shift right before Democrats began theirs left, and two, that Democrats in 2017-19 were more centralized along the spectrum than Republicans.

There is one question of what exactly we’re measuring, though. Scholars have raised the point that analyses of roll call voting behavior, like DW-Nominate, are measuring more party loyalty than ideology. Perhaps partisan polarization is the real phenomena we’re looking at — members who are further to the left or right are simply those most averse to voting with the other party, without that being automatically liberal or conservative.

Either way, the level of polarization in Congress continued to rise from 2017 to 2019. Will it get any better from here?