The Art of Gerrymandering #IronViz

I’m still very much a Tableau newbie. My first ever visualization was a simple election choropleth last June. So this is the first Tableau Iron Viz contest I’ve had a chance to participate in. The challenge for the first feeder is, of course, to leverage the new spatial capabilities in Tableau 10.2. I first experimented with this new functionality right after its release, creating a visualization of the Roman Empire circa 14 AD in just a few minutes.

The connector itself was incredibly easy to use and required pretty much zero prior knowledge of spatial files. So, when I learned the theme of the feeder, I initially went back to this viz and considered doing something with historical empires and how they changed over time. Seemed like a cool idea, but I just couldn’t really find the story in it, so I moved on. I then considered doing something with fantasy worlds—Westeros, Essos, and Sothoryos (Game of Thrones) or Middle Earth (The Hobbit/Lord of the Rings). After all, I had just created a visualization mapping Frodo’s journey through Middle Earth, so this seemed like a cool idea. I really liked this idea and still do, but I didn’t pursue it very far. Ultimately, I wanted to do something that has relevance to our world.

Eventually, I came up with the idea of doing something on gerrymandering. If you’re not sure what gerrymandering is, I’ll give you a quick explanation taken directly from my viz.

In most states in the US, the responsibility for redrawing congressional districts is owned by none other than the state legislature. The majority party in state legislatures often abuse this right by redrawing the districts in a fashion that gives their party an ongoing numeric advantage over its opponents. This is called gerrymandering.

Even though this happens constantly and is often done for extremely dubious reasons, I’m not sure that many Americans really understand it and my guess is that even fewer non-Americans understand it or its impact on America’s Legislature. So, why not create a highly informational visualization on the subject.

The Shapefiles

To create the visualization, I would need shapefiles showing the Congressional Districts for each state from the beginning of the country’s history. Luckily for me, the UCLA website had shapefiles for every Congress from the very first to the 114^th(The 115^th Congress just started in January, but as there has not been time for redistricting to occur, I’ve not included it in my analysis). Unfortunately, all 114 were in separate shapefiles. Since Tableau does not yet have the ability to union multiple shapefiles, I had to turn to a GIS tool to do some preparation. I had never previously worked with such a tool, so this was going to be a bit challenging. I chose to use ArcGIS only because I had heard of it (though, from speaking to others, I may have been better off with QGIS). I loaded a few shapefiles into ArcGIS and started playing with options. To make this work, I’d need to:

1. Add a field to each file which would be used to identify the Congress # (alas, no such field existed in the files already).
2. Update the field to the proper Congress #.
3. Union all the shapefiles together into a single shapefile.

I was able to figure out how to do # 1 and # 2, but for the life of me, I just couldn’t figure out # 3. Luckily, last year’s Iron Viz champion, Curtis Harris, shared some wisdom on Twitter and got me straightened out. Without going into detail, the following needed to be performed on each shapefile to accomplish the steps above (these can, of course, be scripted to save time):

1. Use the “Add Field” tool to add a new field (I called it “Congress”).
2. Use the “Calculate Field” tool to update the value of the field.
3. Use the “Append” tool to append multiple files together.

I was able to complete 1 and 2 without issue, but ran into problems when I started appending the files. The issue was that the data field lengths differed from file to file. So, when I tried to append a field with length 50 to another shapefile containing the same field, except with length 45, it failed. So, I had to create an entirely new shapefile, create the attribute table, and add all fields with the maximum length of 254. I was then able to append all the shapefiles into that one master shapefile. Finally, I had a monstrosity of a shapefile containing every Congressional district for all of the 114 Congresses since the birth of the nation.

Onto Tableau…

The shapefile was huge (2 GB) so I crossed my fingers when I pulled it into Tableau. Though a bit slow, Tableau loaded it without any problems. I created a map with a slider for the Congress # and began clicking through. But I didn’t get very far before it just stopped showing any spatial information at all. Either there was a problem with the spatial connector or I had screwed something up in my ArcGIS work.

I took another look at my gigantic shapefile and noticed that it was exactly 2 GB—not close to 2 GB, but exactly 2 GB. That didn’t make sense, so I became suspicious, thinking that 2 GB must be a maximum file size for shapefiles. Sure enough, a quick Google search for “shapefile maximum size” led me to this:

Sure would have been nice if ArcGIS to have warned me when I reached this maximum. Unfortunately, it gave no warnings and just continued to append one file after another, reporting “success” each time. At this point, I had spent many hours with ArcGIS and had begun getting very frustrated. I would have liked nothing more than to have been able to spend this time in Tableau working on my visualization instead. (Note: This led me to create an “Idea” on the Tableau community to allow unioning of multiple shapefiles. If you think this would be a good addition to the product, then please go vote it up. You can find it here.)

But, I persevered. After all, I really didn’t need to show all 114 Congresses anyway. So I opted for the last 50 years, going back to the 90^thCongress, which began in 1967. I returned to ArcGIS and appended the 25 shapefiles for the 90^th through 114^th Congresses (the resulting shapefile was about 1 GB in size).

Onto Tableau…Take 2

I connected the shapefile to Tableau, created a simple map and everything worked perfectly!! I started doing some work with the data and organizing it into a story, but before doing too much work, I saved it to Public. Unfortunately, I ran into another problem at this point. When viewed on Public, I got this error:

I have never seen that before, but I knew Adam Crahen was having some similar problems so I reached out to him. He had already opened a ticket with Tableau Support and they were actively working on it. Fortunately, the great team at Tableau found a quick fix and will be introducing a permanent one in a future version. Big thanks to Sophie Sparkes and Kent Marten (and the rest of the Tableau team) for their diligence on the issue. And thanks to Adam for keeping me up to speed on the progress.

Is this Guy Ever Going to Shut Up?

Okay, I’m taking way too much time here—sorry, I just want to share some of the difficulties and challenges I experienced creating this viz. I’ll try to pick up the pace now.

With all of my technical issues corrected, I started building my story in Tableau. Since I felt that gerrymandering was not a well-known concept, I knew this visualization had to be very informational. I decided that the viz would have three main sections:

1. Explain the concept using both words and a simple visual.
2. Give a specific real-world example of gerrymandering.
3. Allow the user to explore gerrymandering further.

For the first part, I used a visual created by a gentleman named Stephen Nass, which explained how gerrymandering works. He had originally posted the visual on Facebook and it made its rounds on Reddit before being picked up by the Washington times. The original visual used red and blue, but in an attempt to remove any potential partisanship—since red and blue are generally associated with Republicans and Democrats, respectively—I changed mine to orange and green (yes, I know there’s a Green Party in the US, but just humor me…).

I considered creating these visuals completely in Tableau, but eventually decided that this would only add complexity to the visualization and would not really add any value. So, instead, I used my favorite graphics editor, PowerPoint (thanks Josh Tapley!!) to create the visuals, then used the images as custom shapes. And, having used orange and green in this visual, I decided to use it as my theme colors throughout the visualization.

For the second part, I chose to provide six maps of North Carolina, showing the changes over time to the 12^th district, which is generally considered the most gerrymandered district in the United States.

Finally, I created a section which allows the user to select a state, then analyze changes over time. By default, the visualization shows the 90^thCongress (1967-1968), the 102^nd Congress (1991-1992) and the 114^thCongress (2015-2016), but you can change this as needed in order to compare changes over time.

So, enough already! Here’s the finished visualization. Click on the image to go to the full viz on Tableau Public.

Update, March 25, 2017: Since first publishing this visualization, I made a few additional changes, which I'd like to note here. 

First, based on some feedback from Jeremy Poole, I added some demographic information to the congressional district tooltips. Since gerrymandering has historically had a disproportionate effect on areas with high minority populations, the tooltips will now show a breakdown of race, based on 2015 US Census estimates. 

In order to provide additional context, I have also updated the tooltips to show information on how each district voted in the 2008, 2012, and 2016 Presidential elections.

Finally, after adding Presidential election results, I felt it was necessary to explain how gerrymandering impacts Presidential elections. To address this, I've added a sidebar to the visualization. By hovering over the Presidential Seal, you will get some additional information on this subject.

Ken Flerlage, March 21, 2017

Twitter | LinkedIn | Tableau Public