GBBO continued: an interactive Plotly jitter plot

How to chart baker performance across all seasons?

As part of the good old Great British Bake Off (GBBO) project, I wanted to create a chart that would succinctly capture every baker’s performance in every season. First, this necessitated the invention of a “score” feature. On the actual show, no one gets a numerical score. However, on the (very helpful) wikipedia pages for each season, fans have recorded which bakers were favorites, least favorites, Star Bakers and eliminated each episode (read this post to learn more about how I wrangled the data from wikipedia tables). This information can be used to create our own numerical representation of baker performance. I decided to give each baker 1 point for “favorite”, -1 point for “least favorite”, and 2 points for being named the Star Baker.

To compare every baker’s performance in every season, I summed all bakers’ scores to get their “final score” when they were eliminated (or when they won). For each season, a dot plot comparing each baker’s final score could look something like this:

Combining all seasons would give you something like this:

Full code here

The dot-plot-specific portion of the code is below:

geom_dotplot(binaxis='y', stackdir='center',
             aes(fill = winflag)
             ,dotsize = .5
             ,alpha = .8
             ,stackgroups = TRUE)

We can see that the points at around 0 Final Score start to run into each other when stacked, and things begin to look a little cluttered. This is where the jitter plot comes in.

Jitter plots

The jitter plot strikes me as a strange being in the world of data visualization. Unlike almost all other plots, the jitter plot adds an element of chaos. It forgoes absolute precision for the sake of readability.

In essence, a jitter plot is just a dot-plot for situations when there are too many dots for stacking. Instead of spacing the dots equally apart from each other without overlap, a jitter plot “jitters” the dots in a random manner, within a given area.

Using geom_jitter() instead of geom_dotplot():

The jitter-plot-specific portion of the code is below:

geom_jitter(height = .1,width = .25,
            aes(color = winflag), 
              alpha =.8,
              size = 4) 

The points here are less precise, but flow better. There is some overlap, but the randomness makes it easier to tell points apart, and a sense of the density of the points is not lost.

The height and width variables determine how much wiggle room the jitter plot has to work with in the x and y dimensions. smaller numbers would mean a tighter radius and more overlapping. Bigger numbers would give a wider radius to jitter within.

It is possible to achieve jittering while using geom_dotplot() using position = position_jitter(width = ?, height = ?), though there are other advantages of using geom_jitter(). We will use geom_jitter() from here on out for this post.

One advantage to geom_jitter is that it is easy to change dot attributes based on a group variable (not so with geom_dotplot – this is because there is no size or shape variable built into the parameters, only “dotsize”). To highlight winners and runners up in my jitter plot, I was able to change the code as follows:

Full code here

To change size based on a categorical variable, I added size as an attribute to the geom_jitter aesthetics. I then had to add a scale_size_manual() function to define the size for each category. If you have multiple aesthetics that you want to show up in one legend, t’s important to define the same name (even if it’s blank) for the all of the aesthetic functions:

geom_jitter(height = .1,width = .25,
              aes(color = winflag,
                  size = winflag), 
              alpha =.8) +
scale_color_manual(name = "",
                     values = c('Winner' = 'goldenrod',
                                'Runner-up' = '#86dba5',
                                'Eliminated before final' = '#e68a95')) +
scale_size_manual(name = "",
                    values = c('Winner' = 5,
                               'Runner-up' = 4,
                               'Eliminated before final' = 2))

The trouble with labels

I’m sure you noticed that there were helpful data labels in my single-season example. If we were to add the same labels to our jitter plot, we would get this beauty:

Clearly, there are too many observations here to have both intelligible labels and visible points. Wouldn’t it be nice if we could have an interactive plot that would allow a user to choose a point and see more information dynamically? We can!

Plotly

Using Plotly, we can turn our static plot into a dynamic plot that provides much more information upon hover or click:

Charts or apps can be made completely in Plotly, or we can use the ggplotly() function to turn a plot originally made in ggplot into an interactive Plotly visualization.

On a very basic level, all I did to turn our jitter plot into a Plotly plot was save the jitterplot and apply the ggplotly() function. The full code to get the formatted, interactive Plotly visualization is below:

#save fully formatted ggplot jitterplot

p <- ggplot(jitter, aes(season, endsum), group = baker) +
  geom_jitter(height = .1,width = .25,
              aes(color = winflag,
                  size = winflag,
                  text = paste('Baker:', baker,
                               '<br>Status:', winflag,
                               '<br>Max Episode:', maxep,
                               '<br>Final Score:', endsum)), 
              alpha =.8
  ) +
  scale_x_continuous(limits = c(1.8,12.2), breaks=seq(2,12,by=1)) +
  scale_y_continuous(limits = c(-4,13), breaks=seq(-4,13,by=2)) +
  coord_flip() +
  geom_vline(xintercept=2.5, color = "gray30", linetype = "dashed", size = .5) +
  geom_vline(xintercept = 3.5,color = "gray30", linetype = "dashed", size = .5) +
  geom_vline(xintercept = 4.5,color = "gray30", linetype = "dashed", size = .5) +
  geom_vline(xintercept=5.5,color = "gray30", linetype = "dashed", size = .5) +
  geom_vline(xintercept = 6.5,color = "gray30", linetype = "dashed", size = .5) +
  geom_vline(xintercept=7.5,color = "gray30", linetype = "dashed", size = .5) +
  geom_vline(xintercept = 8.5,color = "gray30", linetype = "dashed", size = .5) +
  geom_vline(xintercept=9.5,color = "gray30", linetype = "dashed", size = .5) +
  geom_vline(xintercept=10.5,color = "gray30", linetype = "dashed", size = .5) +
  geom_vline(xintercept = 11.5,color = "gray30", linetype = "dashed", size = .5) +
  labs(x = "Season", y = "Final Score") +
  theme_minimal() +
  theme(
    panel.grid.major.x = element_blank(),
    panel.grid.major.y = element_blank(),
    axis.text.x = element_text(family = "Arial"),
    text = element_text(size = 14, family = 'Arial')
  ) +
  scale_color_manual(name = "",
                     values = c('Winner' = 'goldenrod',
                                'Runner-up' = '#86dba5',
                                'Eliminated before final' = '#e68a95')) +
  scale_size_manual(name = "",
                    values = c('Winner' = 5,
                               'Runner-up' = 4,
                               'Eliminated before final' = 2)) 

#apply ggplotly() function to our plot, specify the text for the tooltip, remove the toolbar and format the legend

ggplotly(p,tooltip = "text") %>%
  config(displayModeBar = F) %>%
  layout(legend = list(orientation = "v", 
                       xanchor = "center", 
                       x = 1,
                       y=.3,
                       bordercolor = "#edd99f",
                       borderwidth = 2,
                       bgcolor = "#ffdbfa",
                       font = list(
                         family = "Arial",
                         size = 14,
                         color = "#000")))

Only one tweak had to be made to the original ggplot jitter plot code to get the Plotly visualization to work correctly. This was the addition of the “text” aesthetic in the geom_jitter() function:

text = paste('Baker:', baker,
                               '<br>Status:', winflag,
                               '<br>Max Episode:', maxep,
                               '<br>Final Score:', endsum)

You’ll notice that in the ggplotly() function, the tooltip parameter was set to “text”.

ggplotly(p,tooltip = "text") 

This code is crucial in defining what text the user will see when hovering over a point.

The other additions to the ggplotly code are aesthetic. I use config(displayModeBar = F) to remove the toolbar that automatically gets added to Plotly plots (not necessary, I just don’t like how it looks). The layout() function is used to create the custom legend.

And there you have it: making a static plot dynamic was that easy!

ggplot’s geom_tile… not just for heat maps!

If you’ve read my previous blog post, you’ll know that I was able to convince a group of unsuspecting peers (ok, maybe one was suspecting) to create a final project Shiny app all about the Great British Bake Off (GBBO) using data I had previously scraped and wrangled from Wikipedia.

An unexpected challenge that came up while making the app was conceptualizing and creating charts or visualizations that displayed descriptive information about the GBBO. The ability to plot two variables to view a relationship has been drilled so deeply into my head that at this point, it’s second nature. This makes it easy to think of ideas for bar charts or line graphs that compare measures of baker performance. The ability to visually display information that isn’t necessarily trying to prove a point is not drilled into a science major’s head at all. Simple information like each episode’s theme is also important to summarize visually, and counter-intuitively more difficult to plot in a way that adds value.

geom_tile

As someone who works with SQL tables endlessly day after day, I think I naturally gravitate toward organizing and understanding information in a grid. This, to me, is the beauty of geom_tile.

Before making this app, I had only ever used geom_tile to create heat maps. Based on my google searching, this is probably what geom_tile is used for 99% of the time. A standard heat map compares two categorical variables with one on each axis, forming a grid. The squares in the grid are most often colored based on the value of a continuous variable, like temperature, or perhaps something like age:

ggplot(aes(as.factor(episode),as.factor(season.x), fill = avg.age)) +
  geom_tile(color = "white",
            lwd = .5,
            linetype = 1) +
  scale_fill_gradient(low = "pink", high = "brown", name = "Avg. Age") +
  geom_text(aes(label = round(avg.age,0)), color="white", size=rel(4)) +
  xlab("Episode") + 
  ylab("Season") +
  ggtitle("Average Baker Age by Season and Episode") +
  theme_minimal() +
  theme(panel.grid.major = element_blank())

*I made this as a quick example but it’s actually quite interesting. You can see that for many seasons, there is a tendency for average age to decrease as a season progresses – meaning younger bakers make it further in the competition. You can also see that some seasons have older or younger bakers in general (for example season 10 vs. season 5).

Though heat maps are usually colored with a continuous variable, they also work if you make the continuous variable discrete, or use a categorical variable:

colorpal <- colorRampPalette(c('lightsalmon','royalblue'))(5)

ggplot(aes(as.factor(episode),as.factor(season.x), fill = agegroup)) +
  geom_tile(color = "white",
            lwd = .5,
            linetype = 1) +
  geom_text(aes(label = round(avg.age,0)), color="white", size=rel(4)) +
  xlab("Episode") + 
  ylab("Season") +
  ggtitle("Average Baker Age by Season and Episode") +
  theme_minimal() +
  theme(panel.grid.major = element_blank()) +
  scale_fill_manual(values = colorpal, name = "Age Group") 

*This version conveys the same information as the original, but it’s simplified and maybe slightly easier to pick up on the main point of the chart.

Plotting GBBO episode themes using geom_tile

In the Great British Bake Off, each episode in a season has a theme. Some of these themes are repeated season after season, such as Cake, Biscuits, and Bread, while other themes are unique to a season. Unique themes tend to fall into basic types, like countries (Japan, Germany, Denmark etc.) or ingredients (Chocolate, Dairy, Spice etc.). I wanted to find a way to show which themes are repeated across all seasons and which themes are unique, as well as when a theme occurs in each season, in one concise visual.

We will use a dataset that I scraped called weekthemes, which has four fields: season, week, week_theme and category. I coded theme categories myself:

Let’s start with a basic geom_tile with seasons on one axis and episode theme on the other:

weekthemes %>%
  mutate(season = as.factor(season)) %>%
  
  ggplot(aes(season, week_theme)) + 
  geom_tile(size = .5) 

Not very beautiful without formatting, but a start. Notice that the themes are organized alphabetically. That’s not necessarily bad, but it’s also not the most logical way to organize themes. We could perhaps organize them by category, popularity, or something else. I know from watching GBBO that certain themes usually occur around the same time each season – for instance, cake week is almost always the first episode, while (obviously) the final is always last.

Wrangling week themes to order the y axis:

This is where good old data wrangling comes in. I find that it’s easiest to order categorical variables by sorting the variable’s factor levels before putting the data into ggplot.

First, let’s create a variable that sorts themes by their average episode across all seasons they occur:

weekthemes %>%
  group_by(week_theme) %>%
  summarize(avgweek = mean(week)) %>%
  mutate(ranking = rank(avgweek, ties.method = 'first')) %>%

Now that we have a ranking for each theme, we can join this back to the original weekthemes dataset, and sort the week_theme variable by our new ranking variable. Let’s also use geom_text to add a data label to our chart, so that we can see which week each theme occurs in each season:

weekthemes %>%
  group_by(week_theme) %>%
  summarize(avgweek = mean(week)) %>%
  mutate(ranking = rank(avgweek, ties.method = 'first')) %>%
  inner_join(weekthemes) %>%
  mutate(week_theme = factor(week_theme, levels= unique(week_theme[order(desc(ranking))]))) %>%
  mutate(season = as.factor(season)) %>%
  
  ggplot(aes(season, week_theme)) + 
  geom_tile(size = .5) +
  geom_text(aes(label = week), color="white", size=rel(4)) 

This already feels more organized, and the chart now tells a slightly different story. The eye is drawn to the top of the chart, where the viewer can clearly see that Cake week is almost always first. As you navigate down the chart, other themes that tend to occur in the middle of the seasons become more unique and less common. Pâtisserie is almost always second to last and the Final of course last.

Adding another categorical variable:

To break up the large number of themes on the y axis, I want to add another level of organization with an additional category variable. I coded this variable myself for a few reasons. The first reason is that over the course of GBBO, certain themes have morphed into similar themes. Take for instance Pastry; in earlier seasons, there was no Pastry week. Seasons 2 and 3 had a Tarts episode and a Pies episode. In seasons 4 and 5, Tarts and Pies appear to have morphed into “Tarts and Pies,” and a separate Pastry episode was introduced. From season 6 on, only Pastry remains. Though these themes are all slightly different, they all deal with the same basic category of short crust pastries.

The second reason I decided to code a category variable was to organize unusual themes together. As the seasons of GBBO have progressed, certain patterns in themes have arisen. Take for example country themes. There have been several seasons with a week devoted to the baking of one country, though the same country has never been repeated in multiple episodes. The country themes are ideologically related, though not technically the same. I wanted a way to group these themes together.

Let’s add Category as the fill variable to our chart:

mycolors <- colorRampPalette(c('#fa8ca9','#ffdbfa','lightgoldenrod','#cce0ff',"#d4b7a9"))(12)

weekthemes %>%
  group_by(week_theme) %>%
  summarize(avgweek = mean(week)) %>%
  mutate(ranking = rank(avgweek, ties.method = 'first')) %>%
  inner_join(weekthemes) %>%
  mutate(week_theme = factor(week_theme, levels= unique(week_theme[order(desc(ranking))]))) %>%
  mutate(season = as.factor(season)) %>%
  
  ggplot(aes(season, week_theme, fill=category)) + 
  geom_tile(size = .5) +
  geom_text(aes(label = week), color="white", size=rel(4)) +
  scale_fill_manual(values = mycolors, name = "Category") 

Immediately, the addition of the category variable to color the tiles adds another level of structure to the chart. The chart becomes more interactive, as viewers can now choose to examine themes across seasons by category, looking for patterns, similarities or differences.

Ultimately, for this chart, I decided to change the sorting of the themes to go by theme category, to emphasize each category more than individual themes. I accomplished this by changing the ranking variable to group by category rather than theme:

weekthemes %>%
  group_by(category) %>%
  summarize(avgweek = mean(week)) %>%
  mutate(ranking = rank(avgweek, ties.method = 'first')) %>%
  inner_join(weekthemes) %>%
  mutate(week_theme = factor(week_theme, levels= unique(week_theme[order(desc(ranking))]))) %>%
  mutate(season = as.factor(season)) %>%
  
  ggplot(aes(season, week_theme, fill=category)) + 
  geom_tile(size = .5) +
  geom_text(aes(label = week), color="white", size=rel(4)) +
  scale_fill_manual(values = mycolors, name = "Category") 

Sorting the themes by category so that all themes in the same category are next to each other allows viewers to see which themes fit into which category much more easily, while still getting a sense of which themes tend to occur at specific times in a season. I preferred this sorting method, but there is no right or wrong answer here – it would have been equally valid to level the themes as they were before.

Formatting and refining

The only thing left is to refine the formatting of the chart. There is truly no limit here, but for easy understandability I made the following changes to get my final product:

mycolors <- colorRampPalette(c('#fa8ca9','#ffdbfa','lightgoldenrod','#cce0ff',"#d4b7a9"))(12)

weekthemes %>%
  group_by(category) %>%
  summarize(avgweek = mean(week)) %>%
  mutate(ranking = rank(avgweek, ties.method = 'first')) %>%
  inner_join(weekthemes) %>%
  mutate(week_theme = factor(week_theme, levels= unique(week_theme[order(desc(ranking))]))) %>%
  mutate(season = as.factor(season)) %>%
  
  ggplot(aes(season, week_theme, fill=category)) + 
  geom_tile(color = 'gray20', size = .5) +
  scale_fill_manual(values = mycolors, name = "Category") +
  scale_x_discrete(position = "top",
                   labels=c("2" = "S2", "3" = "S3",
                            "4" = "S4", "5" = "S5",   
                            "6" = "S6", "7" = "S7", 
                            "8" = "S8", "9" = "S9", 
                            "10" = "S10", "11" = "S11", 
                            "12" = "S12")) +   
  labs(color = "Category") +
  geom_text(aes(label = week), color="black", size=rel(5)) +
  xlab("") + 
  ylab("") +
  ggtitle("Great British Bake Off Week Themes Season by Season Comparison") +
  theme(panel.grid.major.y = element_line(color = "#edd99f"),
        panel.grid.major.x = element_blank(),
        panel.grid.minor = element_line(),
        panel.border = element_rect(fill=NA,color="white", size=0.5, linetype="solid"),
        axis.line = element_blank(),
        axis.ticks = element_blank(),
        panel.background = element_rect(fill="white"),
        plot.background = element_rect(fill="white"),
        legend.text = element_text(size=12),
        legend.title = element_text(size=14),
        title = element_text(size =14),
        axis.text = element_text(color="black", size=14))

It might not be the prettiest possible version, but I was going for complete readability here.

In conclusion

geom_tile is an incredibly versatile tool to plot much more than just heat maps. I had a lot of fun hacking it to create this informative chart, and will certainly use it again for future charting needs!

Code and data used in this post can be found here.

Mapping California wine regions using Shapefiles and tmap

California’s Napa Valley is only about an hour’s drive from Berkeley, where I grew up. Even as a sub-drinking age child, I was often given a good reason to spend a day in or around Napa. Hiking, visiting a friend’s grandparents with a pool, driving my younger sister to artistic rollerskating practice (a story for another time). Add to this a family-wide obsession with the early 2000’s classic Sideways and frequent trips through the Santa Ynez Valley for some classic Solvang aebelskiver, and you get the recipe for a lifelong love of the California Wine Country.

My partner works in the world of fancy beverages, and has recently started cataloguing the wines he’s tasted. There are some good apps available (Vivino) for tracking and reviewing wines, and generating predictions based on taste profiles. However, he has found that Vivino falls short when it comes to geographically tracking specific wines and varietals. He wants to be able to see which regions and subregions he has tried wines from in an interactive map format rather than a list. I agree that this would be more interesting. Though after doing my own research, I can see why Vivino lacks a robust mapping feature. There is very little reliable geographic data freely available for world wine regions online. I think the primary reason for this is probably gatekeeping, because the information exists, it’s just in very expensive published books. Who would have guessed that there would be snobbery surrounding data on wine 🙂

The good news is that there is some high quality data available, at least for US wine regions, thanks to the UC Davis library: here. It’s a really cool project – they’ve catalogued viticultural areas for California and other wine producing states in the US in great detail. I decided to start exploring this data by creating my own wine region map for California. I also used this little project as an opportunity to explore some of R’s mapping features with the tmap package.

Get CA Viticultural Area Polygons via Shapefile

The first step in plotting any kind of area on a map is getting the actual data that holds information about its shape. This can come in several forms – raster (pixels, like a JPEG) or polygons (area boundaries are stored as formulas for a polygon, i.e. vectorized like a PDF). Polygon areas are commonly stored as either GeoJSON files or Shapefiles. Either can be easily read into R using a package called rgdal. The polygons I wanted to use for my map come as either GeoJSON or Shapefiles.

I chose to use the Shapefile to import the polygons into R. When you download a Shapefile, it will come as a folder with several files like this:

Don’t worry about what exactly all these separate files are. All you need to do to import the full Shapefile into R is run a very simple line of code using the readOGR function from the rgdal package:

cawine <- readOGR(dsn = "Desktop/CA_avas_shapefile", layer = "CA_avas")

All I did here was add the path to the Shapefile folder, and specify the layer name, which is the name of all the files within the folder. This will get you a “Large SpatialPolygonsDataFrame” in the object space, which looks like this when you click it:

You’ll notice that the data and polygons object both have 142 items, one for each viticultural area. When you expand the data object, you can see that it is formatted as a data frame with columns that provide information and additional context for the polygons:

If you expand the polygons object, you’ll see that each item contains coordinate information for a specific viticultural area:

Plot the areas with tmap

Now that we have the viticultural area Shapefile loaded, we can begin to plot the wine regions using the tmap package:

library(tmap)

tm_shape(cawine) +
   tm_fill("name",legend.show = FALSE, alpha = .5) +
   tm_borders()

tmap works like ggplot in that plots are built by adding function layers on top of each other. Each plot needs a tm_shape() object, which provides the data for the layers that follow. tmap will send back an error if you call only tm_shape() without at least one layer following it. Here I’ve called tm_shape() using our cawine shapefile, added a fill layer with tm_fill() based on the name of each area, and added a border for each area with tm_borders(). The output looks like this:

We’re getting somewhere, but obviously the areas are floating in space with no background map for context. There are several ways we can fix this, and which one we use depends on the end goal. Perhaps the most straightforward fix is to add an additional Shapefile layer with the shape of California to our plot. This will require finding a Shapefile for the outline of California. I just googled this, clicked on one of the first links, here, and downloaded the state boundary file. I added this Shapefile as a base layer for the viticultural areas like so:

ca <- readOGR(dsn = "Desktop/CA-state-boundary", layer = "CA_State_TIGER2016")

tmap_mode("plot")
  tm_shape(ca) +
    tm_fill("white") +
    tm_borders()  +
  tm_shape(cawine) +
    tm_fill("name",legend.show = FALSE, alpha = .5) +
    tm_borders() +
  tm_layout(bg.color = "#cfe4e8") 

This doesn’t look bad, though there is a bit of an awkward edge around the coastline for some reason – this could potentially be a result of the viticultural area Shapefile being mapped to a different projection than the California Shapefile. To check this, I changed the map projection for both Shapefiles to the Equal Earth projection like so:

tmap_mode("plot")
  tm_shape(ca, projection = 8857) +
    tm_fill("white") +
    tm_borders()  +
  tm_shape(cawine, projection = 8857) +
    tm_fill("name",legend.show = FALSE, alpha = .5) +
    tm_borders() +
    tm_layout(bg.color = "#cfe4e8") 

A fascinating new shape for California! We see that the edge on the coastline is still there, so we can safely assume that this is by design. In fact, I believe it is because the California Shapefile extends partially into the ocean territory that belongs to the state.

This static map is nice to get a broad overview of the regions, or to build an attractive map that could be printed out, but if we wanted more detailed information or additional context about the viticultural areas, it would be worth our while to construct a more dynamic map.

Make the map interactive

Making a map interactive using tmap is surprisingly simple. To get started, all you need to do is change the mapping mode to “view” using tmap_mode():

tmap_mode("view")
  tm_shape(ca) +
    tm_fill("white") +
    tm_borders()  +
  tm_shape(cawine) +
    tm_fill("name",legend.show = FALSE, alpha = .5) +
    tm_borders() 

Clearly there is some refining to do here, but this is a great start toward building interactivity that took essentially no work. If you click on the layers icon you can toggle between different default Esri base maps, and add or remove the ca and cawine Shapefile layers.

In the next iteration of our interactive map, I’ll add more information in the popup labels using the popup.vars argument in our fill layer. I also changed the id argument to use the name variable so that the nicely formatted viticultural area name is displayed upon hover. I removed the California Shapefile, as it was no longer needed, and added a custom tm_basemap() layer with Stamen’s TonerLite map:

tmap_mode("view")
  tm_shape(cawine) +
    tm_fill("name",legend.show = FALSE, alpha = .5,
            id="name",
            popup.vars = c("Within" = "within",
                           "Counties" = "county",
                           "Created" = "created")) +
    tm_borders() +
  tm_basemap("Stamen.TonerLite")

Now we have a map that actually provides a good deal of information on the geographic location of all the viticultural areas in the Shapefile dataset. Of course there are infinite ways to iterate on this basic map, but for my purposes, simply visualizing where each region is and getting some basic information displayed is enough, and I’m pleased with the simplicity of the final product. Stay tuned for more wine data exploration in the future!

Code and data used to create the maps in this tutorial are here

Visualizing Winter Olympics success

This semester has been busy! I haven’t had a chance to dive into the side projects I’ve been brewing but I’ve really been enjoying the data visualization class I’m taking with Thomas Brambor. Our first assignment was to build some visualizations using historical Winter Olympics data going up to 2014, using ggplot and Plotly. I had a lot of fun doing this assignment and discovered some very cool tips and tricks in the process of perfecting my charts. If you’re viewing this on a phone – sorry, the Plotly charts do not take kindly to that. The full assignment with code is below:

Visualizing gender-neutral baby names with ggplot and Plotly

Visualizing gender-neutral baby names with ggplot and Plotly

I’m finally taking a much-anticipated (by me) class for my MA program called “Data Visualization.” An optional exercise was to play around with a dataset of baby names from US census data. I had some fun creating this interactive chart of the most popular gender-neutral baby names over time.

Names included in this chart must have been in the top 10% of all names for a given year, with a boy:girl or girl:boy ratio of no more than 100:1.

The design of this chart exposes patterns in the predominant sex of a given name over time. Interestingly, it looks like a majority of popular baby names move from a higher ratio of boys to girls to a lower ratio over time. There are many more fascinating insights to find!

The code I wrote to generate this chart is below:

library(babynames)
library(ggplot2)   
library(magrittr)   
library(dplyr)  
library(RColorBrewer)
library(colorways2) #my color package
library(ggthemes)

f <- babynames %>% filter(sex=="F")
m <- babynames %>% filter(sex=="M")

unisex1 <- merge(f,m ,by=c("name","year"),all = TRUE)

base1 <- unisex1 %>%
  group_by(year) %>%
  mutate(overall=n.x+n.y) %>%
  mutate(ratio= n.y/n.x) %>%
  arrange(desc(ratio)) %>%
  mutate(logratio=log(ratio)) %>%
  mutate(overallcentile = ntile(overall,10)) %>%
  filter(tolower(name) != "unknown") %>%
  filter(tolower(name) != "infant") %>%
  filter(tolower(name) != "baby") %>%
  filter(overallcentile >=  10) %>%
  filter(abs(logratio) <= 2) 

d <- highlight_key(base1, ~name)

#had to make a new palette out of an existing one with 74 colors, one for each name
nb.cols <- 74
mycolors <- colorRampPalette(ballpit)(nb.cols)

p <- ggplot(d, aes(year, logratio, col= name)) + 
  geom_hline(yintercept=0, linetype="dashed", color = "black") +
  geom_line() + 
  theme_tufte() +
  geom_point() + 
  scale_y_continuous(labels = c("1:100", "1:10", "1:1","10:1","100:1")) +
  labs(title="Gender Distribution of Most Popular Gender-Neutral Names Over Time", x ="", y = "Boy:Girl ratio (log scale)") +
  theme( text=element_text(family="Helvetica",size = 14),plot.title = element_text(size = 14),axis.text = element_text(size = 12), axis.title = element_text(size = 14))+
  scale_x_continuous(breaks = round(seq(min(1880), max(2020), by = 10),1)) +
  scale_color_manual(values = mycolors) 
  
gg <- ggplotly(p)   
  
highlight(gg, dynamic = F, color = "black",selected = attrs_selected(showlegend = FALSE)) %>% 
 layout(margin = list(b = 40)) %>%
 layout(legend=list(title=list(text='')))

Designing functions to generate and display color palettes in R

Designing functions to generate and display color palettes in R

In this post, I will go over the methodology I used to design the color palettes and functions to display them that comprise the colorways package referred to in my previous blog post.

Identifying the Problem

When it comes to color, I definitely believe in being adventurous and having many options based on mood and context. I think visualizations are best when they take an artistic and sometimes unexpected approach to color. Though blending in is sometimes necessary (for publication, or serious work-related presentations, maybe), when I work on projects for myself, I like to push the envelope a bit. It’s an aspect of data analysis that I truly enjoy.

I’ve found that palettes from RColorBrewer or default palettes work fine for certain situations, like heat maps that need a spectrum of color. But where they break down for me is in the fairly common occurrence of graphing several distinct subgroups. This calls for palettes with colors that are different enough from each other that no two subgroups are perceived as more similar than the others. RColorBrewer has a few “qualitative palettes” for this purpose, but I find them to be a bit generic.

Of course, RColorBrewer isn’t the only package for color palettes out there. This website has compiled a ton of color palette packages in one place, with a handy tool to select palettes. The palettes can then be retrieved in R using the Paletteer package, which is incredibly useful for accessing color palettes from many sources in a standardized way. Now that I found it, I will certainly be saving this resource and using it in the future. Some examples of favorites I’ve found from this site:

The current crop of color packages still lack some features that in my opinion are essential to choosing a palette. As far as I know, they don’t come with easy to use, built in color shuffling, and they don’t come with dynamic color previews in charts or graphs.

So, I set out to create my own package with three distinct functionalities:

  1. Save my own palettes
  2. Display any palette (either native or from another package) in a variety of forms (basic palette, charts, graphs)
  3. Shuffle color palettes if desired and save the newly ordered list of colors

Saving my own palettes

This is just as easy as choosing colors and putting them in lists. The fun part of course is naming the palettes based on my own whimsy. Some examples:

krampus <- c("#0782A6","#A66507","#994846","#CDD845","#624FAF","#52735D","#BBAE92","#FED2E7","#FFE402")
ballpit <- c("#5C33FF","#FF8E07","#E2E442","#42E44F","#C67CF9","#F64EBC","#ACF64E" ,"#C11736","#00B6A0")
donut <- c("#FA88F1","#2DEC93","#8FE2FF","#FF882B","#D80D0D","#D0A321","#369830","#B681FF","#858585")

I decided to go with 9 colors in each palette for uniformity and to maximize novel color combinations when shuffling. I ordered the palettes intentionally with my favorite color combinations come at the beginning, so that when the palettes aren’t shuffled, optimal color combinations are preselected.


Paletteprint: view all palettes

The first thing I wanted to do was write a function that would display all the palettes at once. To do this, I had to first create a list of palettes and their names:

palettes <- list(dino, hive, rumpus, taffy, sleuth, martian, krampus, tulip, donut, donette, creme, farmhand, mayhem, ballpit, january, pair1)

names(palettes) <- c("dino", "hive", "rumpus", "taffy", "sleuth", "martian", "krampus", "tulip", "donut", "donette", "creme", "farmhand", "mayhem","ballpit","january","pair1")

Then I wrote a function using rectangles in base R graphing to make a chart:

paletteprint <- function() {
  bordercolor <- "black"
  x <- c(-8,27)
  y <- c(0,(length(palettes)*3))
  i <- (length(palettes)*3)
  n <- 1
  plot(1, type="n", xlab="", ylab="", xlim=x, ylim=y,axes=FALSE, frame.plot=FALSE)
  for (p in palettes) {
    rect(0,i,3,i-1, col = p[1], border = bordercolor, lwd = 2)
    rect(3,i,6,i-1, col = p[2], border = bordercolor, lwd = 2)
    rect(6,i,9,i-1, col = p[3], border = bordercolor, lwd = 2)
    rect(9,i,12,i-1, col = p[4], border = bordercolor, lwd = 2)
    rect(12,i,15,i-1, col = p[5], border = bordercolor, lwd = 2)
    rect(15,i,18,i-1, col = p[6], border = bordercolor, lwd = 2)
    rect(18,i,21,i-1, col = p[7], border = bordercolor, lwd = 2)
    rect(21,i,24,i-1, col = p[8], border = bordercolor, lwd = 2)
    rect(24,i,27,i-1, col = p[9], border = bordercolor, lwd = 2)
    text(x = -8, y = i-.5, # Coordinates
         label = names(palettes[n]), pos =4)
    i = i-3
    n= n+1
  }
}

The output when calling paletteprint() looks like this:


Colordisplay: view, shuffle and choose the number of colors in a palette

Next I wanted a function that would display the colors in a selected palette, display them in a straightforward way, allow me to choose how many colors I wanted to see, shuffle the colors if desired, and return a list of the colors displayed.

colordisplay <- function(palette, number = 9, bordercolor = "black", shuffle = "no") {

  if (shuffle == "yes"){
    shuff <- sample(seq(from = 1, to = length(palette), by = 1), size = length(palette), replace = FALSE)
  }

  else {
    shuff <- seq(1, length(palette), by=1)
  }

  if (number == 9) {
    names = c(palette[shuff[1]], palette[shuff[2]],palette[shuff[3]],palette[shuff[4]],palette[shuff[5]],palette[shuff[6]],palette[shuff[7]],palette[shuff[8]],palette[shuff[9]])
    title <- paste(names, collapse = ", ")
    x <- c(0,3)
    y <- c(7,10)
    plot(1, type="n", xlab="", ylab="", xlim=x, ylim=y,axes=FALSE, main = title, frame.plot=FALSE)
    rect(0,10,1,9, col = palette[shuff[1]], border = bordercolor, lwd = 4)
    rect(1,10,2,9, col = palette[shuff[2]], border = bordercolor, lwd = 4)
    rect(2,10,3,9, col = palette[shuff[3]], border = bordercolor, lwd = 4)
    rect(0,9,1,8, col = palette[shuff[4]], border = bordercolor, lwd = 4)
    rect(1,9,2,8, col = palette[shuff[5]], border = bordercolor, lwd = 4)
    rect(2,9,3,8, col = palette[shuff[6]], border = bordercolor, lwd = 4)
    rect(0,8,1,7, col = palette[shuff[7]], border = bordercolor, lwd = 4)
    rect(1,8,2,7, col = palette[shuff[8]], border = bordercolor, lwd = 4)
    rect(2,8,3,7, col = palette[shuff[9]], border = bordercolor, lwd = 4)

    return(title)
  }

  else if (number == 8) {
    names = c(palette[shuff[1]], palette[shuff[2]],palette[shuff[3]],palette[shuff[4]],palette[shuff[5]],palette[shuff[6]],palette[shuff[7]],palette[shuff[8]])
    title <- paste(names, collapse = ", ")
    x <- c(0,4)
    y <- c(8,10)
    plot(1, type="n", xlab="", ylab="", xlim=x, ylim=y,axes=FALSE, main=title, frame.plot=FALSE)
    rect(0,10,1,9, col = palette[shuff[1]], border = bordercolor, lwd = 4)
    rect(1,10,2,9, col = palette[shuff[2]], border = bordercolor, lwd = 4)
    rect(2,10,3,9, col = palette[shuff[3]], border = bordercolor, lwd = 4)
    rect(3,10,4,9, col = palette[shuff[4]], border = bordercolor, lwd = 4)
    rect(0,9,1,8, col = palette[shuff[5]], border = bordercolor, lwd = 4)
    rect(1,9,2,8, col = palette[shuff[6]], border = bordercolor, lwd = 4)
    rect(2,9,3,8, col = palette[shuff[7]], border = bordercolor, lwd = 4)
    rect(3,9,4,8, col = palette[shuff[8]], border = bordercolor, lwd = 4)

    return(title)
  }

# and so on until number == 2

Ok, yes, there might have been a better way to set up a loop that doesn’t require that I write a new if statement for every number of colors BUT I did want control of how the display looks for each number of colors chosen so honestly I don’t think it’s really that needlessly wordy.

If I call colordisplay, choose my palette and use all default parameters, the result will look like this (fully zoomed out):

colordisplay(january)

I will also get this as output:

[1] "#F1F07C, #BB7EEE, #98EAC8, #65859C, #8C2438, #ADA99D, #AD840C, #398726, #EC5570"

If I call colordisplay with shuffle on, I get a randomly shuffled output with the corresponding list of colors:

colordisplay(january, shuffle = "yes")
[1] "#AD840C, #98EAC8, #8C2438, #EC5570, #F1F07C, #65859C, #BB7EEE, #398726, #ADA99

Changing the number parameter between 2-8 colors will result in the following shapes:

You can also choose to display a palette from another package, or using paletteer:

colordisplay(paletteer_d("nationalparkcolors::DeathValley"), number = 6, shuffle = "yes")
[1] "#E79498FF, #73652DFF, #F7E790FF, #514289FF, #B23539FF, #FAB57CFF"

Colorbar: view a palette as a bar chart:

I wanted to write a function that would allow me to input a palette, the number of colors I want from that palette, and whether I want the colors to be shuffled, and output a sample bar chart. This is the main functionality that I feel current color packages lack. Of course, you can keep your own code for a sample bar chart and test colors manually, but this function has made it so much easier to quickly assess whether a set of colors will work for a visualization. I also think it’s a ton of fun to shuffle the colors again and again and see what comes up!

The code behind this function is pretty similar to what I wrote for colordisplay. Full code for all functions can be found in this github repo.

Let’s try colorbar with the tulip palette and default parameters:

colorbar(palette = tulip)

We can add up to 7 colors in the bar chart:

colorbar(palette = tulip, number = 7)

We can choose to stack the bars:

colorbar(palette = tulip, number = 7, stacked = "yes")

We can shuffle the colors:

colorbar(palette = tulip, number = 7, stacked = "yes", shuffle = "yes")

Like colordisplay, colorbar will always output the ordered list of colors for each chart:

[1] “#D7DDDE, #A25E5F, #FFC27E, #FFFBC7, #9B8054, #E0E38C, #E0C2F6”

Lastly, we can use palettes from other packages:

colorbar(palette = paletteer_d("wesanderson::IsleofDogs2"), number = 4)

Colorscatter: view a palette as a scatter plot:

Colorscatter is virtually the same function as colorbar, but instead of a bar chart, color scatter will display a scatter plot. The only difference in parameters between the two is that color scatter lacks the “stacked” feature, for obvious reasons.

Let’s try the default colorscatter function using the “ballpit” palette:

colorscatter(palette = ballpit)

Like colorbar, we can view up to 7 colors using colorscatter:

colorscatter(palette = ballpit, number = 7)

We can also shuffle colors:

colorscatter(palette = ballpit, number = 5, shuffle = "yes")

Colorscatter will also return an ordered list of colors each time it is run:

[1] “#C11736, #E2E442, #5C33FF, #00B6A0, #ACF64E”

And of course like colorbar, colorscatter will accept external palettes:

colorscatter(palette = paletteer_d("tvthemes::simpsons"), number = 5, shuffle = "yes")

In conclusion:

I’ve certainly only scratched the surface of what’s out there and what’s capable in the world of R color packages. I’m happy with the package I’ve put together but I can already think of some improvements or additional functionalities I’d like to add (for example, a parameter for switching the charts to dark backgrounds). For now I think this is a great start and I’m excited to do more research and make more updates!

For full code, visit the colorways package GitHub repo here