Understanding ANOVA and Interaction Effects using Python
Analyze the effects of the fictional snail performance supplement Snail Fuel on olympic snail athletes!
Let’s pretend we’ve been hired by the fictional snail energy drink and performance supplement company Snail FuelTM to conduct an analysis on the effectiveness of their product. They want to know if snails who drink their supplement can expect a significant increase in muscle mass. Anecdotally, Snail Fuel has heard that snails with red shells in particular have been able to make impressive gains using their product. If this can be proven true, Snail Fuel will change their marketing strategy accordingly.
We have set up a small clinical trial with 30 snails olympians split into 6 groups across two dimensions: shell color and placebo/Snail Fuel. Each snail has continued exercising as usual while drinking either a placebo fuel made of plain protein powder, or Snail Fuel. The snails were scored on % increase in muscle mass after one month.
We plan to analyze this data using a two-way ANOVA test – also known as an analysis of variance. The ultimate goal of an ANOVA is to explain variance in our sample and where the variance is coming from. In our case, in a two-way ANOVA, all variance can be explained from the following sources:
grand mean + row effect (first independent variable) + column effect (second independent variable) + interaction effect (interaction of first and second variable) + error
An ANOVA will be able to tell us how much of the variance in our sample can be attributed to each source, and whether that variance meets a threshold of significance.
Let’s generate a Pandas dataframe with the snail’s scores:
import pandas as pd
yellow_placebo = [2, 4, 3, 1, 5 ]
yellow_fuel = [4, 7, 6, 6, 6]
blue_placebo = [5, 6, 4, 2, 8 ]
blue_fuel = [5, 6, 4, 7, 8]
red_placebo = [1, 7, 5, 2, 4]
red_fuel = [11, 13, 12, 14, 10]
data = []
for i in yellow_placebo:
data.append((i,"yellow","N"))
for i in yellow_fuel:
data.append((i, "yellow", "Y"))
for i in blue_placebo:
data.append((i, "blue", "N"))
for i in blue_fuel:
data.append((i, "blue","Y"))
for i in red_placebo:
data.append((i, "red","N"))
for i in red_fuel:
data.append((i, "red","Y"))
columns = ['Percent_Muscle_Increase','Shell_Color','Snail_Fuel']
snail_fuel_data = pd.DataFrame(data,columns=columns)
head(snail_fuel_data)
We can place each snail’s score in a 3×2 grid according to which subgroup the snail falls into:
| Placebo | Snail Fuel | |
| Yellow | 2, 4, 3, 1, 5 | 4, 7, 6, 6, 6 |
| Blue | 5, 6, 4, 2, 8 | 5, 6, 4, 7, 8 |
| Red | 1, 7, 5, 2, 4 | 11, 13, 12, 14, 10 |
Before we do any calculations, we can visualize our data using a box plot. This will show the mean and range of our dependent variable (percent muscle increase) broken into groups by the categorical variable placebo/snail fuel. We can color each point based on the shell color variable as well.
plt.figure(figsize=(8,5))
import matplotlib.pyplot as plt
import seaborn as sns
colors = ["yellow", "blue","red"]
ax = sns.boxplot(x='Snail_Fuel', y='Percent_Muscle_Increase', data=snail_fuel_data, color='white')
ax = sns.swarmplot(x="Snail_Fuel", y='Percent_Muscle_Increase', data=snail_fuel_data, hue ='Shell_Color', palette=colors)
plt.show()
Let’s find the average % increase in muscle mass for each subgroup:
| Placebo | Snail Fuel | |
| Yellow | 2, 4, 3, 1, 5 mean = 3 | 5, 7, 6, 6, 6 mean = 6 |
| Blue | 5, 6, 4, 2, 8 mean = 5 | 5, 6, 4, 7, 8 mean = 6 |
| Red | 2, 7, 5, 2, 4 mean = 4 | 11, 13, 12, 14, 10 mean = 12 |
Let’s plot only the means of our data:
means = snail_fuel_data.groupby(['Shell_Color', 'Snail_Fuel'], as_index=False).mean()
plt.figure(figsize=(8,5))
colors = ["blue", "red","goldenrod"]
ax = sns.swarmplot(x="Snail_Fuel", y='Percent_Muscle_Increase', data=means, hue ='Shell_Color', palette=colors, s = 7)
plt.plot( 'Snail_Fuel', 'Percent_Muscle_Increase', data=means[(means.Shell_Color == "red")], marker='', color='red', linewidth=2)
plt.plot( 'Snail_Fuel', 'Percent_Muscle_Increase', data=means[(means.Shell_Color == "blue")], marker='', color='blue', linewidth=2)
plt.plot( 'Snail_Fuel', 'Percent_Muscle_Increase', data=means[(means.Shell_Color == "yellow")], marker='', color='goldenrod', linewidth=2)
plt.show()
Main effects
We can already see that red snails who drank Snail Fuel have the highest average % gain in muscle mass. It also looks like for all shell colors, those who drank Snail Fuel gained more muscle. How can we know the extent that having a red shell increases Snail Fuel’s efficacy if it appears that snails of every shell color saw benefits?
First, we need to know exactly what effects are coming from what variables. How much of an effect does drinking Snail Fuel vs. the placebo have on muscle mass? How much of an effect does shell color have on muscle mass? These are known as the main effects for each variable. We can find this by calculating a grand mean (the average % increase of muscle mass for all snails in the study) and the individual row and column means. The main effect for each row or column will be its distance from the grand mean.
Interaction effects
| Placebo | Snail Fuel | ||
| Yellow | 2, 4, 3, 1, 5 mean = 3 | 4, 7, 6, 6, 6 mean = 6 | Row 1 mean: 4.5 (effect = -1.5) |
| Blue | 5, 6, 4, 2, 8 mean = 5 | 5, 6, 4, 7, 8 mean = 6 | Row 2 mean: 5.5 (effect = -.5) |
| Red | 1, 7, 5, 2, 4 mean = 4 | 11, 13, 12, 14, 10 mean = 12 | Row 3 mean: 8 (effect = +2) |
| Col 1 mean: 4 (effect = -2) | Col 2 mean: 8 (effect = +2) | Grand mean = 6 |
The main effects for each variable will always add to zero.
We see that the main effect for having a red shell is +2, meaning a snail with a red shell, regardless of whether they were assigned the placebo or the Snail Fuel, would be expected to have a % increase in muscle mass 2 units greater than the average value for all snails.
Similarly, we see that the main effect for drinking Snail Fuel is also +2, meaning that regardless of shell color, if we know a snail was assigned to drink Snail Fuel, we would expect that snail’s % gain in muscle mass to be 2 units greater than the grand mean.
By this logic, we can theorize that a red-shelled snail who drank Snail Fuel would have a % increase in muscle mass 2 (red shell) + 2 (Snail Fuel) units greater than the grand mean.
Now that we know the main effects for our variables, we can calculate the expected means taking main effect into account for each subgroup by adding or subtracting the relevant variable effects from the grand mean:
| Placebo | Snail Fuel | ||
| Yellow | Expected mean: 2.5 | Expected mean: 6.5 | (effect = -1.5) |
| Blue | Expected mean: 3.5 | Expected mean: 7.5 | (effect = -.5) |
| Red | Expected mean: 6 | Expected mean: 10 | (effect = +2) |
| (effect = -2) | (effect = +2) | Grand mean = 6 |
Let’s compare the expected means with main effects to the observed means we already calculated:
| Placebo | Snail Fuel | |
| Yellow | Observed mean: 3 Expected mean: 2.5 difference: .5 | Observed mean: 6 Expected mean: 6.5 difference: -.5 |
| Blue | Observed mean: 5 Expected mean: 3.5 difference: 1.5 | Observed mean: 6 Expected mean: 7.5 difference: -1.5 |
| Red | Observed mean: 4 Expected mean: 6 difference: -2 | Observed mean: 12 Expected mean: 10 difference: +2 |
The difference between the observed mean and the grand mean + main effects for each subgroup is known as the subgroup’s interaction effect. In other words, a subgroup’s interaction effect accounts for the variance in the observed mean for that subgroup that is not explained by the row or column effects. It can also be denoted as the row X column effect.
Now we can see clearly that there is a +2 unit positive interaction of red shell color and Snail Fuel. That means that snails with red shells who drank Snail fuel had an average of 2% greater muscle gains than we would have expected based on the shell color and placebo/Snail Fuel main effects alone.
Mathematically, interaction effects must cancel each other out – so a positive interaction of red shell color and Snail Fuel also means a negative interaction of red shell color and placebo. Accordingly, we find that yellow and blue-shelled snails have negative interaction effects for Snail Fuel. On average, blue snails and yellow snails saw 1.5 and .5 fewer percentage point gains in muscle mass than expected.
Running a two-way ANOVA in Python using statsmodels
We can test for the significance of the main effects and interaction effects in our data by running a two-way ANOVA test using the statsmodels library. The code is quite simple and easy to read:
import statsmodels.api as sm
from statsmodels.formula.api import ols
#perform two-way ANOVA
model = ols('Percent_Muscle_Increase ~ C(Snail_Fuel) + C(Shell_Color) + C(Snail_Fuel):C(Shell_Color)', data=snail_fuel_data).fit()
sm.stats.anova_lm(model, typ=3)We have specified that the ANOVA should test the relation of the dependent variable (percent muscle increase) to the categorical variables “Snail_Fuel” and “Shell_Color” in addition to their interaction “C(Snail_Fuel):C(Shell_Color).” We have also specified that the model should use a type 3 sum of squares. This is the sum of squares methodology used when an interaction effect is thought to be present in the data. For more information, see this helpful article from Towards Data Science.
The output of our test is below:
Interestingly and perhaps not surprisingly, the model tells us that the only significant effect in our ANOVA is the interaction of shell color and snail fuel. The p values (PR(>F)) for the snail fuel and shell color variables on their own would not meet the <.05 standard of significance.
Given these results, we can say that for the total population of snails in the study, there was no statistically significant difference in muscle increase between those who drank Snail Fuel and those who drank a placebo energy drink. However, there was a statistically significant interaction between shell color and drinking Snail Fuel, such that snails with red shells saw larger gains in muscle mass than snails with yellow and blue shells.
If we wanted to dig even deeper, we can take a look at the model summary:
print(model.summary())
Digging into the model summary, we can discern that the only significant p value corresponds to a positive 4.583 t score for “C(Snail_Fuel)[T.Y]:C(Shell_Color)[T.Red].” That translates to a significant positive interaction of drinking snail fuel and having a red shell.
So it looks like the anecdotal evidence has proven true after all, at least in our very small sample.