Module 4: Comparing 3+ Groups

Fill-in-the-blank coding exercises for Module 4: ANOVA and the Kruskal-Wallis Test
Author

Haley Grant

ANOVA

ANOVA (Analysis of Variance) extends the two-sample t-test to three or more independent groups, testing whether any of the group means differ.

Conditions:

  • Random samples from each population
  • Independent samples (both within and across groups)
  • Equal variances across groups (a bigger concern here than in the two-sample case!)
  • Each group: \(n > 30\), or normally distributed in the population

\[H_0: \mu_1 = \mu_2 = \dots = \mu_k \qquad \text{vs.} \qquad H_A: \text{at least one } \mu_i \text{ differs}\]

Motivating example: Suppose the hypertension trial was expanded to test a second drug (“Drug B”) alongside Drug A and Placebo—a 3-arm trial with 80 patients per arm. We want to know: does average SBP change differ across the three treatment groups?

Creating a Factor Variable

Fill in the code below to create a factor column, group, from group_num (1 = Drug A, 2 = Drug B, 3 = Placebo).

# create a labeled factor variable
htn3 = htn3 %>%
  mutate(group = factor(, levels = c(1, 2, 3),
                         labels = c(, , )))

head(htn3)
# create a labeled factor variable
htn3 = htn3 %>%
  mutate(group = factor(group_num, levels = c(1, 2, 3),
                         labels = c("Drug A", "Drug B", "Placebo")))

head(htn3)

Now group has three labeled levels instead of the numeric codes.

Plotting and Summarizing the Data

Fill in the code below to create a boxplot of sbp_change by group, and calculate the sample size, mean, and sd within each group.

# boxplot
htn3 %>%
  ggplot(aes(x = , y = , fill = group)) +
  geom_boxplot(alpha = 0.75) +
  theme_bw() +
  theme(legend.position = "none")

# summary statistics (saved for later use)
group_stats = htn3 %>%
  group_by() %>%
  summarise(n = n(), mean = mean(sbp_change), sd = sd())
group_stats
# boxplot
htn3 %>%
  ggplot(aes(x = group, y = sbp_change, fill = group)) +
  geom_boxplot(alpha = 0.75) +
  theme_bw() +
  theme(legend.position = "none")

# summary statistics (saved for later use)
group_stats = htn3 %>%
  group_by(group) %>%
  summarise(n = n(), mean = mean(sbp_change), sd = sd(sbp_change))
group_stats

You should see Drug A with the most negative (largest decrease) mean change, Drug B intermediate, and Placebo closest to 0. We’ve saved this as group_stats so we can reuse it below, rather than recalculating.

Calculating the F-statistic by Hand

Let’s calculate the F-statistic directly from group_stats (the real sample sizes, means, and standard deviations we just computed from the data).

The ANOVA F-statistic compares the variance between groups to the variance within groups:

\[F = \dfrac{MSB}{MSW} = \dfrac{SSB / (k-1)}{SSW / (N-k)}\]

The code below walks through the calculation using the columns of group_stats—run it to see the grand mean, SSB, and SSW for our data.

# total sample size and number of groups
N = sum(group_stats$n)
k = 3

# grand mean (weighted average, here all n's equal so just the average)
grand_mean = sum(group_stats$n * group_stats$mean) / N
grand_mean

# sum of squares between groups
SSB = sum(group_stats$n * (group_stats$mean - grand_mean)^2)
SSB

# sum of squares within groups
SSW = sum((group_stats$n - 1) * group_stats$sd^2)
SSW

Since these are calculated directly from our simulated htn3 data, your exact numbers will be close to (but not identical to) a grand mean around -8.7, with Drug A pulling the grand mean down and Placebo pulling it up.

Now use SSB and SSW to calculate MSB, MSW, and the F-statistic.

# mean squares
MSB = SSB / ()
MSW = SSW / ()

# F statistic
F_obs = MSB / MSW
F_obs
# mean squares
MSB = SSB / (k - 1)
MSW = SSW / (N - k)

# F statistic
F_obs = MSB / MSW
F_obs

[1] 25.7227

Compared to a critical value of about 3.03 (from an F distribution with 2 and 237 degrees of freedom at \(\alpha = 0.05\)), you should find your observed F-statistic is far larger—strong evidence that at least one group mean differs from the others. (Share your exact number with me once you’ve run this, and I can update this solution text to match exactly, the same way we did for the two-sample t-test file!)

Using lm() and anova()

In R, we typically fit ANOVA as a linear model and then use anova() to get the F-test.

Fill in the code below to fit a linear model of sbp_change predicted by group, and print the ANOVA table.

# fit model and get ANOVA table
model = lm( ~ , data = htn3)
anova(model)
# fit model and get ANOVA table
model = lm(sbp_change ~ group, data = htn3)
anova(model)

Since our hand calculation above was computed directly from this same data (rather than from hypothetical summary statistics), the F-statistic and p-value here should match our hand-calculated F_obs almost exactly.

Post-Hoc Comparisons

A significant ANOVA only tells us some group differs—it doesn’t say which. For that, we need pairwise comparisons, adjusted for the fact that we’re running multiple tests (to control the overall Type I error rate).

Fill in the code below to get all pairwise comparisons between groups, with a Bonferroni correction.

# pairwise comparisons with Bonferroni correction
model_emm = emmeans(model, pairwise ~ , adjust = )
model_emm$contrasts
# pairwise comparisons with Bonferroni correction
model_emm = emmeans(model, pairwise ~ group, adjust = "bonferroni")
model_emm$contrasts

Given how clearly separated the three group means are in group_stats, we’d expect all three pairwise comparisons to come out significant even after the Bonferroni correction: Drug A vs. Drug B, Drug A vs. Placebo, and Drug B vs. Placebo.

Why do we need to adjust for multiple comparisons when running all pairwise tests, instead of just using an unadjusted significance level of 0.05 for each one?

Factor vs. Numeric (Warning!)

This is an important gotcha: if your grouping variable is stored as a number instead of a factor, R will (silently and incorrectly) treat it as a continuous predictor instead of a categorical one!

Fill in the code below to fit the incorrect model using the numeric group_num instead of the factor group, and compare its ANOVA table to the correct one.

# INCORRECT: treating group as numeric
model_incorrect = lm( ~ , data = htn3)
anova(model_incorrect)
# INCORRECT: treating group as numeric
model_incorrect = lm(sbp_change ~ group_num, data = htn3)
anova(model_incorrect)

This model now has only 1 degree of freedom for group_num instead of 2—because R is fitting a straight line through the numeric codes 1, 2, 3, rather than estimating a separate mean for each group. This is almost never what you want for a categorical predictor!

Why does it matter that group_num has only 1 degree of freedom in the incorrect model, while the properly-coded group factor has 2? What is R actually assuming about the relationship between the numeric codes and the outcome in the incorrect model?

Kruskal-Wallis Test

The Kruskal-Wallis test is the nonparametric alternative to ANOVA—it’s based on ranks (like the Wilcoxon tests), rather than means, and doesn’t require a normality assumption.

Fill in the code below to visualize the shape of sbp_change within each group, faceted by group.

# check shape within each group
htn3 %>%
  ggplot(aes(x = , fill = )) +
  geom_histogram(color = "black", alpha = 0.75) +
  facet_wrap(~ group, nrow = 3) +
  theme_bw() +
  theme(legend.position = "none")
# check shape within each group
htn3 %>%
  ggplot(aes(x = sbp_change, fill = group)) +
  geom_histogram(color = "black", alpha = 0.75) +
  facet_wrap(~ group, nrow = 3) +
  theme_bw() +
  theme(legend.position = "none")

With n = 80 in each group, we’d be comfortable with ANOVA regardless, but this is good practice for whenever sample sizes are smaller or shapes look questionable.

Fill in the code below to run the Kruskal-Wallis test comparing sbp_change across the three levels of group.

# Kruskal-Wallis test
kruskal.test( ~ , data = htn3)
# Kruskal-Wallis test
kruskal.test(sbp_change ~ group, data = htn3)

Like ANOVA, Kruskal-Wallis only tells us whether some group differs—we’d still need pairwise follow-up tests to see which ones.

Fill in the code below to run pairwise Wilcoxon rank-sum tests between all three groups, with a Bonferroni correction.

# pairwise Wilcoxon tests with Bonferroni correction
pairwise.wilcox.test(x = htn3$, g = htn3$,
                     p.adjust.method = )
# pairwise Wilcoxon tests with Bonferroni correction
pairwise.wilcox.test(x = htn3$sbp_change, g = htn3$group,
                     p.adjust.method = "bonferroni")

Given how clearly separated the three groups are in our simulated data, we’d expect all three pairwise comparisons to come out significant here too, similar to the ANOVA/emmeans results.

ANOVA and Kruskal-Wallis led to the same overall conclusion in this example. In general, when might you prefer Kruskal-Wallis over ANOVA?