Module 3: Inference for Numerical Outcomes, Part III

Fill-in-the-blank coding exercises for Module 3: Paired t-Tests
Author

Haley Grant

Paired t-tests

A paired t-test is used when two measurements are taken on the same subjects (e.g., before and after treatment), rather than on two independent groups. Instead of comparing two separate samples, we compute the difference for each subject and run a one-sample t-test on those differences, testing whether the average difference is 0.

Conditions:

  • Random sample of pairs/subjects
  • Independent pairs (the differences themselves are independent across subjects)
  • \(n > 30\), or the differences follow a normal distribution in the population

\[H_0: \mu_d = 0 \qquad \text{vs.} \qquad H_A: \mu_d \ne 0 \ \ (\text{or} \ <, \ >)\]

where \(\mu_d\) is the population mean of the paired differences.

In our hypertension trial, each patient has both a baseline and a follow-up SBP measurement—the same patient, measured twice. That makes sbp_change (follow-up minus baseline) a natural candidate for a paired analysis: did SBP change significantly from baseline to follow-up, within each treatment group?

Visualizing Paired Data

Fill in the code below to create a boxplot of sbp_change, faceted by group, with a horizontal reference line at 0 (no change).

# boxplot of change in SBP, faceted by group
htn_trial %>%
  ggplot(aes(y = )) +
  geom_boxplot() +
  geom_hline(yintercept = , linetype = 2, color = "red") +
  facet_wrap(~ ) +
  theme_bw()
# boxplot of change in SBP, faceted by group
htn_trial %>%
  ggplot(aes(y = sbp_change)) +
  geom_boxplot() +
  geom_hline(yintercept = 0, linetype = 2, color = "red") +
  facet_wrap(~ group) +
  theme_bw()

The Drug A group’s box should sit clearly below the dashed zero line (a decrease), while the Placebo group’s box should be closer to (but likely still somewhat below) zero.

Splitting the Data by Group

Fill in the code below to create two separate data frames: data_a (Drug A only) and data_p (Placebo only).

# split into Drug A and Placebo data frames
data_a = htn_trial %>% filter(group == )
data_p = htn_trial %>% filter(group == )
# split into Drug A and Placebo data frames
data_a = htn_trial %>% filter(group == "Drug A")
data_p = htn_trial %>% filter(group == "Placebo")

Now we have a separate data frame for each group, which will make the rest of our code simpler.

Drug A: Test by Hand

We will now calculate the test statistic for the Drug A group:

Fill in the code below to calculate the test statistic for a paired t-test on the Drug A group’s change scores.

# Drug A summary statistics
diff_a = mean(data_a$sbp_change)
sd_a = sd(data_a$sbp_change)
n_a = nrow(data_a)

# print values
diff_a ; sd_a ; n_a

# test statistic
t_a =  / ( / sqrt())
t_a
# Drug A summary statistics
diff_a = mean(data_a$sbp_change)
sd_a = sd(data_a$sbp_change)
n_a = nrow(data_a)

# print values
diff_a ; sd_a ; n_a

# test statistic
t_a = diff_a / (sd_a / sqrt(n_a))
t_a

[1] -10.0667

This is an extremely large test statistic in magnitude, suggesting very strong evidence that SBP decreased from baseline to follow-up in the Drug A group.

Now calculate the two-sided p-value and a 95% confidence interval for the mean change.

# two-sided p-value
2 * pt(abs(t_a), df = , lower.tail = F)

# 95% confidence interval
diff_a + qt(c(, ), df = n_a - 1) * (sd_a / sqrt(n_a))
# two-sided p-value
2 * pt(abs(t_a), df = n_a - 1, lower.tail = F)

[1] 7.825647e-17

# 95% confidence interval
diff_a + qt(c(0.025, 0.975), df = n_a - 1) * (sd_a / sqrt(n_a))

[[1] -12.950227 -8.685646]]{style=“font-family: ‘Monaco’; font-weight: 500; font-size : 12px;”}

The p-value very small, and the 95% CI (-12.95, -8.69) is entirely below 0—strong evidence of a real decrease in SBP for the Drug A group.

Drug A: Test with t.test()

Fill in the code below to run a paired t-test using t.test(). There are two equivalent ways to do this: running a one-sample test directly on the sbp_change column, or passing the original sbp_followup and sbp_baseline columns with paired = T. Try the second approach here.

# paired t-test (Drug A)
t.test(x = data_a$, y = data_a$, paired = )
# paired t-test (Drug A)
t.test(x = data_a$sbp_followup, y = data_a$sbp_baseline, paired = T)

This should give (approximately) the same test statistic, p-value, and conclusion as our hand calculation above. Try also running t.test(data_a$sbp_change) as a one-sample test—you should get identical results, since a paired t-test is just a one-sample t-test on the differences.

Placebo: Test by Hand

Now we will move on to the Placebo group:

Repeat the hand calculation above, but for the Placebo group’s summary statistics.

# Placebo group summary statistics
diff_p = mean(data_p$sbp_change)
sd_p = sd(data_p$sbp_change)
n_p = nrow(data_p)

# print values
diff_p ; sd_p ; n_p

# test statistic
t_p =  / ( / sqrt())
t_p
# Placebo group summary statistics
diff_p = mean(data_p$sbp_change)
sd_p = sd(data_p$sbp_change)
n_p = nrow(data_p)

# print values
diff_p ; sd_p ; n_p

# test statistic
t_p = diff_p / (sd_p / sqrt(n_p))
t_p

[1] -1.837283

This test statistic is quite a bit smaller in magnitude than our Drug A test statistic.

Even though the Placebo group received no active treatment, we still found a statistically significant decrease in SBP. Does this mean the placebo “worked”? What are some other explanations for this result?

What If We Didn’t Pair?

It’s important to use a paired test when data are paired! Let’s see what happens if we (incorrectly) run an unpaired (independent samples) t-test on the same baseline and follow-up values for the Drug A group.

Fill in the code below to run the incorrect, unpaired version of the test on the Drug A group’s baseline and follow-up values (paired = F), for comparison.

# INCORRECT: unpaired test on the same (paired) data
t.test(x = data_a$, y = data_a$, paired = )
# INCORRECT: unpaired test on the same (paired) data
t.test(x = data_a$sbp_followup, y = data_a$sbp_baseline, paired = F)

The test statistic here is smaller in magnitude than our paired version, even though we’re using the exact same data! That’s because the unpaired test ignores the fact that baseline and follow-up measurements come from the same patients, and so it fails to account for (and remove) the variability between patients—variability that the paired test cancels out by looking at within-patient differences.

Based on what you just saw, why does using the wrong (unpaired) test when data are actually paired tend to make it harder to detect a real effect, rather than easier?

Power and Sample Size for Paired t-tests

Power and sample size calculations for a paired t-test work exactly like the one-sample case, using the standard deviation of the differences.

Suppose we want to detect a mean difference (delta) of 2 mmHg, assuming a standard deviation of the differences of 10, with a significance level of 0.05 and 80% power, using a one-sided test. Fill in the code below to find the required sample size.

# required sample size for 80% power
power.t.test(delta = , 
             sd = , 
             sig.level = , 
             power = ,
             type = , 
             alternative = )
# required sample size for 80% power
power.t.test(delta = 2, 
             sd = 10, 
             sig.level = 0.05, 
             power = 0.8,
             type = "one.sample", 
             alternative = "one.sided")

n = 155.9257

We’d need about 156 patients to reliably detect a mean difference of 2 mmHg given this much variability in the differences.

Our actual Drug A group only has n = 100 patients. Using power.t.test() with n = 100 instead of solving for sample size, what power do we actually have to detect a 2 mmHg difference? Is it above or below 80%? (You should get something close to 63%.)