Module 3: Inference for Numerical Outcomes, Part I

Fill-in-the-blank coding exercises for Module 3: One-Sample t-Tests
Author

Haley Grant

One-Sample t-tests

A one-sample t-test is used to make inference about a population mean using a single sample.

Conditions:

  • Random sample from the population
  • \(n > 30\), or the original variable follows a normal distribution in the population

\[H_0: \mu = \mu_0 \qquad \text{vs.} \qquad H_A: \mu \ne \mu_0 \ \ (\text{or} \ <, \ >)\]

where \(\mu\) is the population mean.

Motivating example: A clinical trial is testing a new antihypertensive drug (“Drug A”) against a placebo in patients with Stage 1 hypertension. Each of the 200 patients (100 per group) has a baseline systolic blood pressure (SBP, in mmHg) and a 12-week follow-up SBP. We’ll use this same trial throughout Module 3 to practice different tests. The data are stored in htn_trial, with columns group, sbp_baseline, sbp_followup, and sbp_change (follow-up minus baseline).

For this first test, we’ll focus on a simple question: is the average follow-up SBP in the Drug A group below the clinical hypertension threshold of 130 mmHg?

Creating a Factor Variable

Right now, the treatment group is stored in htn_trial$group_num as a number (1 = Drug A, 2 = Placebo). If we tried to use this numeric column directly in later analyses (like ANOVA, in Module 4), R would incorrectly treat it as a continuous predictor instead of a grouping variable! Let’s fix that now by creating a proper factor column called group.

Fill in the code below to create a new column, group, that labels group_num == 1 as "Drug A" and group_num == 2 as "Placebo".

# create a labeled factor variable for treatment group
htn_trial = htn_trial %>%
  mutate(group = factor(, levels = c(1, 2), labels = c(, )))

# check that it worked
head(htn_trial)
# create a labeled factor variable for treatment group
htn_trial = htn_trial %>%
  mutate(group = factor(group_num, levels = c(1, 2), labels = c("Drug A", "Placebo")))

# check that it worked
head(htn_trial)

You should now see a new column, group, with labels "Drug A" and "Placebo" instead of 1 and 2. We’ll use this labeled version throughout the rest of this document (and throughout Module 3).

Checking Conditions

Filter htn_trial to only the Drug A group (recall group_num == 1 is Drug A), and plot a histogram of sbp_followup to check the shape of the distribution.

# filter to the Drug A group and plot a histogram of follow-up SBP
htn_trial %>%
  filter( == 1) %>%
  ggplot(aes(x = )) +
  geom_histogram(fill = "blue", bins = 20) +
  theme_bw()
# filter to the Drug A group and plot a histogram of follow-up SBP
htn_trial %>%
  filter(group_num == 1) %>%
  ggplot(aes(x = sbp_followup)) +
  geom_histogram(fill = "blue", bins = 20) +
  theme_bw()

You should see a roughly bell-shaped, symmetric distribution centered somewhere in the 120s. Either way, with n = 100, we’d be comfortable relying on the CLT even if the data were somewhat non-normal.

Calculating the Test Statistic

Suppose a summary of the Drug A group’s follow-up SBP showed a sample mean of \(\bar{x} = 128\) and sample standard deviation of \(s = 9\), based on \(n = 100\) patients. We want to test \(H_0: \mu = 130\) vs. \(H_A: \mu < 130\).

Recall the formula for a one-sample t-test statistic:

\[t_{obs} = \dfrac{\bar{x} - \mu_0}{s / \sqrt{n}}\]

Fill in the code below to calculate the standard error and the test statistic, testing against \(\mu_0 = 130\).

# summary statistics
xbar = mean(htn_trial$sbp_followup[htn_trial$group_num==1])
mu0 = 130
s = sd(htn_trial$sbp_followup[htn_trial$group_num==1])
n = sum(htn_trial$group_num==1)

# print values
xbar ;  s; n

# standard error
se =  / sqrt()

# test statistic
t_obs = ( - ) / 
t_obs
# summary statistics
xbar = mean(htn_trial$sbp_followup[htn_trial$group_num==1])
mu0 = 130
s = sd(htn_trial$sbp_followup[htn_trial$group_num==1])
n = sum(htn_trial$group_num==1)

# print values
xbar ; s; n

# standard error
se = s / sqrt(n)

# test statistic
t_obs = (xbar - mu0) / se
t_obs

[[1] 127.3865 [1] 130 [1] 8.059693 [1] 100 [1] -3.24262{style=“font-family: ‘Monaco’; font-weight: 500; font-size : 12px;”}

Our observed test statistic is negative, consistent with a sample mean (127.4) that’s below the hypothesized value (130).

Rejection Region

Since \(H_A: \mu < \mu_0\), this is a one-sided, lower-tailed test. The rejection region is in the left tail of the \(t\) distribution with \(n - 1\) degrees of freedom.

Find the critical value (rejection region cutoff) for a lower-tailed test with \(\alpha = 0.05\) and \(n - 1 = 99\) degrees of freedom.

# significance level
alph = 

# lower-tailed rejection region cutoff
qt(alph, df = )
# significance level
alph = 0.05

# lower-tailed rejection region cutoff
qt(alph, df = n - 1)

[1] -1.660391

Since our observed test statistic (-3.24) is more negative than this cutoff (-1.66), it falls in the rejection region.

Based on the rejection region above, would we reject or fail to reject \(H_0\)? What would that mean in the context of this study?

P-value

Calculate the one-sided (lower-tailed) p-value corresponding to our test statistic, using pt(). Since we want the area to the left of our test statistic, we can use the default lower.tail = TRUE (the default, so you don’t need to specify it).

# one-sided (lower-tailed) p-value
pt(t_obs, df = )
# one-sided (lower-tailed) p-value
pt(t_obs, df = n - 1)

[1] 0.0008073392

This p-value (0.0008) is less than 0.05, so we would reject \(H_0\) and conclude that average follow-up SBP in the Drug A group is below 130—consistent with our rejection region result above.

Confidence Interval

A one-sided 95% confidence interval for \(H_A: \mu < \mu_0\) has the form \(\left(-\infty, \ \bar{x} + t^* \cdot se\right)\), where \(t^*\) is the upper 95th-percentile critical value (note: this is a different cutoff than the one we used for the rejection region above, since that one captured the lower 5% rather than the upper 5%).

Calculate the upper bound of a one-sided 95% confidence interval for the population mean follow-up SBP.

# one-sided 95% CI upper bound
xbar + qt(, df = ) * se
# one-sided 95% CI upper bound
xbar + qt(0.95, df = n - 1) * se

[1] 128.7248

We are 95% confident that the true average follow-up SBP in the Drug A group is no more than about 128.7—which is indeed below our threshold of 130. This agrees with our rejection region and p-value results, as it should, since all three methods lead to the same conclusion.

Using t.test()

All of this can be done in a single step using the built-in t.test() function.

Fill in the code below to run a one-sample t-test on the Drug A group’s sbp_followup, testing against \(\mu_0 = 130\) with the alternative hypothesis that the mean is less than 130.

# one-sample t-test
drug_a_followup = htn_trial %>% filter( == 1) %>% pull(sbp_followup)
t.test(drug_a_followup, mu = , alternative = )
# one-sample t-test
drug_a_followup = htn_trial %>% filter(group_num == 1) %>% pull(sbp_followup)
t.test(drug_a_followup, mu = 130, alternative = "less")

We can see that the numbers here match out by-hand calculations.

Compare the output of t.test() to our manual calculations above. Are the general conclusions (reject vs. fail to reject) consistent?

One-Sided vs. Two-Sided Tests

Modify the code above to run the same test, but using a two-sided alternative instead.

# two-sided one-sample t-test
t.test(drug_a_followup, mu = , alternative = )
# two-sided one-sample t-test
t.test(drug_a_followup, mu = 130, alternative = "two.sided")

The test statistic stays the same, but the p-value doubles compared to the one-sided test, and the confidence interval becomes two-sided (both a lower and upper bound) instead of having an infinite lower bound.

In general, when is it appropriate to use a one-sided test instead of a two-sided test? Why is it considered poor practice to choose the direction of a one-sided test after looking at your data?

Sample Size and Power

Before collecting data, we often want to know how large a sample we’ll need to detect a meaningful effect, or how much power a given sample size will provide. R’s power.t.test() function handles this for one-sample (and paired) t-tests.

Suppose we want to detect a true difference (delta) of 2 mmHg from the null value, assuming a standard deviation of 9, with a significance level of 0.05 and 80% power, using a one-sided test. Fill in the code below to find the required sample size.

# required sample size for 80% power
power.t.test(delta = , 
             sd = , 
             sig.level = ,
             power = , 
             type = , 
             alternative = )
# required sample size for 80% power
power.t.test(delta = 2, 
             sd = 9, 
             sig.level = 0.05,
             power = 0.8, 
             type = "one.sample", 
             alternative = "one.sided")

n = 126.5606

We would need about 127 participants (always round up!) to have an 80% chance of detecting a true difference of 2 mmHg, if one exists.

Now suppose we only have 100 participants available (as in our actual Drug A group). Fill in the code below to calculate the power of our test under the same assumptions (delta = 2, sd = 9, one-sample, one-sided).

# power with n = 100
power.t.test(n = , 
             delta = , 
             sd = ,
             type = , 
             alternative = )
# power with n = 100
power.t.test(n = 100, 
             delta = 2, 
             sd = 9,
             type = "one.sample", 
             alternative = "one.sided")

power = 0.71299

With 100 participants, our power is about 71.3%—a bit short of the 80% we were aiming for.

Based on what you know about power, would you expect this power (71.3%) to be higher or lower if we used a two-sided test instead of one-sided? What about if the true effect size (delta) were larger, say 4 instead of 2? You can check your intuition by modifying the code above.