Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 5

Question 1

A researcher finds that cities with more hospitals per capita have higher death rates than cities with fewer hospitals per capita. A news headline reads: 'Hospitals Increase Risk of Death.' Which of the following best explains why this conclusion is flawed?

Accepted Answer

Confounding variables, such as population age and pre-existing illness rates, likely explain the association.

Answer

This is a classic confounding variable problem. Sicker and older populations are concentrated in areas with more hospitals — the hospitals are responding to need, not causing death. The association is driven by a lurking variable (illness severity), not a causal relationship between hospital presence and mortality.

Question 2

In a randomized controlled experiment, participants are randomly assigned to take either a new supplement or a placebo. At the end of the study, those who took the supplement reported significantly better sleep. However, 23% of participants in the supplement group dropped out before the study ended, compared to 4% in the placebo group. What is the most serious threat to the validity of this study's conclusion?

Accepted Answer

Differential attrition may have biased the results by removing participants who responded poorly.

Answer

Differential attrition — when dropout rates differ significantly between groups — is a major threat to internal validity. If participants who dropped out of the supplement group did so because the supplement worsened their sleep, the remaining participants are a self-selected group of those who tolerated or benefited from it, biasing the results in favor of the supplement.

Question 3

A study reports that students who eat breakfast score an average of 12 points higher on standardized tests than students who skip breakfast. The study was observational, with a sample of 2,400 students drawn from 15 schools. Which of the following conclusions is best supported by these data?

Accepted Answer

There is an association between eating breakfast and higher test scores, but causation cannot be established.

Answer

Observational studies can identify associations but cannot establish causation because participants are not randomly assigned to conditions. Socioeconomic status, for example, may influence both whether a student eats breakfast and their academic performance. A 2,400-student sample across 15 schools is reasonably large; the issue is the study design, not the sample size.

Question 4

Researchers want to test whether a new tutoring program improves SAT math scores. They recruit 200 student volunteers and allow students to choose whether to enroll in the tutoring program or not. After 8 weeks, the tutoring group scores 40 points higher on average. Which of the following design flaws most directly undermines a causal interpretation?

Accepted Answer

Self-selection bias means students who chose tutoring may differ systematically from those who did not.

Answer

Because students chose their own group, the two groups may differ in motivation, prior preparation, or access to other resources — all of which could independently raise scores. This self-selection bias confounds the treatment effect and prevents causal claims. Random assignment is the mechanism that controls for these pre-existing differences.

Question 5

A well-designed, double-blind randomized experiment with 500 participants finds that a new medication reduces headache frequency by a statistically significant margin (p = 0.03). A critic argues the result is not practically meaningful. Which of the following, if true, would most strongly support the critic's position?

Accepted Answer

The medication reduced headaches by an average of 0.2 episodes per month compared to placebo.

Answer

Statistical significance (p < 0.05) indicates that a result is unlikely to be due to chance alone, but it does not measure practical or clinical significance. A reduction of only 0.2 headaches per month — even if real — may be too small to meaningfully improve patients' lives. Effect size, not just p-value, determines practical relevance.

Bluebook SAT Practice Test

Bluebook SAT Practice Test

Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 5