Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 8 — Questions and Answers
Question 1: A study randomly assigns 300 adults to participate in a strength training program or a stretching-only program. After 12 weeks, the strength training group shows greater bone density. The researcher concludes strength training causes increased bone density. Which additional question is most important for evaluating this conclusion?
- Was the study conducted in a gym or a laboratory?
- Are the study participants representative of the broader adult population of interest? (Correct answer)
- Was the stretching program the same duration as strength training?
- Were the participants told which group they were in?
Correct answer: Are the study participants representative of the broader adult population of interest?
While the randomized design supports causation within the study, you must consider whether the participants represent the target population to assess whether results can be generalized.
Question 2: A poll of 1,200 randomly selected U.S. voters finds that 54% favor a specific immigration policy. Which conclusion is best supported?
- Exactly 54% of all U.S. voters favor this policy.
- A majority of U.S. voters likely favor this policy, subject to a margin of error. (Correct answer)
- The policy will pass because more than half favor it.
- Immigration policy preferences cause certain voting behaviors.
Correct answer: A majority of U.S. voters likely favor this policy, subject to a margin of error.
A random sample supports a generalization about the broader voting population, but the result is an estimate with uncertainty (margin of error), not a precise count or a prediction of outcomes.
Question 3: A researcher notices that ice cream sales and drowning rates are highly correlated throughout the year. She concludes that ice cream causes drowning. What is the most likely explanation for this correlation?
- People who eat ice cream are more likely to swim unsafely.
- A confounding variable — hot weather — increases both ice cream sales and swimming (and therefore drowning). (Correct answer)
- The data was collected incorrectly, inflating the correlation.
- Drowning rates cause ice cream sales to increase.
Correct answer: A confounding variable — hot weather — increases both ice cream sales and swimming (and therefore drowning).
Hot weather increases both ice cream consumption and swimming frequency, creating a spurious correlation between ice cream and drowning that has no causal mechanism.
Question 4: A randomized clinical trial shows that Drug X reduces migraines better than a placebo in a diverse national sample of 2,000 adults. A neurologist concludes that Drug X should be the first-line treatment for all adults with migraines. Which statement about this conclusion is correct?
- The conclusion is fully supported; large randomized trials prove universal effectiveness.
- The conclusion is reasonable but should be further evaluated against side effects, cost, and results in subgroups. (Correct answer)
- The conclusion is invalid because randomized trials only work for acute conditions.
- The conclusion requires replication in observational studies before being accepted.
Correct answer: The conclusion is reasonable but should be further evaluated against side effects, cost, and results in subgroups.
Even strong randomized trial evidence supports — but does not fully determine — treatment decisions; real-world factors like side effects, cost, and subgroup differences must also be evaluated.
Question 5: A high school conducts an experiment where 10 volunteering teachers adopt a new grading method and their students are compared to students of non-participating teachers. Students in the new grading method classes earn higher grades. What is the most significant flaw?
- The sample of 10 teachers is too small for meaningful analysis.
- Teachers who volunteered may be more skilled or motivated, confounding the relationship between grading method and grades. (Correct answer)
- The experiment should have measured standardized test scores instead of grades.
- The non-participating teachers were not randomly selected as a control group.
Correct answer: Teachers who volunteered may be more skilled or motivated, confounding the relationship between grading method and grades.
Volunteer teachers may be more effective or motivated regardless of the grading method, making teacher quality a confounding variable in this observational design.
Question 6: Based on an observational study, a health organization claims that people who eat more vegetables have fewer depressive episodes. Which of the following, if true, would most weaken a causal interpretation?
- The study included over 10,000 participants.
- People with lower depression are more likely to have energy and motivation to prepare and eat healthy meals. (Correct answer)
- The study measured vegetable intake through self-reporting.
- The study controlled for age and gender.
Correct answer: People with lower depression are more likely to have energy and motivation to prepare and eat healthy meals.
If lower depression leads to better dietary habits (reverse causation), then depression may be causing the vegetable intake difference rather than the other way around.
Question 7: A study randomly selects 400 schools nationwide and observes that schools with larger libraries have higher reading proficiency. A policy advisor concludes that expanding libraries will improve reading proficiency. What is the best evaluation of this recommendation?
- The recommendation is justified because the study used a nationally representative random sample.
- The recommendation is premature because the observational study cannot establish that larger libraries cause higher proficiency. (Correct answer)
- The recommendation is justified because library size directly measures reading opportunity.
- The recommendation requires only a larger sample size to be valid.
Correct answer: The recommendation is premature because the observational study cannot establish that larger libraries cause higher proficiency.
Even with a nationally representative random sample, an observational study can only show association; confounding variables (e.g., school funding, community investment) may explain both library size and reading proficiency.
Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 8
A study randomly assigns 300 adults to participate in a strength training program or a stretching-only program.
After 12 weeks, the strength training group shows greater bone density.
The researcher concludes strength training causes increased bone density.
Which additional question is most important for evaluating this conclusion?