Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 2 — Questions and Answers
Question 1: Researchers want to determine whether a new tutoring program improves standardized test scores. They randomly assign 150 students to receive tutoring and 150 students to a control group with no tutoring, then compare scores. Why is random assignment important in this study?
- It ensures all students have an equal chance of being surveyed.
- It controls for confounding variables by balancing them across groups on average. (Correct answer)
- It guarantees the tutoring group will show improvement.
- It increases the sample size, reducing the margin of error.
Correct answer: It controls for confounding variables by balancing them across groups on average.
Random assignment distributes known and unknown confounding variables roughly equally between groups, allowing differences in outcomes to be attributed to the treatment.
Question 2: A study observes that neighborhoods with more parks have lower obesity rates. A city planner concludes that building more parks will reduce obesity. Which type of additional evidence would most strengthen the claim that parks reduce obesity?
- A larger observational study with more neighborhoods
- A randomized experiment in which some neighborhoods are randomly assigned new parks (Correct answer)
- A survey asking residents whether they use parks
- A meta-analysis of observational studies about parks and health
Correct answer: A randomized experiment in which some neighborhoods are randomly assigned new parks
Only a randomized experiment with random assignment can establish causation; more observational evidence strengthens association but not causation.
Question 3: A researcher surveys 400 college students randomly selected from one university about social media use and GPA. The results show a negative association. Which statement correctly describes what the researcher can conclude?
- Heavy social media use causes lower GPA among all college students.
- There is a negative association between social media use and GPA among students at this university. (Correct answer)
- All students who reduce social media use will see GPA improvements.
- The results apply to all college students because the sample was random.
Correct answer: There is a negative association between social media use and GPA among students at this university.
Because this is an observational study with a sample from one university, the results can identify an association in that population but cannot establish causation or broad generalizability.
Question 4: A clinical trial randomly assigns 500 patients to either a new drug or a placebo. The drug group shows significantly fewer symptoms. A doctor generalizes this to all patients with the condition. What concern is most valid?
- The random assignment may have introduced bias.
- The study can only be generalized to patients similar to those who participated in the trial. (Correct answer)
- Placebo effects make all clinical trials invalid.
- A sample of 500 is too small to draw any conclusions.
Correct answer: The study can only be generalized to patients similar to those who participated in the trial.
Even with random assignment, generalizability is limited to populations represented by the study participants; the doctor's broad generalization may be unwarranted.
Question 5: An education researcher randomly assigns 40 classrooms to use a new math curriculum and 40 to use the standard curriculum. At year-end, the new curriculum classrooms score significantly higher. What is the most valid conclusion?
- The new curriculum causes higher math scores in classrooms like those in the study. (Correct answer)
- Every student who uses the new curriculum will score higher on math tests.
- The new curriculum is superior to all other existing math curricula.
- There is only a correlation between curriculum type and scores, not causation.
Correct answer: The new curriculum causes higher math scores in classrooms like those in the study.
Random assignment of classrooms supports a causal conclusion for classrooms similar to those in the study, though it may not generalize to all possible classroom environments.
Question 6: A company surveys all of its 200 employees and finds that those who participate in the wellness program have fewer sick days. The CEO announces that the wellness program reduces sick days. What is the most significant flaw in this conclusion?
- The sample size of 200 is too small for any conclusions.
- Employees who are already healthier may be more likely to join the program, creating selection bias. (Correct answer)
- A census of all employees is less valid than a random sample.
- The wellness program only covers physical health, not mental health.
Correct answer: Employees who are already healthier may be more likely to join the program, creating selection bias.
Without random assignment, healthier employees may self-select into the program; this selection bias means the program may not be causing the difference in sick days.
Question 7: A study compares cancer rates in people who live near power lines versus those who do not, and finds a higher cancer rate near power lines. What is the most important limitation of this study?
- The study did not measure the exact distance from power lines.
- The study is observational, so the higher cancer rate could be due to confounding variables rather than power line proximity. (Correct answer)
- Cancer rates cannot be measured in observational studies.
- The study should have used a larger sample to be valid.
Correct answer: The study is observational, so the higher cancer rate could be due to confounding variables rather than power line proximity.
Observational studies cannot rule out confounding variables (e.g., socioeconomic factors, industrial pollution) that may explain both proximity to power lines and higher cancer rates.
Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 2
Researchers want to determine whether a new tutoring program improves standardized test scores.
They randomly assign 150 students to receive tutoring and 150 students to a control group with no tutoring, then compare scores.
Why is random assignment important in this study?