Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 9 — Questions and Answers
Question 1: A study finds that students who eat breakfast score higher on standardized tests. The researchers conclude that eating breakfast causes higher test scores. What is the most significant flaw in this conclusion?
- The study is observational and cannot establish causation because confounding variables may explain the relationship. (Correct answer)
- The study did not use a large enough sample.
- The study should have used a placebo group.
- The study only measured test scores and not other academic outcomes.
Correct answer: The study is observational and cannot establish causation because confounding variables may explain the relationship.
In an observational study, the association between eating breakfast and test scores may be explained by confounding factors such as socioeconomic status or home environment.
Question 2: A clinical trial randomly assigns 300 patients to receive either a new arthritis medication or a placebo for 12 weeks, and the medication group reports significantly less pain. What conclusion is best supported?
- The new medication appears to cause a reduction in arthritis pain based on this randomized experiment. (Correct answer)
- The new medication is the best available treatment for arthritis pain.
- Patients who take arthritis medication always report less pain.
- The results can be generalized to all arthritis patients worldwide.
Correct answer: The new medication appears to cause a reduction in arthritis pain based on this randomized experiment.
Random assignment in a controlled experiment supports a causal conclusion about the medication's effect within the study population.
Question 3: Researchers observe that people who regularly attend concerts report higher life satisfaction than those who do not. A magazine concludes: 'Attending concerts improves life satisfaction.' What is the main limitation of this conclusion?
- The study is observational; people who attend concerts may differ from non-attendees in other ways that affect satisfaction. (Correct answer)
- The study should have used a random sample of concert types.
- The researchers should have measured concert attendance more precisely.
- The conclusion is valid because the sample is large.
Correct answer: The study is observational; people who attend concerts may differ from non-attendees in other ways that affect satisfaction.
Without random assignment, the association between concert attendance and satisfaction may be due to other differences between the groups, such as income or social connectedness.
Question 4: Researchers want to study whether a new fertilizer increases crop yield. They randomly assign 50 plots of land to receive the new fertilizer and 50 plots to receive standard fertilizer, then measure yield at harvest. Why is random assignment important in this study?
- It helps ensure that differences in yield are due to the fertilizer and not to pre-existing differences between plots. (Correct answer)
- It guarantees that all plots produce the same yield before the study.
- It allows the researchers to generalize results to all types of crops.
- It eliminates all measurement error in yield calculations.
Correct answer: It helps ensure that differences in yield are due to the fertilizer and not to pre-existing differences between plots.
Random assignment distributes confounding variables evenly across groups, allowing researchers to attribute differences in yield to the fertilizer treatment.
Question 5: An observational study of 2,000 adults finds that people who meditate daily have lower rates of anxiety. A therapist concludes that meditation eliminates anxiety. Which limitation is most important to acknowledge?
- Self-selection bias: people who choose to meditate may already have lower baseline anxiety. (Correct answer)
- The sample size is too small to draw conclusions.
- The study should have included a placebo group.
- Meditation is not a measurable variable.
Correct answer: Self-selection bias: people who choose to meditate may already have lower baseline anxiety.
In an observational study, participants self-select into the meditation group; those who meditate may already be less prone to anxiety, creating self-selection bias.
Question 6: A school randomly assigns 30 classrooms to use standing desks and 30 to use traditional desks for one academic year. At year-end, students with standing desks show better focus scores. What design feature allows the strongest causal conclusion?
- Random assignment of classrooms to conditions (Correct answer)
- The large number of classrooms studied
- Measuring focus scores at the end of the year
- Comparing two types of desks rather than one
Correct answer: Random assignment of classrooms to conditions
Random assignment of classrooms to desk conditions controls for confounders and allows a causal inference about the effect of desk type on focus.
Question 7: An observational study follows 10,000 adults for 15 years and finds that those who consume more whole grains have lower rates of diabetes. Which of the following would most strengthen the argument that whole grain consumption prevents diabetes?
- A randomized controlled experiment in which participants are randomly assigned to eat more whole grains or a control diet and followed for the same period (Correct answer)
- A larger observational study with 100,000 participants
- An observational study conducted in a different country
- A survey of participants about their grain preferences
Correct answer: A randomized controlled experiment in which participants are randomly assigned to eat more whole grains or a control diet and followed for the same period
Only a randomized controlled experiment with assignment to whole grain or control diet can establish causation by eliminating confounding differences between groups.
Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 9
A study finds that students who eat breakfast score higher on standardized tests.
The researchers conclude that eating breakfast causes higher test scores.
What is the most significant flaw in this conclusion?