Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 1 — Questions and Answers
Question 1: A researcher observes that cities with more coffee shops have higher average incomes. The researcher concludes that opening more coffee shops causes income to rise. What is the primary flaw in this reasoning?
- The researcher used too small a sample.
- Correlation does not imply causation; a confounding variable may explain both. (Correct answer)
- The researcher should have used a randomized experiment instead of a survey.
- Income data is unreliable in observational studies.
Correct answer: Correlation does not imply causation; a confounding variable may explain both.
Observational studies can identify associations but cannot establish causation because confounding variables (e.g., city wealth) may explain both variables.
Question 2: A study randomly assigns 200 volunteers to two groups: Group A takes a vitamin supplement daily and Group B takes a placebo for 6 months. At the end, Group A reports significantly fewer colds. What conclusion is best supported?
- All adults who take this supplement will get fewer colds.
- The supplement may cause a reduction in colds among people similar to those in this study. (Correct answer)
- The supplement definitely cures colds for all populations.
- The placebo group's results are invalid because they were deceived.
Correct answer: The supplement may cause a reduction in colds among people similar to those in this study.
Randomized experiments support causal conclusions, but the results can only be generalized to populations similar to those in the study.
Question 3: A school district surveys 500 randomly selected students across all its schools about their reading habits and finds that students who read more also have higher GPA. Can the district conclude that reading more causes higher GPA?
- Yes, because the sample was randomly selected.
- No, because this is an observational study and causation cannot be inferred from correlation. (Correct answer)
- Yes, because a large random sample always establishes causation.
- No, because GPA is not a reliable measure of academic performance.
Correct answer: No, because this is an observational study and causation cannot be inferred from correlation.
Even with random sampling, observational studies cannot establish causation because subjects were not randomly assigned to treatments; a third variable could explain the relationship.
Question 4: Researchers conduct a study in which participants are randomly assigned to eat breakfast or skip it, then measure their morning productivity for 4 weeks. The breakfast group showed significantly higher productivity. Which conclusion is best supported?
- Eating breakfast is the cause of higher productivity in the general population.
- For people similar to study participants, eating breakfast likely causes higher morning productivity. (Correct answer)
- All people who eat breakfast will be more productive.
- Skipping breakfast is correlated with lower productivity for all adults worldwide.
Correct answer: For people similar to study participants, eating breakfast likely causes higher morning productivity.
Random assignment supports a causal conclusion, but generalizability is limited to populations similar to those studied.
Question 5: A nutritionist conducts an observational study of 1,000 adults and finds that those who eat fish twice a week have lower rates of heart disease. Which of the following is the most appropriate conclusion?
- Eating fish twice a week causes lower rates of heart disease.
- There is an association between eating fish and lower rates of heart disease in this sample. (Correct answer)
- All adults should eat fish twice a week to prevent heart disease.
- Fish consumption is the single most important factor in preventing heart disease.
Correct answer: There is an association between eating fish and lower rates of heart disease in this sample.
An observational study can only identify associations, not causal relationships, since participants were not randomly assigned to a diet.
Question 6: A medical study randomly selects 300 patients from a clinic in one city, then randomly assigns half to a new treatment and half to standard care. The new treatment shows better outcomes. To which population can the results be most validly generalized?
- All patients worldwide with this condition
- Patients similar to those at this clinic who would be eligible for the treatment (Correct answer)
- Only the 300 patients in the study
- All patients in the city
Correct answer: Patients similar to those at this clinic who would be eligible for the treatment
Because the sample was drawn from one clinic, results can be generalized to similar patients at that type of clinic, not to all patients globally.
Question 7: A study finds that people who own pets have lower blood pressure. A journalist writes: 'Owning a pet reduces blood pressure.' Which error did the journalist make?
- Confusing the independent and dependent variables
- Concluding causation from an observational association (Correct answer)
- Misreporting the direction of the relationship
- Using too small a sample for the study
Correct answer: Concluding causation from an observational association
The journalist incorrectly inferred a causal relationship from an observational association; confounding variables (e.g., lifestyle, stress level) may explain the link.
Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 1
A researcher observes that cities with more coffee shops have higher average incomes.
The researcher concludes that opening more coffee shops causes income to rise.
What is the primary flaw in this reasoning?