Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 4 — Questions and Answers
Question 1: A biology teacher randomly selects 30 students from her class to test a new lab activity, then compares their quiz scores to the remaining students. She finds the selected students score higher. Can she conclude the new lab activity causes higher quiz scores?
- Yes, because she randomly selected the group.
- No, because random selection for one group does not constitute random assignment to treatment and control groups. (Correct answer)
- Yes, because the sample is representative of her class.
- No, because quiz scores are too variable to analyze.
Correct answer: No, because random selection for one group does not constitute random assignment to treatment and control groups.
The remaining students are not an equivalent control group; without random assignment, pre-existing differences between groups could explain the score difference.
Question 2: Researchers randomly assign 100 trees in an orchard to receive a new fertilizer and 100 trees to receive standard fertilizer. The treated trees yield 20% more fruit. What is the most appropriate conclusion?
- The new fertilizer causes more fruit yield in all fruit trees worldwide.
- The new fertilizer likely causes higher fruit yield in orchards similar to the one studied. (Correct answer)
- There is a correlation between fertilizer type and fruit yield, but no causal link.
- The 20% increase is too small to be meaningful.
Correct answer: The new fertilizer likely causes higher fruit yield in orchards similar to the one studied.
Random assignment within the orchard supports a causal conclusion limited to orchard conditions similar to those in the study.
Question 3: A university study randomly assigns incoming students to two groups: one receives a financial literacy workshop and one does not. After one year, the workshop group has higher savings rates. Which conclusion is most appropriate?
- The financial literacy workshop causes higher savings rates for all college students.
- Among students similar to those in this university, attending the workshop likely causes higher savings. (Correct answer)
- The students who didn't attend the workshop were irresponsible with money.
- The correlation between workshops and savings is not statistically meaningful.
Correct answer: Among students similar to those in this university, attending the workshop likely causes higher savings.
The random assignment supports a causal conclusion, but it applies to students like those at this university, not necessarily all college students.
Question 4: A study surveys adults about coffee consumption and heart disease risk, finding that higher coffee drinkers have lower heart disease rates. A cardiologist concludes that drinking coffee prevents heart disease. Which of the following is the most important criticism?
- The study should have measured caffeine levels, not coffee cups.
- This is an observational study; the relationship could be explained by confounding variables. (Correct answer)
- The study sample was too small to draw conclusions about heart disease.
- Heart disease rates cannot be measured in survey-based studies.
Correct answer: This is an observational study; the relationship could be explained by confounding variables.
In an observational study, people who drink more coffee may differ from non-drinkers in other ways (e.g., lifestyle, diet) that reduce heart disease risk, making the causal claim unjustified.
Question 5: A researcher conducts a randomized controlled trial with 400 participants drawn from a single urban hospital. The treatment is found to be effective. A policy maker wants to apply the findings to rural populations. What is the main concern?
- Randomized trials cannot establish effectiveness.
- The study sample from an urban hospital may not represent rural populations, limiting generalizability. (Correct answer)
- Rural populations are too small to be studied separately.
- Random assignment is only valid when samples are from multiple locations.
Correct answer: The study sample from an urban hospital may not represent rural populations, limiting generalizability.
The study establishes causation within the sample but cannot automatically generalize to rural populations that differ in demographics, access to care, or other factors.
Question 6: An observational study finds that students who eat breakfast score higher on standardized tests. Which of the following, if true, would most weaken the claim that breakfast causes higher test scores?
- The study included both rural and urban students.
- Students from higher-income families are more likely to eat breakfast and also have greater access to test-prep resources. (Correct answer)
- The test score difference was statistically significant.
- The study was conducted over three consecutive years.
Correct answer: Students from higher-income families are more likely to eat breakfast and also have greater access to test-prep resources.
If income is a confounder — affecting both breakfast habits and access to tutoring/resources — then it, not breakfast, may be responsible for the higher scores.
Question 7: Which of the following study designs allows researchers to conclude that Treatment X causes Outcome Y?
- An observational study with a large, nationally representative random sample
- A randomized controlled experiment in which participants are randomly assigned to receive X or a placebo (Correct answer)
- A longitudinal survey tracking participants over 10 years
- A cross-sectional study comparing groups at a single point in time
Correct answer: A randomized controlled experiment in which participants are randomly assigned to receive X or a placebo
Only a randomized controlled experiment with random assignment to treatment and control groups allows researchers to infer causation.
Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 4
A biology teacher randomly selects 30 students from her class to test a new lab activity, then compares their quiz scores to the remaining students.
She finds the selected students score higher.
Can she conclude the new lab activity causes higher quiz scores?