Bluebook SAT Test Evaluating Statistical Claims: Observational Studies and Experiments 6

Question 1

A university randomly assigns 60 first-year students to peer mentoring and 60 to no mentoring.
Those with mentors have higher GPA after one year.
A dean applies this to all students at the university.
What is the main limitation of this generalization?

Accepted Answer

The 120 students may not be representative of all first-year students at the university.

Answer

GPAs are too variable to measure across all students.

Answer

Random assignment makes the study results impossible to generalize.

Answer

Peer mentoring programs cannot be implemented university-wide.

Question 2

Researchers study whether a new teaching strategy improves reading comprehension by observing classrooms that voluntarily adopt the strategy versus those that do not. What is the primary limitation of this study?

Accepted Answer

Because teachers self-selected the strategy, more motivated or skilled teachers may have adopted it, confounding results.

Answer

The sample size is too large for valid comparison.

Answer

Reading comprehension cannot be measured objectively.

Answer

The study would have been stronger with even more volunteers.

Question 3

A well-designed randomized experiment conducted on college athletes in one state shows that a protein supplement improves sprint times. A coach in a different state wants to apply this finding to high school students. What concern is most valid?

Accepted Answer

High school students may differ from college athletes in age, physiology, and training level, limiting generalizability.

Answer

Randomized experiments are only valid in laboratory settings.

Answer

The protein supplement would work on all humans regardless of age or fitness level.

Answer

Sprint times cannot be reliably measured in field settings.

Question 4

An observational study shows that people who drink red wine moderately have fewer cardiovascular events than abstainers. A doctor recommends wine to all patients to protect their heart. Which of the following best explains why this recommendation is premature?

Accepted Answer

The association may be explained by confounders; moderate drinkers may also follow healthier overall lifestyles.

Answer

The study used too few participants.

Answer

Cardiovascular events are rare and cannot be studied in observational data.

Answer

Wine is too inconsistent in quality across studies.

Question 5

A study randomly assigns 50 households to receive a water filter and 50 households to not receive one. After 6 months, the filter group reports fewer gastrointestinal illnesses. Which of the following is a valid conclusion?

Accepted Answer

In households similar to those in the study, water filters likely cause fewer gastrointestinal illnesses.

Answer

Using a water filter causes fewer gastrointestinal illnesses in all households worldwide.

Answer

The households without filters were negligent about their water supply.

Answer

There is only a correlation between water filters and illness — not a causal link.

Question 6

A survey finds that people who own dogs walk more steps per day than non-dog-owners. The researcher concludes: 'Dog ownership increases daily step count.' What type of evidence would strengthen this causal claim?

Accepted Answer

A randomized experiment in which some participants are randomly assigned a dog

Answer

A larger survey with a more diverse sample

Answer

A longitudinal study following dog-owners over 10 years

Answer

A survey in multiple countries confirming the same association

Question 7

A large randomized experiment with diverse participants from 20 states finds that a new reading intervention improves comprehension scores. A state education board generalizes this to all students in the country. Which concern about generalizability is least valid?

Accepted Answer

The study used random assignment, which undermines generalizability.

Answer

The study was conducted in only 20 of 50 states.

Answer

The sample may not represent every demographic group equally.

Answer

The intervention may work differently in under-resourced schools.