Watson Statistical Reasoning and Sampling — Questions and Answers
Question 1: A survey of 50 university students finds that 80% prefer online learning. A newspaper headline reads: 'Most people prefer online learning.' What is the main flaw in this conclusion?
- The survey was conducted at a university, not a business
- The sample is too small and too specific to represent the general population (Correct answer)
- Online learning is not available to everyone
- The percentage should be expressed as a fraction
Correct answer: The sample is too small and too specific to represent the general population
University students are a narrow, self-selected group whose preferences are unlikely to reflect those of the broader population. Both the small sample size and the lack of representativeness undermine the generalization. A valid conclusion would need a larger, diverse, randomly selected sample.
Question 2: A study finds that cities with more libraries have higher average incomes. A researcher concludes that 'building more libraries increases income.' What is wrong with this reasoning?
- Libraries are publicly funded and cannot increase income
- The researcher has confused correlation with causation (Correct answer)
- Average income is not a reliable economic measure
- The study should have measured individual incomes, not city averages
Correct answer: The researcher has confused correlation with causation
Correlation — two variables moving together — does not establish that one causes the other. A third factor (e.g., wealthier cities can afford both more libraries and higher wages) could explain the relationship. Causal claims require evidence beyond statistical association.
Question 3: Drug A reduced symptoms in 60% of 10 trial participants. Drug B reduced symptoms in 55% of 1,000 trial participants. Which result provides stronger evidence of effectiveness?
- Drug A, because its success rate is higher
- Drug B, because its larger sample produces a more reliable estimate (Correct answer)
- Both are equally strong because percentages are directly comparable
- Neither, because neither drug achieved 100% success
Correct answer: Drug B, because its larger sample produces a more reliable estimate
With only 10 participants, Drug A's 60% result (6 people) could easily be due to chance. Drug B's result from 1,000 participants is statistically far more stable and reliable. A larger sample reduces random variation, making the percentage estimate more trustworthy even if slightly lower.
Question 4: A company surveys employees by placing a feedback form on the staff intranet. 90% of respondents say they are satisfied with management. Why should this result be interpreted cautiously?
- Digital surveys are less accurate than paper surveys
- The result is too high to be believable
- Only motivated respondents self-select into the survey, creating a biased sample (Correct answer)
- The survey should have been anonymous to be valid
Correct answer: Only motivated respondents self-select into the survey, creating a biased sample
Self-selection bias occurs when participation is voluntary — those with strong positive feelings (or strong negative feelings) are more likely to respond than those with moderate views. The sample therefore does not represent all employees, making it impossible to generalize the 90% figure to the full workforce.
Question 5: A clinical study reports that a new therapy reduces anxiety scores by an average of 15 points. The study notes a standard deviation of 18 points. What does the high standard deviation indicate?
- The therapy is on average highly effective
- The 15-point average conceals wide variation in individual responses (Correct answer)
- The measurement instrument was unreliable
- The therapy should be retested with a different population
Correct answer: The 15-point average conceals wide variation in individual responses
A standard deviation larger than the mean improvement signals that outcomes varied enormously: some participants improved far more than 15 points, others barely at all or worsened. Reporting only the average without this context would overstate the therapy's predictable benefit for any given individual.
Question 6: A poll conducted three weeks before an election shows Candidate X leading with 52% support among 400 respondents, with a margin of error of ±5%. What is the most accurate interpretation?
- Candidate X will definitely win the election
- Candidate X has an insurmountable lead and the race is over
- The true support for Candidate X could plausibly be anywhere between 47% and 57% (Correct answer)
- The poll is unreliable because it was conducted too early
Correct answer: The true support for Candidate X could plausibly be anywhere between 47% and 57%
A margin of error of ±5% means the true value falls within 52% ± 5%, i.e., between 47% and 57%. Since this range includes values below 50%, the race may be statistically tied. The reported lead is within the margin of error, making it impossible to conclude Candidate X is definitively ahead.
Watson Statistical Reasoning and Sampling
A survey of 50 university students finds that 80% prefer online learning.
A newspaper headline reads: 'Most people prefer online learning.' What is the main flaw in this conclusion?