Statistics Communication & Stakeholder Relations 5 — Questions and Answers
Question 1: A stakeholder asks why you are using a two-tailed test instead of a one-tailed test to detect improvement. What is the best justification?
- Two-tailed tests always have lower p-values
- A two-tailed test guards against missing unexpected effects in either direction and avoids bias toward expected outcomes (Correct answer)
- One-tailed tests require larger samples
- Two-tailed tests are required by software defaults
Correct answer: A two-tailed test guards against missing unexpected effects in either direction and avoids bias toward expected outcomes
Two-tailed tests are conservative and unbiased; one-tailed tests are appropriate only when direction is firmly established before data collection.
Question 2: A communications director wants to report that customer satisfaction 'increased significantly.' What must a statistician verify before approving this language?
- That the sample used ordinal scaling
- That both statistical significance and practical significance support the claim (Correct answer)
- That the increase was at least 10 percentage points
- That a chi-square test was used
Correct answer: That both statistical significance and practical significance support the claim
The word 'significantly' should only be used when both tests confirm significance and the magnitude is meaningful enough to matter to stakeholders.
Question 3: When distributing a statistical report across stakeholder groups with different expertise levels, the best practice is to:
- Send the same full technical report to everyone
- Use a layered structure: executive summary, key findings, then full technical methodology (Correct answer)
- Send only the conclusions with no supporting data
- Adjust the conclusions for each audience
Correct answer: Use a layered structure: executive summary, key findings, then full technical methodology
A layered report allows each audience to read to their depth of interest without forcing non-technical readers through dense methodology.
Question 4: A stakeholder challenges your analysis by citing a contradictory result from a different study. What is the most productive response?
- Dismiss the other study without review
- Compare sample sizes, methods, populations, and time periods to explain potential sources of divergence (Correct answer)
- Retract your findings immediately
- Claim your software is more accurate
Correct answer: Compare sample sizes, methods, populations, and time periods to explain potential sources of divergence
Contradictory findings often arise from methodological differences; a systematic comparison identifies why results diverge rather than asserting one is simply correct.
Question 5: Which data visualization mistake most often misleads stakeholders in statistical reports?
- Using color in bar charts
- Truncating the y-axis to exaggerate small differences (Correct answer)
- Labeling axes clearly
- Using a line chart for time series data
Correct answer: Truncating the y-axis to exaggerate small differences
A y-axis that doesn't start at zero visually amplifies differences, making small changes appear dramatic and misleading non-expert readers.
Question 6: A regulatory agency requires a written description of how missing data were handled. What is the best response in the methods section?
- State 'missing data were deleted' without further detail
- Describe the missing data mechanism assumed (MCAR/MAR/MNAR), the imputation or exclusion method used, and any sensitivity analyses (Correct answer)
- Say the software handled it automatically
- Report only the percentage of missing values
Correct answer: Describe the missing data mechanism assumed (MCAR/MAR/MNAR), the imputation or exclusion method used, and any sensitivity analyses
Full transparency about missing data treatment — including assumptions and sensitivity analyses — is required for reproducibility and regulatory compliance.
Question 7: A project sponsor pressures a statistician to rerun the analysis until it yields a significant p-value. This practice is known as:
- Bayesian updating
- p-hacking or data dredging, which inflates Type I error (Correct answer)
- Cross-validation
- Bootstrapping
Correct answer: p-hacking or data dredging, which inflates Type I error
Running multiple analyses and reporting only significant results dramatically inflates the false-positive rate beyond the nominal alpha level.
Statistics Communication & Stakeholder Relations 5
A stakeholder asks why you are using a two-tailed test instead of a one-tailed test to detect improvement.
What is the best justification?