Statistics Research & Evidence-Based Practice 5 — Questions and Answers
Question 1: Criterion validity is best assessed by:
- Comparing a measure's internal consistency using Cronbach's alpha
- Examining agreement between a new measure and an established gold standard (Correct answer)
- Having expert judges rate the measure's content coverage
- Testing whether the measure predicts theoretically related constructs
Correct answer: Examining agreement between a new measure and an established gold standard
Criterion validity examines how well a new measure correlates with a gold-standard criterion measured concurrently or in the future.
Question 2: In intention-to-treat (ITT) analysis, data are analyzed:
- Only for participants who completed all protocol procedures
- According to the treatment actually received
- According to the group to which participants were originally assigned (Correct answer)
- After excluding participants with protocol deviations
Correct answer: According to the group to which participants were originally assigned
ITT preserves the benefits of randomization by analyzing all participants in their assigned groups regardless of adherence.
Question 3: A researcher finds r = 0.85 between two raters scoring the same essays. This is an example of:
- Predictive validity
- Inter-rater reliability (Correct answer)
- Internal consistency
- Construct validity
Correct answer: Inter-rater reliability
Inter-rater reliability quantifies agreement between different scorers or observers measuring the same thing.
Question 4: Which statistical approach is used to control for multiple potential confounders simultaneously in an observational study?
- Pearson correlation
- Chi-square test
- Multivariable regression (Correct answer)
- Wilcoxon signed-rank test
Correct answer: Multivariable regression
Multivariable regression models allow researchers to adjust for multiple confounders and isolate the effect of a primary predictor.
Question 5: Attrition bias in a longitudinal study occurs when:
- Participants are not randomly assigned to conditions
- Participants who drop out differ systematically from those who remain (Correct answer)
- The outcome measure changes over time
- Researchers know the study hypothesis when collecting data
Correct answer: Participants who drop out differ systematically from those who remain
If dropout is related to the treatment or outcome, the remaining sample becomes unrepresentative, biasing results.
Question 6: The number needed to treat (NNT) is calculated as:
- Relative risk × 100
- 1 / Absolute risk reduction (Correct answer)
- Odds ratio − 1
- Relative risk reduction / Baseline risk
Correct answer: 1 / Absolute risk reduction
NNT = 1 / ARR; it tells clinicians how many patients must be treated to prevent one additional adverse outcome.
Question 7: A study has high internal validity but low external validity. This means:
- The findings are generalizable but methodologically flawed
- The study measures what it claims but uses an unreliable instrument
- The causal conclusions are sound but may not generalize to other populations or settings (Correct answer)
- The sample is large but the effect size is small
Correct answer: The causal conclusions are sound but may not generalize to other populations or settings
High internal validity means causal inferences are trustworthy; low external validity means results may not apply beyond the study context.
Statistics Research & Evidence-Based Practice 5
Criterion validity is best assessed by: