FAST Experimental Design 3 — Questions and Answers
Question 1: A scientist measures the same participants' anxiety levels before and after a mindfulness intervention. Which statistical concern is most relevant?
- Regression to the mean only affects between-subjects designs
- Carryover effects may confound the post-test results (Correct answer)
- Random assignment eliminates all measurement bias
- Pre-test scores cannot be used as covariates
Correct answer: Carryover effects may confound the post-test results
In repeated-measures designs, carryover effects occur when the first condition influences performance or responses in the second condition.
Question 2: Which of the following best defines 'ecological validity' in an experiment?
- The degree to which results generalize to real-world settings (Correct answer)
- The accuracy of environmental measurements taken during the study
- The extent to which participants represent the natural population
- The use of natural materials rather than synthetic ones
Correct answer: The degree to which results generalize to real-world settings
Ecological validity refers to how well experimental findings apply to real-world contexts beyond the laboratory.
Question 3: A researcher randomly selects 200 people from a city directory and then randomly assigns them to treatment or control. Which statement is correct?
- Random selection ensures internal validity; random assignment ensures external validity
- Random assignment ensures internal validity; random selection ensures external validity (Correct answer)
- Both procedures serve the same purpose
- Neither procedure is necessary if the sample is large enough
Correct answer: Random assignment ensures internal validity; random selection ensures external validity
Random assignment controls for confounds to establish causality (internal validity), while random selection from a population allows generalization (external validity).
Question 4: What is an interaction effect in a factorial experiment?
- The combined total effect of all independent variables
- When the effect of one independent variable differs depending on the level of another (Correct answer)
- A statistically significant main effect for both factors
- When two dependent variables are measured simultaneously
Correct answer: When the effect of one independent variable differs depending on the level of another
An interaction effect occurs when the influence of one factor on the outcome depends on the level of another factor.
Question 5: In experimental research, what does 'attrition' refer to and why is it a threat to validity?
- Measurement error that accumulates over multiple assessments
- Participant dropout that may systematically bias the remaining sample (Correct answer)
- The weakening of treatment effects over time
- Gradual loss of statistical power due to small effect sizes
Correct answer: Participant dropout that may systematically bias the remaining sample
Attrition (dropout) threatens internal validity if participants who leave differ systematically from those who remain, creating a biased sample.
Question 6: A researcher assigns participants to groups based on their scores on a pre-test, putting the top half in Group A and the bottom half in Group B. What threat to internal validity does this introduce?
- Selection bias leading to non-equivalent groups
- Demand characteristics affecting participant behavior
- Instrumentation changes between testing sessions
- Regression to the mean affecting extreme scorers (Correct answer)
Correct answer: Regression to the mean affecting extreme scorers
Participants selected for extreme pre-test scores tend to score closer to the mean on post-tests regardless of treatment, a phenomenon called regression to the mean.
Question 7: Which of the following correctly distinguishes a confounding variable from a mediating variable?
- A confound explains the causal mechanism; a mediator distorts the relationship
- A confound distorts the observed relationship; a mediator explains the causal mechanism (Correct answer)
- Both confounds and mediators are types of independent variables
- Mediators must be controlled; confounds should be studied
Correct answer: A confound distorts the observed relationship; a mediator explains the causal mechanism
A confounding variable is a third variable that distorts the true relationship between X and Y, while a mediator explains the causal pathway through which X affects Y.
FAST Experimental Design 3
A scientist measures the same participants' anxiety levels before and after a mindfulness intervention.
Which statistical concern is most relevant?