SIOP Research Methods & Data Analysis in I-O Psychology 4 — Questions and Answers
Question 1: An I-O researcher finds a statistically significant result with p = .04 but a Cohen's d of 0.08. What is the most accurate interpretation?
- The finding is both statistically and practically significant
- The effect is statistically significant but practically trivial (Correct answer)
- The study must have had low statistical power
- The null hypothesis should be retained
Correct answer: The effect is statistically significant but practically trivial
A very small effect size (d = 0.08) indicates minimal practical significance even when the result is statistically significant, often due to large sample size.
Question 2: Which threat to external validity occurs when the effect of a treatment is specific to the type of people who volunteered for the study?
- History
- Interaction of selection and treatment (Correct answer)
- Instrumentation bias
- Regression to the mean
Correct answer: Interaction of selection and treatment
The interaction of selection and treatment limits generalizability when findings only hold for the specific volunteer sample studied.
Question 3: Which reliability estimate is most appropriate for assessing the consistency of a single administration of a test?
- Test-retest reliability
- Alternate forms reliability
- Inter-rater reliability
- Coefficient alpha (Cronbach's α) (Correct answer)
Correct answer: Coefficient alpha (Cronbach's α)
Coefficient alpha estimates internal consistency reliability from a single administration by examining inter-item correlations.
Question 4: A researcher controls for a variable by including it as a covariate in ANCOVA. What is the primary goal of this procedure?
- Increase Type I error
- Remove variance due to the covariate to clarify treatment effects (Correct answer)
- Replace random assignment
- Ensure the covariate predicts the grouping variable
Correct answer: Remove variance due to the covariate to clarify treatment effects
ANCOVA statistically removes variance attributable to the covariate, reducing error variance and isolating the treatment's effect.
Question 5: Which type of research question is best answered using qualitative methods rather than quantitative methods?
- What is the mean level of burnout across nurses nationwide?
- Does leadership style predict turnover intent?
- How do employees make sense of a corporate merger? (Correct answer)
- What percentage of workers report high engagement?
Correct answer: How do employees make sense of a corporate merger?
Qualitative methods are best suited for exploring how and why questions, including how individuals interpret and give meaning to events.
Question 6: In structural equation modeling (SEM), what does a latent variable represent?
- An observed score measured without error
- A directly observable construct
- An unobserved theoretical construct inferred from indicators (Correct answer)
- A residual error term
Correct answer: An unobserved theoretical construct inferred from indicators
Latent variables in SEM represent hypothetical constructs that are estimated from multiple observed indicator variables.
Question 7: Which index in confirmatory factor analysis (CFA) indicates acceptable model fit when its value is ≥ .90?
- Root Mean Square Error of Approximation (RMSEA)
- Chi-square statistic
- Comparative Fit Index (CFI) (Correct answer)
- Standardized Root Mean Square Residual (SRMR)
Correct answer: Comparative Fit Index (CFI)
CFI values of .90 or higher (with .95+ preferred) indicate that the proposed model fits the data acceptably.
SIOP Research Methods & Data Analysis in I-O Psychology 4
An I-O researcher finds a statistically significant result with p = .04 but a Cohen's d of 0.08.
What is the most accurate interpretation?