SIOP Research Methods & Data Analysis in I-O Psychology 3 — Questions and Answers
Question 1: Which type of validity evidence is gathered by examining correlations between a measure and theoretically related and unrelated constructs?
- Content validity
- Criterion-related validity
- Convergent and discriminant validity (Correct answer)
- Face validity
Correct answer: Convergent and discriminant validity
Convergent validity is shown by high correlations with related constructs; discriminant validity by low correlations with unrelated constructs.
Question 2: In meta-analysis, what does the Q statistic test?
- The overall effect size across studies
- Heterogeneity of effect sizes across studies (Correct answer)
- Publication bias in the sample
- Statistical power of each study
Correct answer: Heterogeneity of effect sizes across studies
The Q statistic tests whether observed variance in effect sizes across studies exceeds what is expected by sampling error alone.
Question 3: A job analysis reveals that a selection test measures knowledge not needed until 6 months on the job. This is most relevant to which type of criterion-related validity?
- Concurrent validity
- Predictive validity (Correct answer)
- Construct validity
- Incremental validity
Correct answer: Predictive validity
Predictive validity involves measuring a predictor now and relating it to future criterion performance, making the timing concern relevant.
Question 4: What is the primary purpose of conducting a pilot study before a large-scale survey?
- To maximize statistical power
- To identify flaws in survey items and procedures (Correct answer)
- To establish population norms
- To meet IRB requirements
Correct answer: To identify flaws in survey items and procedures
Pilot studies reveal ambiguous items, poor response formats, and procedural problems before full deployment.
Question 5: Which data collection method is most susceptible to demand characteristics?
- Archival records
- Objective performance data
- Self-report questionnaires in a lab (Correct answer)
- Unobtrusive observation
Correct answer: Self-report questionnaires in a lab
Lab self-report measures are most vulnerable because participants may guess study hypotheses and respond accordingly.
Question 6: In hierarchical regression, a researcher adds a block of predictors in Step 2. The change in R² (ΔR²) represents:
- Total variance explained by all predictors
- Variance explained by Step 2 predictors above and beyond Step 1 (Correct answer)
- Correlation between Step 1 and Step 2 predictors
- Effect size for individual predictors in Step 2
Correct answer: Variance explained by Step 2 predictors above and beyond Step 1
ΔR² in hierarchical regression reflects the incremental variance explained by the new block of predictors over those entered in prior steps.
Question 7: Which sampling strategy ensures every member of the population has an equal probability of selection?
- Convenience sampling
- Purposive sampling
- Simple random sampling (Correct answer)
- Snowball sampling
Correct answer: Simple random sampling
Simple random sampling assigns each population member an equal probability of being chosen, supporting representativeness.
SIOP Research Methods & Data Analysis in I-O Psychology 3
Which type of validity evidence is gathered by examining correlations between a measure and theoretically related and unrelated constructs?