PeopleAnswers Science and Validation 3 — Questions and Answers
Question 1: In the context of behavioral assessments, what does 'adverse impact' refer to?
- Disproportionate negative outcomes for a protected group (Correct answer)
- Test questions that create stress
- Low reliability coefficients
- Negative candidate feedback about the test
Correct answer: Disproportionate negative outcomes for a protected group
Adverse impact occurs when a selection procedure results in substantially different selection rates across groups defined by race, sex, or other protected characteristics.
Question 2: The four-fifths (4/5) rule is used to evaluate:
- Whether adverse impact exists in a selection process (Correct answer)
- The minimum reliability threshold for a valid assessment
- The required sample size for a validity study
- The number of questions per competency dimension
Correct answer: Whether adverse impact exists in a selection process
The 4/5 rule flags potential adverse impact when the selection rate of a protected group is less than 80% of the highest-selecting group's rate.
Question 3: What does 'incremental validity' measure in the context of adding an assessment to a hiring process?
- The additional predictive power beyond existing selection tools (Correct answer)
- The total number of valid items in a test
- The increase in candidate pool size
- The change in test score scale over time
Correct answer: The additional predictive power beyond existing selection tools
Incremental validity quantifies how much predictive accuracy an assessment adds over and above what other tools (e.g., interviews, resumes) already provide.
Question 4: Which psychometric term describes the consistency of results when two different evaluators score the same response?
- Inter-rater reliability (Correct answer)
- Test-retest reliability
- Internal consistency
- Parallel-forms reliability
Correct answer: Inter-rater reliability
Inter-rater reliability measures agreement between different scorers rating the same response, ensuring objectivity in assessment scoring.
Question 5: A norm group in a pre-employment assessment is best described as:
- A reference sample used to interpret individual scores (Correct answer)
- Employees who performed below average
- The standardization committee for the test
- The minimum acceptable score threshold
Correct answer: A reference sample used to interpret individual scores
A norm group provides a comparative baseline, allowing an individual's score to be interpreted relative to a representative population.
Question 6: What does a reliability coefficient of 0.90 indicate about a behavioral assessment?
- 90% of score variance reflects true individual differences, not error (Correct answer)
- 90% of candidates passed the assessment
- The test predicts performance with 90% accuracy
- Nine out of ten questions are well-written
Correct answer: 90% of score variance reflects true individual differences, not error
A reliability coefficient of 0.90 means 90% of score variance is attributable to true differences among test-takers rather than measurement error.
Question 7: Criterion-related validity studies in employment assessments most commonly use which criterion measure?
- Supervisor job performance ratings (Correct answer)
- Candidate self-reported satisfaction
- Number of training hours completed
- Length of employment tenure only
Correct answer: Supervisor job performance ratings
Supervisor performance ratings are the most widely used criterion in employment validation studies because they directly reflect job effectiveness.
PeopleAnswers Science and Validation 3
In the context of behavioral assessments, what does 'adverse impact' refer to?