CODESP Evaluating Selection Process Effectiveness 5 — Questions and Answers
Question 1: Which approach to setting cut scores relies on judgments about the minimum acceptable level of competence for a position?
- Angoff method (Correct answer)
- Contrasting groups method
- Bookmark method
- Borderline group method
Correct answer: Angoff method
The Angoff method asks subject matter experts to estimate the probability that a minimally competent candidate would answer each item correctly, then sums these probabilities.
Question 2: What does a Taylor-Russell table help practitioners determine in selection?
- The proportion of successful employees expected given a validity coefficient, base rate, and selection ratio (Correct answer)
- The number of items needed for a reliable test
- The adverse impact ratio for a given selection procedure
- The expected utility in dollar terms
Correct answer: The proportion of successful employees expected given a validity coefficient, base rate, and selection ratio
Taylor-Russell tables show the expected proportion of successful hires as a function of test validity, the base rate of success, and the selection ratio.
Question 3: An organization discovers that structured interviews predict performance but so does a cognitive test alone. The interviews add no unique predictive variance. This finding describes:
- Low incremental validity of the interview (Correct answer)
- Low criterion-related validity of the test
- High adverse impact of the interview
- Test-retest unreliability of the interview
Correct answer: Low incremental validity of the interview
When a new predictor explains no additional variance in the criterion beyond what an existing predictor captures, it has low or zero incremental validity.
Question 4: Which of the following best describes 'synthetic validity'?
- Inferring validity for a composite job by combining validity evidence from component job elements (Correct answer)
- Using statistical simulations to estimate future test validity
- Creating artificial job scenarios to test candidates
- Combining multiple validity studies through meta-analysis
Correct answer: Inferring validity for a composite job by combining validity evidence from component job elements
Synthetic validity builds overall job validity by assembling component-level validity evidence for each job element identified through job analysis.
Question 5: A selection process is found to be valid but results in disparate impact. Under what legal doctrine may it still be defensible?
- Business necessity (Correct answer)
- Bona fide occupational qualification (BFOQ)
- Affirmative action mandate
- Adverse impact exemption
Correct answer: Business necessity
Employers may defend a selection practice causing disparate impact by demonstrating business necessity—that the practice is job-related and required for safe, effective job performance.
Question 6: What is the primary advantage of using multiple hurdles (sequential screening) over a compensatory model in public safety selection?
- Ensures candidates meet minimum thresholds on all critical requirements before advancing (Correct answer)
- Maximizes overall predicted performance across all dimensions
- Reduces total testing time for all applicants
- Eliminates the need for job analysis
Correct answer: Ensures candidates meet minimum thresholds on all critical requirements before advancing
Multiple hurdles require candidates to pass each stage separately, preventing a high score on one dimension from compensating for failure on a critical safety requirement.
Question 7: Which of the following represents a criterion deficiency in a selection validity study for police officers?
- The performance measure excludes community relations skills that are critical to the job (Correct answer)
- The test includes items unrelated to police duties
- Supervisors know candidates' test scores when rating them
- The sample size is too small to detect significant correlations
Correct answer: The performance measure excludes community relations skills that are critical to the job
Criterion deficiency occurs when important job performance dimensions are omitted from the criterion measure, limiting the validity study's scope.
CODESP Evaluating Selection Process Effectiveness 5
Which approach to setting cut scores relies on judgments about the minimum acceptable level of competence for a position?