CODESP Making Final Selection Decisions 3 — Questions and Answers
Question 1: An agency is considering using a 'rule of three' in final selection, meaning only the top three ranked candidates are referred to the hiring manager. What is the primary criticism of this practice?
- It gives the hiring manager too little discretion in the final decision
- It may be unnecessarily restrictive and ignore score differences that fall within measurement error (Correct answer)
- It always produces adverse impact against minority groups
- It violates the compensatory selection model requirements
Correct answer: It may be unnecessarily restrictive and ignore score differences that fall within measurement error
The rule of three can be arbitrary because rank differences near the cutoff may not be meaningfully different given measurement error.
Question 2: In the context of final selection decisions, what is the purpose of a 'structured credential review' as a final step before making an offer?
- To re-score the candidate's written examination under standardized conditions
- To verify that the selected candidate meets all stated minimum qualifications and certifications (Correct answer)
- To conduct an additional unscored behavioral interview
- To compare the candidate's salary expectations against the job's pay grade
Correct answer: To verify that the selected candidate meets all stated minimum qualifications and certifications
A structured credential review ensures that the top-scoring candidate actually possesses the required licenses, education, or certifications listed in the job announcement.
Question 3: Which of the following scenarios best illustrates the concept of 'incremental validity' in building a final selection battery?
- Adding a second test that measures the same construct as the first test and increases overall test length
- Adding a personality inventory that predicts job performance above and beyond what a cognitive ability test already predicts (Correct answer)
- Replacing an invalid test with a valid one
- Increasing the cut score on an existing test to hire fewer candidates
Correct answer: Adding a personality inventory that predicts job performance above and beyond what a cognitive ability test already predicts
Incremental validity refers to the additional predictive power a new measure provides over and above existing predictors in the battery.
Question 4: When setting a cut score using the Angoff method, what is each panel member asked to do?
- Score a sample of actual candidate responses under timed conditions
- Estimate the probability that a minimally competent candidate would answer each item correctly (Correct answer)
- Rank all items from easiest to hardest based on their own job experience
- Identify which items show differential item functioning across demographic groups
Correct answer: Estimate the probability that a minimally competent candidate would answer each item correctly
In the Angoff method, subject matter experts estimate, for each item, the proportion of minimally competent candidates who would answer it correctly, and these estimates are averaged to set the cut score.
Question 5: A final selection decision that results in adverse impact is legally defensible if the employer can demonstrate:
- That adverse impact was unintentional and not motivated by discriminatory intent
- That the selection procedure is job-related and consistent with business necessity (Correct answer)
- That the proportion of minority hires meets the 80% rule in subsequent hiring cycles
- That the procedure is less costly than alternative assessment methods
Correct answer: That the selection procedure is job-related and consistent with business necessity
Under Title VII and the Uniform Guidelines, adverse impact can be justified by demonstrating that the procedure is job-related and consistent with business necessity.
Question 6: In a concurrent validity study used to support a cut score, which limitation must selection professionals be most cautious about?
- The study uses current employees rather than applicants, which may underestimate predictor range (Correct answer)
- The study collects criterion data before predictor data, reversing the intended sequence
- The study requires more participants than predictive validity studies
- The study cannot be used for any public sector selection procedure
Correct answer: The study uses current employees rather than applicants, which may underestimate predictor range
Concurrent studies use incumbent employees who represent a restricted range of ability, which can attenuate validity coefficients and make cut score generalization to applicants problematic.
Question 7: When a selection battery uses a multiple cutoff approach with fixed minimum scores on each predictor, what assumption about the predictors underlies this design?
- Each predictor contributes equally to a single composite score
- Each predictor measures a competency that cannot be compensated for by high performance on another (Correct answer)
- The predictors are all intercorrelated above 0.70 and measure the same construct
- Candidates who fail one cutoff will automatically fail all remaining cutoffs
Correct answer: Each predictor measures a competency that cannot be compensated for by high performance on another
Multiple cutoffs assume that minimum proficiency on each measured dimension is required, meaning deficiency in one area cannot be compensated by strength in another.
CODESP Making Final Selection Decisions 3
An agency is considering using a 'rule of three' in final selection, meaning only the top three ranked candidates are referred to the hiring manager.
What is the primary criticism of this practice?