CODESP Evaluating Selection Process Effectiveness 4 — Questions and Answers
Question 1: Which type of reliability is most critical to establish when a selection test is administered on different occasions to the same candidates?
- Test-retest reliability (Correct answer)
- Internal consistency
- Inter-rater reliability
- Parallel forms reliability
Correct answer: Test-retest reliability
Test-retest reliability assesses score stability over time, which is essential when candidates might be retested or scores used across multiple decision points.
Question 2: A city HR department finds its written exam has a 0.60 adverse impact ratio for Hispanic applicants. According to the 4/5ths rule, what does this indicate?
- Adverse impact is present because the ratio is below 0.80 (Correct answer)
- No adverse impact because the ratio exceeds 0.50
- Adverse impact is absent because it is above 0.40
- The test is invalid for Hispanic applicants
Correct answer: Adverse impact is present because the ratio is below 0.80
The 4/5ths rule flags adverse impact when the pass rate of a protected group is less than 80% (0.80) of the highest group's pass rate.
Question 3: What is the primary purpose of a job analysis in the context of validating a selection procedure?
- To identify the KSAOs required for successful job performance (Correct answer)
- To set salary ranges for the position
- To write employee performance appraisals
- To determine union eligibility for the position
Correct answer: To identify the KSAOs required for successful job performance
Job analysis identifies the knowledge, skills, abilities, and other characteristics (KSAOs) that form the basis for content- and construct-valid selection tools.
Question 4: An agency administers a physical ability test for firefighter candidates. The test is evaluated by checking whether tasks mirror actual firefighting activities. This reflects:
- Content validity evidence (Correct answer)
- Construct validity evidence
- Concurrent validity evidence
- Synthetic validity evidence
Correct answer: Content validity evidence
Content validity is established by demonstrating that the test tasks represent the critical duties identified in a job analysis of the firefighter position.
Question 5: Which statistic best summarizes the relationship between a continuous predictor and a dichotomous criterion (pass/fail on the job)?
- Point-biserial correlation (Correct answer)
- Pearson r
- Spearman rho
- Phi coefficient
Correct answer: Point-biserial correlation
The point-biserial correlation is used when one variable is continuous (test score) and the other is dichotomous (pass/fail criterion).
Question 6: When evaluating whether a cognitive ability test is appropriate for a clerical position, which evidence is most directly relevant?
- Job analysis showing cognitive demands of clerical tasks (Correct answer)
- National norms from the test publisher
- Test-retest reliability in unrelated samples
- Face validity ratings from applicants
Correct answer: Job analysis showing cognitive demands of clerical tasks
Linking test constructs to job-relevant cognitive demands through job analysis provides the most direct validity evidence for the intended use.
Question 7: An organization wants to compare the effectiveness of two selection methods. Which design allows the most direct causal inference about which method produces better hires?
- Random assignment of applicants to selection methods with follow-up performance data (Correct answer)
- Comparing current hires to historical records
- Having managers rate which method seems fairer
- Using the method with the higher face validity
Correct answer: Random assignment of applicants to selection methods with follow-up performance data
Random assignment eliminates selection bias and allows true causal comparison of which method better predicts subsequent job performance.
CODESP Evaluating Selection Process Effectiveness 4
Which type of reliability is most critical to establish when a selection test is administered on different occasions to the same candidates?