SIOP Psychological Assessment & Measurement Techniques 2 — Questions and Answers
Question 1: Which type of validity evidence examines whether a test's internal structure matches its theoretical constructs?
- Criterion-related validity
- Content validity
- Internal structure validity (Correct answer)
- Consequential validity
Correct answer: Internal structure validity
Internal structure validity evidence (part of construct validity) examines whether item relationships and factor structure align with the theoretical model.
Question 2: A selection test with a 0.40 criterion-related validity coefficient is used with a base rate of 50% and a selection ratio of 20%. Using the Taylor-Russell tables, what effect does lowering the selection ratio have on the success ratio?
- It decreases the success ratio
- It increases the success ratio (Correct answer)
- It has no effect on the success ratio
- It only affects utility when base rate exceeds 70%
Correct answer: It increases the success ratio
Lower selection ratios (hiring fewer, more selective) increase the success ratio because you are selecting only top scorers from a larger pool.
Question 3: The Angoff method, Bookmark method, and Contrasting Groups method are all approaches to:
- Calculating interrater reliability
- Setting passing scores (cut scores) (Correct answer)
- Evaluating differential item functioning
- Estimating test-retest reliability
Correct answer: Setting passing scores (cut scores)
The Angoff, Bookmark, and Contrasting Groups methods are standard-setting procedures used to establish defensible cut scores on employment or credentialing tests.
Question 4: Which phenomenon occurs when candidates' scores on a selection instrument improve substantially upon retesting, threatening the validity of the instrument?
- Regression to the mean
- Coaching and practice effects (Correct answer)
- Range restriction
- Criterion contamination
Correct answer: Coaching and practice effects
Coaching and practice effects occur when retesting leads to score gains unrelated to the underlying construct, threatening test validity for repeat test-takers.
Question 5: In Item Response Theory (IRT), the 'b' parameter represents:
- Item discrimination
- Item difficulty (location) (Correct answer)
- Pseudo-guessing probability
- Item information function
Correct answer: Item difficulty (location)
In IRT, the b parameter is the item difficulty or location parameter, indicating the trait level at which the probability of a correct response is 0.50 (in 1PL/2PL models).
Question 6: A company uses a cognitive ability test that shows adverse impact against a protected group. Under the Uniform Guidelines, what is the FIRST recommended action?
- Discontinue use of the test immediately
- Demonstrate the test's job-relatedness and business necessity (Correct answer)
- Lower the cut score to reduce adverse impact
- Switch to an unproctored internet test
Correct answer: Demonstrate the test's job-relatedness and business necessity
Under the Uniform Guidelines, when adverse impact exists the employer must demonstrate job-relatedness and validity evidence to justify continued use of the selection procedure.
Question 7: Which reliability coefficient is most appropriate when a test measures a single, unidimensional construct and all items share equal variance?
- Kuder-Richardson Formula 20 (KR-20)
- Spearman-Brown formula
- KR-21 (Correct answer)
- Test-retest reliability
Correct answer: KR-21
KR-21 is appropriate when items measure a single construct and all items are assumed to have equal difficulty (equal variance), making it simpler to calculate than KR-20.
SIOP Psychological Assessment & Measurement Techniques 2
Which type of validity evidence examines whether a test's internal structure matches its theoretical constructs?