PC PC Test Development and Validation 2 — Questions and Answers
Question 1: Criterion-related validity evidence is obtained by:
- Having experts review item content
- Correlating test scores with an external measure of the same construct (Correct answer)
- Conducting a confirmatory factor analysis
- Reviewing the test blueprint
Correct answer: Correlating test scores with an external measure of the same construct
Criterion-related validity is established by statistically correlating test scores with a relevant external criterion, such as job performance ratings.
Question 2: In test development, which item format is most susceptible to guessing, potentially inflating scores?
- Short answer
- True/False (Correct answer)
- Constructed response
- Matching
Correct answer: True/False
True/False items have a 50% probability of a correct guess by chance, making them the most susceptible to guessing among common item formats.
Question 3: Test equating in large-scale psychometric programs is performed to:
- Ensure that scores from different test forms are comparable (Correct answer)
- Eliminate poorly performing items from a test form
- Calculate the reliability of a single test form
- Identify biased items using DIF analysis
Correct answer: Ensure that scores from different test forms are comparable
Test equating adjusts scores across multiple test forms so that a given score reflects the same level of ability regardless of which form was administered.
Question 4: Which principle of test development requires that instructions, administration conditions, and scoring procedures be identical for all examinees?
- Standardization (Correct answer)
- Norming
- Calibration
- Triangulation
Correct answer: Standardization
Standardization ensures that all examinees experience the same testing conditions, making scores meaningful and comparable across individuals.
Question 5: Convergent validity evidence for a new psychometric instrument is demonstrated when:
- The instrument correlates highly with established measures of the same construct (Correct answer)
- The instrument shows low correlation with measures of unrelated constructs
- The instrument has high internal consistency
- Expert reviewers agree on item content
Correct answer: The instrument correlates highly with established measures of the same construct
Convergent validity is supported when the new measure correlates strongly with other measures theoretically designed to assess the same or similar constructs.
Question 6: The purpose of anchor items (common items) in psychometric test development is to:
- Provide extra credit opportunities for examinees
- Link different test forms to enable score equating (Correct answer)
- Identify examinees who are guessing
- Measure the difficulty of the easiest content
Correct answer: Link different test forms to enable score equating
Anchor items appear on multiple test forms and provide statistical links between forms, enabling equating so scores can be compared across administrations.
PC PC Test Development and Validation 2
Criterion-related validity evidence is obtained by: