ETS Validity and Reliability 2 — Questions and Answers
Question 1: A test that consistently measures the wrong construct has high _____ but low _____.
- reliability; validity (Correct answer)
- validity; reliability
- internal consistency; test-retest reliability
- content validity; construct validity
Correct answer: reliability; validity
A test can produce consistent (reliable) scores while still failing to measure the intended construct (invalid).
Question 2: Which validity evidence is most important when a test is used to predict future job performance?
- Content validity
- Criterion-related validity (Correct answer)
- Face validity
- Construct validity
Correct answer: Criterion-related validity
Criterion-related validity (specifically predictive validity) examines how well test scores forecast a future criterion like job performance.
Question 3: A researcher administers the same test twice within a short interval and correlates the two sets of scores. This assesses:
- Alternate-form reliability
- Internal consistency
- Test-retest reliability (Correct answer)
- Inter-rater reliability
Correct answer: Test-retest reliability
Test-retest reliability is measured by correlating scores from two administrations of the same test separated by a time interval.
Question 4: When validity evidence is gathered by examining the internal structure of a test, this is called:
- Concurrent validity
- Structural validity (Correct answer)
- Consequential validity
- Face validity
Correct answer: Structural validity
Structural (internal structure) validity evidence examines whether the relationships among test items match the intended construct's theoretical structure.
Question 5: The Standards for Educational and Psychological Testing treats validity as:
- A property of the test itself
- A property of score interpretations and uses (Correct answer)
- Determined solely by reliability coefficients
- Fixed once established during test development
Correct answer: A property of score interpretations and uses
According to the Standards, validity refers to the degree to which evidence supports the interpretation and use of test scores, not the test itself.
Question 6: A math achievement test that includes overly complex reading passages would violate which principle?
- Construct-irrelevant variance must be minimized (Correct answer)
- Tests should have high face validity
- Alternate forms must be equivalent
- Standard error of measurement must be low
Correct answer: Construct-irrelevant variance must be minimized
Construct-irrelevant variance occurs when factors unrelated to the target construct (reading difficulty) affect scores on a math test.
Question 7: Which coefficient is most commonly used to report test-retest reliability?
- Phi coefficient
- Pearson product-moment correlation (Correct answer)
- Kuder-Richardson 20
- Cohen's kappa
Correct answer: Pearson product-moment correlation
The Pearson product-moment correlation coefficient is used to quantify the relationship between scores from two test administrations.
ETS Validity and Reliability 2
A test that consistently measures the wrong construct has high _____ but low _____.