KSA Psychometric Testing & Measurement Theory 1 — Questions and Answers
Question 1: Which type of reliability refers to the consistency of scores produced by the same test administered to the same group on two separate occasions?
- Inter-rater reliability
- Test-retest reliability (Correct answer)
- Internal consistency reliability
- Parallel forms reliability
Correct answer: Test-retest reliability
Test-retest reliability measures the stability of scores over time by correlating results from two administrations of the same test.
Question 2: In classical test theory, which formula expresses the relationship between an observed score, a true score, and measurement error?
- X = T - E
- X = T + E (Correct answer)
- T = X × E
- E = T / X
Correct answer: X = T + E
Classical test theory states that an observed score (X) equals the true score (T) plus random error (E): X = T + E.
Question 3: A KSA assessment that measures what it claims to measure is said to have high:
- Reliability
- Validity (Correct answer)
- Standardization
- Norming
Correct answer: Validity
Validity refers to the degree to which an assessment actually measures the construct or attribute it intends to measure.
Question 4: Which validity type is established by comparing assessment scores against a known, accepted criterion measured at the same time?
- Predictive validity
- Concurrent validity (Correct answer)
- Content validity
- Construct validity
Correct answer: Concurrent validity
Concurrent validity is established when the assessment scores correlate significantly with a criterion measured simultaneously.
Question 5: Item difficulty (p-value) in a dichotomous item refers to:
- The probability an item is biased
- The proportion of respondents who answer the item correctly (Correct answer)
- The variance of item scores
- The correlation between item score and total score
Correct answer: The proportion of respondents who answer the item correctly
Item difficulty (p-value) is calculated as the proportion of test-takers who answered the item correctly, ranging from 0 to 1.
Question 6: Cronbach's alpha is a measure of:
- Test-retest reliability
- Inter-rater agreement
- Internal consistency reliability (Correct answer)
- Predictive validity
Correct answer: Internal consistency reliability
Cronbach's alpha estimates internal consistency by measuring how closely related a set of items are as a group.
Question 7: The standard error of measurement (SEM) is BEST used to:
- Identify biased test items
- Estimate the range within which a candidate's true score likely falls (Correct answer)
- Compare mean scores across demographic groups
- Calculate the predictive validity coefficient
Correct answer: Estimate the range within which a candidate's true score likely falls
The SEM quantifies measurement error and is used to construct a confidence interval around an observed score to estimate where the true score lies.
KSA Psychometric Testing & Measurement Theory 1
Which type of reliability refers to the consistency of scores produced by the same test administered to the same group on two separate occasions?