Scoring Models and Reporting Flashcards
7 cards from real CBT practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 Scoring Models and Reporting flashcards as text
What is the primary advantage of using IRT over Classical Test Theory (CTT) for CBT item calibration?
Answer: IRT item statistics are independent of the sample used for calibration
IRT item parameters are theoretically sample-independent (invariant), meaning item difficulty and discrimination estimates remain stable across different examinee groups.
In which situation would a testing program most appropriately use norm-referenced scoring rather than criterion-referenced scoring?
Answer: Selection exams used to rank applicants for limited spots
Norm-referenced scoring is appropriate when the goal is ranking and selecting a fixed number of candidates, such as admissions or competitive job selection.
What does 'conditional standard error of measurement' (CSEM) provide that a single overall SEM cannot?
Answer: Precision estimates that vary at different points along the score scale
CSEM shows how measurement precision varies at each ability level, which is particularly valuable for CAT where precision near the cut score is most critical.
A testing program reports 'classification consistency.' This statistic measures:
Answer: The likelihood that examinees receive the same pass/fail decision on retesting
Classification consistency estimates the probability that an examinee would receive the same pass or fail decision if they retested under identical conditions.
Which phenomenon explains why a group's average score on a CBT tends to rise over multiple years without any actual improvement in knowledge?
Answer: Teach-to-the-test and score inflation
When item content becomes familiar through coaching and test-prep, scores can rise due to exposure rather than genuine competency gains, a phenomenon known as score inflation.
What is the role of the 'anchor items' (common items) in the Non-Equivalent Anchor Test (NEAT) equating design?
Answer: They link the two test forms by providing a common statistical thread across different examinee groups
Anchor items appear on both test forms and allow score equating by statistically linking the forms even when different groups of examinees took each form.
A CBT score report for a certification exam includes a 'score validity period.' What does this communicate?
Answer: The time window during which the score is officially recognized for credentialing purposes
A score validity period specifies how long the score is accepted for credentialing decisions, after which the candidate must retest because knowledge in the field may have changed.