CBT Scoring Models and Reporting 5

Question 1

What is the primary advantage of using IRT over Classical Test Theory (CTT) for CBT item calibration?

Accepted Answer

IRT item statistics are independent of the sample used for calibration

Answer

IRT item parameters are theoretically sample-independent (invariant), meaning item difficulty and discrimination estimates remain stable across different examinee groups.

Question 2

In which situation would a testing program most appropriately use norm-referenced scoring rather than criterion-referenced scoring?

Accepted Answer

Selection exams used to rank applicants for limited spots

Answer

Norm-referenced scoring is appropriate when the goal is ranking and selecting a fixed number of candidates, such as admissions or competitive job selection.

Question 3

What does 'conditional standard error of measurement' (CSEM) provide that a single overall SEM cannot?

Accepted Answer

Precision estimates that vary at different points along the score scale

Answer

CSEM shows how measurement precision varies at each ability level, which is particularly valuable for CAT where precision near the cut score is most critical.

Question 4

A testing program reports 'classification consistency.' This statistic measures:

Accepted Answer

The likelihood that examinees receive the same pass/fail decision on retesting

Answer

Classification consistency estimates the probability that an examinee would receive the same pass or fail decision if they retested under identical conditions.

Question 5

Which phenomenon explains why a group's average score on a CBT tends to rise over multiple years without any actual improvement in knowledge?

Accepted Answer

Teach-to-the-test and score inflation

Answer

When item content becomes familiar through coaching and test-prep, scores can rise due to exposure rather than genuine competency gains, a phenomenon known as score inflation.

CBT - Computer Based Testing Practice Test

CBT - Computer Based Testing Practice Test

CBT Scoring Models and Reporting 5