CBT Scoring Models and Reporting 3 — Questions and Answers
Question 1: Which equating design collects data from a common group of examinees who take both old and new test forms?
- Random groups design
- Single group design (Correct answer)
- Anchor test (NEAT) design
- Calibration design
Correct answer: Single group design
The single group design administers both forms to the same examinees, directly linking the two forms through shared examinee performance.
Question 2: What is a 'passing score' also commonly called in credentialing CBT contexts?
- Norm score
- Cut score (Correct answer)
- Z-score
- Benchmark score
Correct answer: Cut score
A cut score (or cutoff score) is the minimum score required to pass a credentialing exam, set through a standard-setting process.
Question 3: Standard Error of Measurement (SEM) is used in CBT score reports to:
- Rank examinees against peers
- Indicate the precision of an individual's score (Correct answer)
- Calculate the passing rate
- Determine test form difficulty
Correct answer: Indicate the precision of an individual's score
SEM quantifies the degree of uncertainty in an individual's observed score, typically displayed as a confidence interval around that score.
Question 4: In a 3-parameter logistic (3PL) IRT model, what does the 'c' parameter represent?
- Item difficulty
- Item discrimination
- Lower asymptote / pseudo-guessing (Correct answer)
- Upper asymptote
Correct answer: Lower asymptote / pseudo-guessing
The 'c' parameter in 3PL IRT represents the lower asymptote, modeling the probability that a very low-ability examinee still answers the item correctly by guessing.
Question 5: Which type of score interpretation compares an examinee's performance to a defined standard rather than to other test-takers?
- Norm-referenced interpretation
- Criterion-referenced interpretation (Correct answer)
- Relative interpretation
- Ipsative interpretation
Correct answer: Criterion-referenced interpretation
Criterion-referenced interpretation judges whether a candidate meets a predetermined performance standard, independent of how others performed.
Question 6: A CBT exam has a reliability coefficient of 0.92. What does this indicate?
- 92% of candidates pass the exam
- The exam is 92% valid for its intended purpose
- 92% of score variance is true score variance (Correct answer)
- The cut score is at the 92nd percentile
Correct answer: 92% of score variance is true score variance
A reliability coefficient of 0.92 means that 92% of observed score variance is attributable to true differences in ability, with only 8% due to measurement error.
Question 7: The Bookmark standard-setting method involves panelists placing a bookmark in an ordered item booklet. The bookmark placement represents:
- The hardest item a master should answer correctly
- The first item a borderline candidate is unlikely to answer correctly (Correct answer)
- The median item difficulty in the bank
- The item with the highest discrimination index
Correct answer: The first item a borderline candidate is unlikely to answer correctly
In the Bookmark method, panelists place a bookmark before the first item where they believe a borderline candidate has less than a 67% chance of answering correctly.
CBT Scoring Models and Reporting 3
Which equating design collects data from a common group of examinees who take both old and new test forms?