CBT Scoring Models and Reporting 2 — Questions and Answers
Question 1: What does a scaled score accomplish in CBT reporting?
- It shows the exact number of correct answers
- It converts raw scores to a common scale across different test forms (Correct answer)
- It ranks examinees relative to each other
- It eliminates the need for a passing score
Correct answer: It converts raw scores to a common scale across different test forms
Scaled scores equate different test forms to a common scale so that scores are comparable regardless of which version an examinee received.
Question 2: In Item Response Theory (IRT), the 'b' parameter refers to:
- Item discrimination
- Item difficulty (Correct answer)
- Guessing probability
- Test reliability
Correct answer: Item difficulty
The 'b' parameter in IRT represents item difficulty, indicating the ability level at which an examinee has a 50% chance of answering correctly.
Question 3: Which standard-setting method requires panelists to estimate the probability that a borderline candidate answers each item correctly?
- Angoff method (Correct answer)
- Bookmark method
- Contrasting groups method
- Body of work method
Correct answer: Angoff method
The Angoff method asks panelists to estimate the probability that a minimally competent (borderline) candidate would answer each item correctly, and the average becomes the cut score.
Question 4: A score report shows a candidate scored at the 78th percentile. This means:
- The candidate answered 78% of questions correctly
- The candidate scored higher than 78% of the norm group (Correct answer)
- The candidate needs 22 more points to pass
- The candidate's score is 78 on a 100-point scale
Correct answer: The candidate scored higher than 78% of the norm group
A percentile rank of 78 means the candidate performed better than 78% of the reference (norm) group.
Question 5: What is the primary purpose of vertical scaling in educational CBT programs?
- To equate forms given in different countries
- To compare scores across different grade levels over time (Correct answer)
- To adjust for test-taking time differences
- To convert raw scores to standard scores
Correct answer: To compare scores across different grade levels over time
Vertical scaling links tests across different grade levels so student growth can be measured on a single developmental score scale.
Question 6: In CBT adaptive testing, what does the test information function (TIF) describe?
- The total number of items in the bank
- How precisely the test measures ability at different levels (Correct answer)
- The order in which items are presented
- The time limit for each section
Correct answer: How precisely the test measures ability at different levels
The TIF shows the precision (inverse of measurement error) of the test at different ability levels, helping designers ensure accurate measurement where it matters most.
Question 7: Which score report element is most useful for a candidate who failed a CBT to plan remediation?
- Total scaled score
- Percentile rank
- Domain or section subscores (Correct answer)
- Test form number
Correct answer: Domain or section subscores
Domain or section subscores show where performance was weakest, enabling targeted remediation rather than re-studying everything.
CBT Scoring Models and Reporting 2
What does a scaled score accomplish in CBT reporting?