CBT Adaptive Testing and Computer Adaptive Testing (CAT) 1 — Questions and Answers
Question 1: In a Computer Adaptive Test (CAT), how is the difficulty of the next item typically determined?
- By random selection from the item bank
- Based on the examinee's response to the previous item (Correct answer)
- By the test administrator's judgment
- By fixed sequential order in the blueprint
Correct answer: Based on the examinee's response to the previous item
CAT algorithms select the next item based on the estimated ability level derived from the examinee's responses to prior items.
Question 2: Which psychometric model is most commonly used as the foundation for Computer Adaptive Testing?
- Classical Test Theory (CTT)
- Item Response Theory (IRT) (Correct answer)
- Generalizability Theory (GT)
- Factor Analysis
Correct answer: Item Response Theory (IRT)
IRT provides item-level ability estimates and item characteristic curves that allow CAT algorithms to adaptively select items and estimate examinee ability in real time.
Question 3: What is the primary advantage of a CAT over a fixed-form test of the same length?
- Lower development cost
- Greater measurement precision across all ability levels (Correct answer)
- Simpler scoring algorithms
- Larger item exposure rates
Correct answer: Greater measurement precision across all ability levels
CAT targets items near each examinee's ability level, yielding more precise ability estimates than fixed-form tests that include many items too easy or too hard for a given examinee.
Question 4: Which stopping rule in CAT terminates the test when the standard error of the ability estimate falls below a specified threshold?
- Fixed-length stopping rule
- Variable-length stopping rule (Correct answer)
- Time-based stopping rule
- Content-based stopping rule
Correct answer: Variable-length stopping rule
Variable-length stopping rules end the test once measurement precision reaches a desired level, meaning each examinee may answer a different number of items.
Question 5: Item exposure control in CAT is implemented primarily to:
- Reduce test administration time
- Prevent overuse of items that could compromise test security (Correct answer)
- Increase item bank diversity requirements
- Standardize item difficulty across all administrations
Correct answer: Prevent overuse of items that could compromise test security
Exposure control limits how frequently any single item is administered so that popular items are not memorized and shared by examinees, protecting test security.
Question 6: In IRT-based CAT, the Maximum Information (MI) item selection algorithm selects the next item that:
- Has the highest difficulty parameter regardless of current theta
- Provides the most information at the current ability estimate (theta) (Correct answer)
- Was least recently used in previous administrations
- Has the broadest content coverage in the blueprint
Correct answer: Provides the most information at the current ability estimate (theta)
The MI algorithm selects the item whose Fisher information function value is highest at the current theta estimate, maximizing measurement precision at that ability level.
Question 7: Which of the following is a key challenge when implementing content balancing constraints in a CAT?
- Eliminating the need for an item bank
- Ensuring blueprint specifications are met while still selecting near-optimal items (Correct answer)
- Removing the standard error stopping criterion
- Preventing item parameter drift over time
Correct answer: Ensuring blueprint specifications are met while still selecting near-optimal items
Content balancing requires selecting items that satisfy blueprint content constraints, which may force the algorithm to choose a slightly less informative item, creating a trade-off between optimality and validity.
CBT Adaptive Testing and Computer Adaptive Testing (CAT) 1
In a Computer Adaptive Test (CAT), how is the difficulty of the next item typically determined?