CBT Test Design and Blueprinting 4 — Questions and Answers
Question 1: In adaptive CBT design, a 'shadow test' technique is used to:
- Display a practice exam before the real test
- Ensure content blueprint constraints are satisfied during adaptive item selection (Correct answer)
- Create a backup test when the primary form fails
- Monitor candidate behavior for cheating
Correct answer: Ensure content blueprint constraints are satisfied during adaptive item selection
The shadow test technique assembles a complete feasible test in the background at each step of item selection to guarantee that blueprint constraints can still be met.
Question 2: Which test design strategy helps prevent item exposure in high-volume CBT programs?
- Using only one test form
- Maintaining a large item bank with randomized item selection (Correct answer)
- Publishing all items after each test window
- Reducing the number of candidates tested
Correct answer: Maintaining a large item bank with randomized item selection
A large item bank combined with randomized selection reduces the probability that any candidate sees the same items, protecting against memorization and sharing.
Question 3: What does 'form assembly' mean in CBT test design?
- Physically assembling test booklets
- Selecting and organizing items into a deliverable test that meets blueprint constraints (Correct answer)
- Training item writers to create new questions
- Printing test results for candidates
Correct answer: Selecting and organizing items into a deliverable test that meets blueprint constraints
Form assembly is the process of selecting items from the item bank and arranging them into a complete test that satisfies all content, statistical, and format requirements.
Question 4: A test blueprint section labeled 'item format distribution' would specify:
- How test scores are distributed among candidates
- The proportion of multiple-choice, drag-and-drop, and simulation items (Correct answer)
- The format of the score report sent to candidates
- How items are formatted in the item bank database
Correct answer: The proportion of multiple-choice, drag-and-drop, and simulation items
Item format distribution in a blueprint defines what percentage of items will use each format type (MCQ, hotspot, fill-in, etc.) to ensure variety and appropriate measurement.
Question 5: Which practice violates sound test blueprinting principles?
- Basing domain weights on JTA criticality ratings
- Adding items to meet a minimum bank size (Correct answer)
- Aligning cognitive levels to performance objectives
- Piloting new items before operational use
Correct answer: Adding items to meet a minimum bank size
Adding items solely to reach a bank size target, without regard to content or cognitive level specifications, undermines the validity and coherence of the blueprint.
Question 6: When a blueprint calls for 'context-dependent' item sets, test designers must be careful about:
- Using too many distractor options
- Local item dependence inflating apparent reliability (Correct answer)
- Scheduling delivery in multiple time zones
- Providing answer explanations during the test
Correct answer: Local item dependence inflating apparent reliability
Items sharing a common stimulus can be locally dependent, meaning performance on one item affects another, which can inflate reliability estimates if not properly modeled.
Question 7: In test blueprinting, 'content outline' and 'blueprint' are related because:
- They are identical documents with different names
- The content outline describes what is tested while the blueprint adds item counts and weights (Correct answer)
- The blueprint is created before the content outline
- The content outline is used only for essay exams
Correct answer: The content outline describes what is tested while the blueprint adds item counts and weights
The content outline lists the topics and subtopics to be assessed, while the blueprint builds on it by specifying how many items and at what cognitive level each topic is assessed.
CBT Test Design and Blueprinting 4
In adaptive CBT design, a 'shadow test' technique is used to: