CBT Test Design and Blueprinting 3 — Questions and Answers
Question 1: What does the term 'cut score' mean in the context of a CBT blueprint?
- The maximum score a test can produce
- The minimum score required to pass the exam (Correct answer)
- The score at which items are removed from the bank
- The average score across all test forms
Correct answer: The minimum score required to pass the exam
A cut score is the minimum performance level a candidate must achieve to be classified as passing, established through a standard-setting process.
Question 2: In test design, 'item stem' refers to:
- The correct answer option
- The question or problem posed to the candidate (Correct answer)
- The distractor set
- The scoring rubric
Correct answer: The question or problem posed to the candidate
The item stem is the question, scenario, or problem statement that precedes the answer options.
Question 3: Which of the following best describes 'content domain weighting' in a test blueprint?
- Ranking questions by difficulty
- Assigning percentage of items to each topic area (Correct answer)
- Ordering topics alphabetically
- Selecting only the most recent content
Correct answer: Assigning percentage of items to each topic area
Content domain weighting specifies what proportion of test items should come from each subject area, ensuring coverage matches job relevance findings.
Question 4: A blueprint that includes both 'recall' and 'application' items is addressing which aspect of test design?
- Score equating
- Cognitive level distribution (Correct answer)
- Item format diversity
- Time-on-task analysis
Correct answer: Cognitive level distribution
Including items at multiple cognitive levels (recall vs. application) ensures the test measures different depths of understanding as specified in the blueprint.
Question 5: Why is a 'table of specifications' used during test blueprint development?
- To list hardware requirements for the testing center
- To cross-reference content topics with cognitive levels and item counts (Correct answer)
- To document candidate demographic information
- To record test administration logistics
Correct answer: To cross-reference content topics with cognitive levels and item counts
A table of specifications is a grid that aligns content topics with cognitive levels to show exactly how many items will assess each combination.
Question 6: Which document is typically produced FIRST when designing a new CBT certification exam?
- Item writing guidelines
- Job task analysis report (Correct answer)
- Psychometric scoring model
- Candidate handbook
Correct answer: Job task analysis report
The job task analysis identifies critical job duties and required KSAs, providing the empirical foundation upon which the blueprint and all subsequent documents are built.
Question 7: What is the risk of having too many items from a single content domain in a CBT blueprint?
- Higher test reliability
- Content imbalance that threatens content validity (Correct answer)
- Faster item bank depletion
- Lower candidate pass rates
Correct answer: Content imbalance that threatens content validity
Overweighting one domain means the test does not proportionally represent all required KSAs, undermining the test's ability to validly measure overall competency.
CBT Test Design and Blueprinting 3
What does the term 'cut score' mean in the context of a CBT blueprint?