CBT Test Design and Blueprinting 5 — Questions and Answers
Question 1: Which stakeholder group is most critical to consult during the job task analysis phase of test design?
- Software developers building the platform
- Subject matter experts currently performing the job (Correct answer)
- Candidates who have previously passed the exam
- Psychometricians setting the cut score
Correct answer: Subject matter experts currently performing the job
Subject matter experts (SMEs) who actively perform the job provide the most accurate and current information about the tasks and KSAs that should be assessed.
Question 2: A CBT blueprint that specifies a minimum of 5 items per domain ensures:
- That each domain contributes enough items for stable domain-level score reporting (Correct answer)
- That the test length is capped at 5 questions
- That harder domains have fewer items
- That candidates must answer all 5 items correctly to pass
Correct answer: That each domain contributes enough items for stable domain-level score reporting
Setting a minimum item count per domain ensures there are enough items to provide reliable subscores or domain-level diagnostic feedback.
Question 3: What is 'blueprint fidelity' in the context of computerized test delivery?
- The clarity of the computer screen display
- The degree to which the delivered test matches the content and statistical specifications of the blueprint (Correct answer)
- The accuracy of candidate identity verification
- The speed at which items are delivered
Correct answer: The degree to which the delivered test matches the content and statistical specifications of the blueprint
Blueprint fidelity refers to how closely the actual delivered test conforms to all of the specifications (domain weights, cognitive levels, formats) laid out in the blueprint.
Question 4: Why might a test design team include 'unscored tutorial items' at the start of a CBT?
- To warm up the computer's processing speed
- To familiarize candidates with item formats before scored questions begin (Correct answer)
- To measure baseline anxiety levels
- To collect demographic survey data
Correct answer: To familiarize candidates with item formats before scored questions begin
Tutorial items give candidates practice with the interface and item formats so that unfamiliarity with the technology does not disadvantage them on scored items.
Question 5: A blueprint specifying that no two consecutive items share the same content subdomain is an example of:
- Item sequencing constraint (Correct answer)
- Adaptive branching rule
- Passage dependency mapping
- Cognitive fatigue modeling
Correct answer: Item sequencing constraint
Item sequencing constraints in a blueprint control the order in which items from different content areas appear, often to prevent clustering and reduce context effects.
Question 6: When reviewing a test blueprint for legal defensibility, which element is most important to verify?
- The color scheme of the test interface
- That domain weights are empirically tied to JTA criticality data (Correct answer)
- That the test was translated into multiple languages
- That the test was developed within budget
Correct answer: That domain weights are empirically tied to JTA criticality data
Legal defensibility requires that blueprint domain weights can be traced to documented JTA findings showing the relative importance of each content area for job performance.
Question 7: In blueprint terminology, 'anchor items' are used primarily for:
- Preventing candidates from skipping questions
- Score equating across different test forms or administrations (Correct answer)
- Increasing item difficulty over time
- Marking the beginning of each content domain
Correct answer: Score equating across different test forms or administrations
Anchor items appear in multiple test forms and allow psychometricians to place scores from different forms on the same scale through equating.
CBT Test Design and Blueprinting 5
Which stakeholder group is most critical to consult during the job task analysis phase of test design?