ETS Standardized Test Design 2 — Questions and Answers
Question 1: Which type of item format requires test-takers to supply their own answer rather than select from options?
- Selected-response
- Constructed-response (Correct answer)
- Matching
- True/False
Correct answer: Constructed-response
Constructed-response items require examinees to generate their own answers, unlike selected-response formats where choices are provided.
Question 2: In test design, 'content validity' refers to how well a test:
- Predicts future performance
- Correlates with other established tests
- Represents the full domain being measured (Correct answer)
- Produces consistent scores over time
Correct answer: Represents the full domain being measured
Content validity is the degree to which test items adequately sample the intended knowledge or skill domain.
Question 3: A blueprint used to guide test construction that specifies the number of items per content area and cognitive level is called a:
- Norming table
- Test specification (test blueprint) (Correct answer)
- Scoring rubric
- Item bank
Correct answer: Test specification (test blueprint)
A test specification or blueprint outlines how many items should address each content area and cognitive level to ensure balanced coverage.
Question 4: Differential item functioning (DIF) analysis is primarily used to:
- Estimate test reliability
- Detect items that may be biased against subgroups (Correct answer)
- Set passing scores
- Measure test speededness
Correct answer: Detect items that may be biased against subgroups
DIF analysis identifies items where examinees of equal ability from different demographic groups have systematically different probabilities of answering correctly.
Question 5: Which scaling approach places scores from different test forms on a common scale so they can be compared?
- Content sampling
- Test equating (Correct answer)
- Criterion referencing
- Factor analysis
Correct answer: Test equating
Test equating adjusts scores across different forms so that a given score reflects the same level of ability regardless of which form was administered.
Question 6: The 'ceiling effect' in a test occurs when:
- Most examinees score near zero
- Most examinees score at or near the maximum (Correct answer)
- The test is too long to complete
- Items are arranged from hardest to easiest
Correct answer: Most examinees score at or near the maximum
A ceiling effect happens when a test is too easy, causing high-ability examinees to cluster at the top and limiting score differentiation.
Question 7: In IRT, the parameter 'b' in the three-parameter logistic model represents item:
- Discrimination
- Difficulty (Correct answer)
- Guessing
- Loading
Correct answer: Difficulty
The 'b' parameter in IRT models represents item difficulty, indicating the ability level at which an examinee has a 50% chance of answering correctly (in the 1PL/2PL).
ETS Standardized Test Design 2
Which type of item format requires test-takers to supply their own answer rather than select from options?