DRC DRC Assessment Design 2 — Questions and Answers
Question 1: What is the purpose of a 'pilot study' in DRC's assessment development process?
- To train test administrators
- To collect data on item performance before operational use (Correct answer)
- To publish preliminary results to the public
- To review test content for bias only
Correct answer: To collect data on item performance before operational use
A pilot study gathers empirical data on how items perform with students before those items are used in an operational test.
Question 2: In DRC assessments, what does 'item difficulty' (p-value) indicate?
- The complexity of the item's vocabulary
- The proportion of students who answered the item correctly (Correct answer)
- The number of answer choices available
- The time required to answer the item
Correct answer: The proportion of students who answered the item correctly
The p-value represents the proportion of examinees who answered an item correctly, with higher values indicating easier items.
Question 3: Which term describes the process of reviewing DRC items for fairness across demographic groups?
- Norming
- Bias and sensitivity review (Correct answer)
- Standard setting
- Equating
Correct answer: Bias and sensitivity review
Bias and sensitivity review is a process where content experts evaluate items to ensure they do not disadvantage any particular group of students.
Question 4: What is the function of 'anchor items' in DRC's test form design?
- Items placed at the start to build student confidence
- Common items used across forms to support score equating (Correct answer)
- Items that always appear at the end of the test
- Items worth extra credit points
Correct answer: Common items used across forms to support score equating
Anchor items appear on multiple test forms and allow DRC to statistically link scores across different test versions through equating.
Question 5: How does DRC use 'field test items' embedded in operational tests?
- As extra credit opportunities for students
- To collect performance data without affecting student scores (Correct answer)
- To replace broken calculator functions
- As substitute items when the test is too short
Correct answer: To collect performance data without affecting student scores
Embedded field test items are unscored items placed in operational tests to collect student response data for future use.
Question 6: What is 'universal design for assessment' as applied in DRC test development?
- Using identical test forms for all grade levels
- Designing tests to be accessible to the widest possible range of students (Correct answer)
- Creating assessments only for gifted students
- Using a single language for all assessments
Correct answer: Designing tests to be accessible to the widest possible range of students
Universal design for assessment ensures tests are accessible and valid for diverse learners, including those with disabilities, from the start of development.
DRC DRC Assessment Design 2
What is the purpose of a 'pilot study' in DRC's assessment development process?