CASAS Test Design & Development 2 — Questions and Answers
Question 1: What does the item difficulty index (p-value) represent in CASAS test development?
- The correlation between an item and total test score
- The proportion of test-takers who answered the item correctly (Correct answer)
- The number of distractors in a multiple-choice item
- The average time required to complete the item
Correct answer: The proportion of test-takers who answered the item correctly
The p-value (item difficulty) is the proportion of test-takers who answered correctly, where higher values indicate easier items.
Question 2: An item discrimination index of 0.40 or higher generally indicates:
- The item is too easy for the target population
- Poor item quality that requires revision
- A well-functioning item that differentiates ability levels (Correct answer)
- Potential bias that must be reviewed
Correct answer: A well-functioning item that differentiates ability levels
An item discrimination index of 0.40 or above is considered acceptable and shows the item distinguishes between higher- and lower-performing test-takers.
Question 3: In classical test theory, a negative item discrimination index indicates:
- The item is very difficult for advanced learners
- Lower-performing students answered correctly more often than higher-performing students (Correct answer)
- The item has insufficient answer choices
- The item difficulty is at the optimal level
Correct answer: Lower-performing students answered correctly more often than higher-performing students
A negative discrimination index means lower scorers outperformed higher scorers on that item, signaling a flawed or misleading question.
Question 4: What is the purpose of a distractor analysis in CASAS item development?
- To determine the total number of questions needed on a test form
- To evaluate whether incorrect answer options attract test-takers as intended (Correct answer)
- To measure total administration time per item
- To compare raw scores across different test versions
Correct answer: To evaluate whether incorrect answer options attract test-takers as intended
Distractor analysis checks that wrong answer choices are plausible enough to attract lower-ability test-takers, confirming each option is functioning correctly.
Question 5: What is the primary purpose of piloting test items before placing them in the operational CASAS test bank?
- To reduce overall test administration costs
- To collect statistical data on item performance with real test-takers (Correct answer)
- To certify test administrators on new content
- To establish final cut scores for each level
Correct answer: To collect statistical data on item performance with real test-takers
Piloting exposes items to a representative sample so developers can analyze difficulty, discrimination, and distractor effectiveness before operational use.
Question 6: In Item Response Theory (IRT), the 'b' parameter refers to:
- Item discrimination
- Item difficulty (Correct answer)
- The probability of guessing correctly
- Test length adjustment factor
Correct answer: Item difficulty
In IRT, the 'b' parameter is the item difficulty parameter, representing the point on the ability scale where a test-taker has a 50% chance of answering correctly.
Question 7: Which step in CASAS test development ensures that items from different test forms measure the same constructs at equivalent difficulty levels?
- Field testing
- Equating and scaling (Correct answer)
- Bias review
- Content mapping
Correct answer: Equating and scaling
Equating and scaling procedures link scores across different test forms so that equivalent scale scores reflect equivalent skill levels regardless of which form was taken.
CASAS Test Design & Development 2
What does the item difficulty index (p-value) represent in CASAS test development?