M-STEP Item Analysis 2 — Questions and Answers
Question 1: A test item has a point-biserial correlation of -0.15. What does this indicate?
- The item is too easy
- High-scoring students tend to answer it incorrectly (Correct answer)
- The item discriminates well between ability levels
- The item has four equally attractive distractors
Correct answer: High-scoring students tend to answer it incorrectly
A negative point-biserial correlation means higher-ability students are more likely to get the item wrong, signaling a flawed or ambiguous item.
Question 2: Which distractor characteristic is MOST desirable in a well-constructed multiple-choice item?
- Each distractor is chosen by approximately equal numbers of students
- One distractor is chosen by significantly more students than the others
- No distractor is chosen by high-scoring students
- Distractors are chosen more often by low-scoring students than by high-scoring students (Correct answer)
Correct answer: Distractors are chosen more often by low-scoring students than by high-scoring students
Effective distractors attract low-performing students while being rejected by high-performing students, supporting the item's ability to discriminate.
Question 3: On the M-STEP, what is the primary purpose of conducting item analysis after field testing?
- To determine the total number of test items needed
- To identify items that are too long or wordy
- To evaluate whether items function as intended across student subgroups (Correct answer)
- To set the passing score for the operational test
Correct answer: To evaluate whether items function as intended across student subgroups
Item analysis after field testing checks whether each item functions fairly and effectively for all student subgroups before being included on the operational test.
Question 4: An item's facility index (p-value) is 0.95. How should test developers interpret this?
- The item has strong discriminating power
- Almost all students answered the item correctly, so it may be too easy (Correct answer)
- The item shows strong differential item functioning
- The item's distractors are highly effective
Correct answer: Almost all students answered the item correctly, so it may be too easy
A p-value of 0.95 means 95% of students answered correctly, indicating the item is very easy and may add little discriminating information.
Question 5: Which index is used to detect differential item functioning (DIF) during item analysis?
- Cronbach's alpha
- Mantel-Haenszel statistic (Correct answer)
- Standard error of measurement
- Test-retest reliability coefficient
Correct answer: Mantel-Haenszel statistic
The Mantel-Haenszel statistic is a widely used method for detecting DIF by comparing item performance across focal and reference groups at matched ability levels.
Question 6: In item analysis, what does it mean when an item has a high discrimination index but a very low p-value?
- The item is too easy and should be removed
- The item is difficult but effectively separates high and low performers (Correct answer)
- The item has poor construct validity
- The item is appropriate only for advanced test forms
Correct answer: The item is difficult but effectively separates high and low performers
A high discrimination index with a low p-value indicates the item is hard but still useful for distinguishing between students with different ability levels.
Question 7: A teacher reviews item analysis data and notices that Option C on one item was chosen by 0% of students. What should the teacher conclude?
- Option C is the correct answer
- Option C is an effective distractor
- Option C is a non-functional distractor that should be revised (Correct answer)
- Option C indicates the item has high discrimination
Correct answer: Option C is a non-functional distractor that should be revised
A distractor chosen by no students is non-functional and provides no diagnostic information; it should be rewritten to be more plausible.
M-STEP Item Analysis 2
A test item has a point-biserial correlation of -0.15.
What does this indicate?