Item Analysis Flashcards

During M-STEP item review, an item is flagged for C-DIF (non-uniform DIF). What does this mean?

Answer: The group performance gap changes direction or magnitude depending on ability level

Non-uniform DIF means the interaction between group membership and ability level is not constant — the relative advantage switches or changes across the ability scale.

A set of M-STEP items has an average point-biserial correlation of 0.45. What does this suggest about the item set?

Answer: The items generally discriminate well between higher and lower achievers

Point-biserial correlations averaging around 0.45 indicate items that effectively differentiate students with higher and lower overall test performance.

Which of the following is a limitation of using only classical test theory statistics for M-STEP item analysis?

Answer: Item difficulty and discrimination statistics depend on the specific sample of students tested

In CTT, item statistics such as p-values and discrimination indices are sample-dependent, meaning they may differ across different groups of test-takers.

What is the purpose of flagging items for sensitivity review as part of item analysis?

Answer: To ensure items do not contain content that could offend or disadvantage specific demographic groups

Sensitivity review identifies content that might be culturally biased, offensive, or unfair to specific groups of students, independent of statistical bias measures.

An educator analyzing M-STEP data notices that Item 7 has a p-value of 0.30 and a discrimination index of 0.40. Which conclusion is BEST supported?

Answer: Item 7 is hard but effectively distinguishes high from low performers

A discrimination index of 0.40 is quite strong; even with a low p-value, this item meaningfully separates students by achievement level.

What does the term 'item parameter drift' mean in the context of longitudinal M-STEP item analysis?

Answer: An item's statistical parameters shift significantly from one testing year to the next

Item parameter drift refers to significant changes in an item's difficulty or discrimination parameters across administrations, which can threaten score comparability.

Which action should a test developer take when an item shows large DIF favoring one subgroup but is also highly discriminating and content-valid?

Answer: Conduct a content review to determine whether the DIF reflects real ability differences or construct-irrelevant bias

Large DIF does not automatically indicate bias; a content review helps determine whether the group difference reflects genuine differences in the measured construct or unfair item characteristics.