Item Analysis Flashcards

When comparing item analysis results for ELA and math items on the M-STEP, which statement about p-values is accurate?

Answer: Acceptable p-value ranges may differ depending on the purpose and content of the test

Acceptable difficulty ranges vary by test purpose; criterion-referenced assessments may tolerate different p-value ranges than norm-referenced tests.

Which scenario BEST describes an item exhibiting uniform DIF favoring one student group?

Answer: The item is harder for one group at all ability levels compared to another group

Uniform DIF occurs when one group consistently performs better or worse than the other across all ability levels, not just at certain points.

A test developer sees that an item's discrimination index is 0.00. What is the most likely explanation?

Answer: High scorers and low scorers answered the item correctly at the same rate

A discrimination index of 0.00 means there is no difference in the proportion of high scorers versus low scorers who answered correctly.

Why might a constructed-response item on the M-STEP require inter-rater reliability analysis in addition to standard item analysis?

Answer: Scores depend on human judgment, introducing potential scoring inconsistency

Because human raters score constructed-response items, inter-rater reliability measures how consistently different raters apply the scoring rubric.

Which statement accurately describes the relationship between item difficulty and test reliability?

Answer: Items with p-values near 0.50 tend to maximize score variance and support higher reliability

Items near p = 0.50 produce maximum variance in student responses, which increases the test's overall score variance and supports higher reliability coefficients.

An item analysis report shows that 40% of students in the upper group and 38% in the lower group answered Item 12 correctly. What does this suggest?

Answer: Item 12 is a poor discriminator and may need revision

When the upper and lower groups perform nearly identically on an item, the discrimination index is close to zero, indicating the item fails to differentiate ability levels.

How does item response theory (IRT) improve on classical test theory (CTT) when analyzing M-STEP items?

Answer: IRT provides item parameter estimates that are sample-independent

A key advantage of IRT is that item parameters (difficulty, discrimination) are theoretically invariant across different samples of test-takers.