DRC DRC Psychometrics and Research 1 — Questions and Answers
Question 1: What is 'Item Response Theory' (IRT) as used by DRC in test development?
- A theory about how students respond emotionally to test-taking
- A statistical framework that models the relationship between student ability and item responses (Correct answer)
- A method for reviewing items based on teacher responses
- A theory about improving student response rates on surveys
Correct answer: A statistical framework that models the relationship between student ability and item responses
IRT is a psychometric framework that models the probability of a correct response as a function of both student ability and item characteristics such as difficulty and discrimination.
Question 2: What does 'test reliability' mean in the context of DRC's psychometric quality standards?
- Whether the test was reliably delivered on time to all schools
- The consistency of test scores if the same students took equivalent versions of the test (Correct answer)
- Whether the test reliably predicts college success
- The reliability of the test printing and shipping process
Correct answer: The consistency of test scores if the same students took equivalent versions of the test
Reliability refers to the consistency and stability of scores—a reliable test produces similar results for similar students under similar conditions, minimizing random measurement error.
Question 3: What is a 'standard error of measurement' (SEM) in DRC psychometrics?
- The number of errors made by standard scorers
- An estimate of the variability in a student's score due to measurement error (Correct answer)
- The standard procedure for measuring student desks
- The margin of error in DRC's delivery schedule
Correct answer: An estimate of the variability in a student's score due to measurement error
The standard error of measurement quantifies how much a student's observed score is expected to vary from their true score due to imperfect measurement reliability.
Question 4: What is 'differential item functioning' (DIF) in DRC's psychometric analysis?
- Items that function differently depending on which computer model is used
- Statistical detection of items that perform differently for matched groups of students (e.g., by gender or ethnicity) (Correct answer)
- Items that have different point values for different grade levels
- The difference in how items function in online versus paper formats
Correct answer: Statistical detection of items that perform differently for matched groups of students (e.g., by gender or ethnicity)
DIF analysis identifies items where students from different groups who have the same overall ability level show different probabilities of answering correctly, signaling potential bias.
Question 5: What is a 'test characteristic curve' (TCC) in DRC's IRT framework?
- A curve showing how test scores change over time for a single student
- A graph showing the expected total score for the test as a function of student ability (Correct answer)
- A chart tracking the number of tests shipped to districts
- A curve displaying the distribution of test difficulty across items
Correct answer: A graph showing the expected total score for the test as a function of student ability
The test characteristic curve plots the expected total test score against the underlying ability continuum, showing how well the test discriminates across the ability range.
Question 6: What is 'classical test theory' (CTT), which DRC may also use alongside IRT?
- A historical theory about how tests were developed in ancient civilizations
- A psychometric framework based on observed scores, true scores, and error scores (Correct answer)
- A theory that classical literature comprehension is the best predictor of academic success
- A classification theory for organizing test items by topic
Correct answer: A psychometric framework based on observed scores, true scores, and error scores
Classical test theory provides a framework in which an observed score is the sum of a student's true score and random error, supporting analyses of reliability and item statistics.
DRC DRC Psychometrics and Research 1
What is 'Item Response Theory' (IRT) as used by DRC in test development?