PC Test Development and Validation Flashcards

In test development, discriminant validity is demonstrated by showing that a test:

Answer: Correlates weakly with measures of unrelated constructs

Discriminant validity confirms that a test does not correlate too strongly with measures assessing different constructs, supporting its specificity.

Which type of reliability is assessed by having two different raters score the same responses and comparing their results?

Answer: Inter-rater reliability

Inter-rater reliability measures the degree of agreement between two or more independent scorers evaluating the same responses.

A psychometrician discovers that high-scoring examinees are selecting a particular distractor more often than low-scoring examinees. This indicates:

Answer: The item or distractor may be technically flawed or misleading

When high-ability examinees choose a distractor more than low-ability examinees, it suggests the distractor may be ambiguous, technically defensible, or confusing.

The unidimensionality assumption in psychometric test development means that:

Answer: All items primarily measure a single underlying trait or ability

Unidimensionality assumes that a single dominant factor explains the variance in item responses, which is a key requirement for many psychometric models including IRT.

During test development, a review for item fairness involves checking that items are free from:

Answer: Language, content, or context that disadvantages specific subgroups

Fairness review ensures that items do not contain cultural references, language complexity, or stereotypes that give an unfair advantage or disadvantage to specific groups.

When a certification exam uses computer adaptive testing (CAT), item selection is based on:

Answer: The examinee's estimated ability level from previous responses

In CAT, each item is selected to be optimally informative for the examinee's current estimated ability, making the exam more efficient and precise.