ATP Research & Evidence-Based Practice 4

Question 1

Which reliability estimation method is most appropriate when test items measure multiple distinct dimensions?

Accepted Answer

Alternate forms reliability

Answer

Coefficient alpha assumes unidimensionality and underestimates reliability for multidimensional tests; alternate forms reliability does not carry this assumption.

Question 2

In evidence-based assessment, an incremental validity study demonstrates that a new test:

Accepted Answer

Explains additional variance in a criterion beyond existing predictors

Answer

Incremental validity shows that a test adds predictive power beyond what can be explained by already-available information, justifying its practical use.

Question 3

The 'table of specifications' in test development primarily serves to ensure:

Accepted Answer

Content coverage aligns with the intended domain blueprint

Answer

A table of specifications maps items to content areas and cognitive levels, ensuring the test adequately and proportionally samples the target knowledge domain.

Question 4

Which statistical index is used to evaluate how well a set of items in a CAT fits the IRT model for an individual examinee?

Accepted Answer

Infit and outfit mean square statistics

Answer

Infit and outfit statistics from Rasch/IRT analysis quantify the degree to which individual item responses conform to model expectations for that examinee.

Question 5

A test publisher conducts a predictive validity study with a two-year follow-up. Which threat to internal validity is most relevant?

Accepted Answer

Attrition/mortality

Answer

Over a two-year period, differential attrition—where certain types of participants drop out—can systematically bias the sample and distort the observed validity coefficient.

ATP - Association of Test Publishers Practice Test

ATP - Association of Test Publishers Practice Test

ATP Research & Evidence-Based Practice 4