ATP Case Studies & Practical Application 4

Question 1

A test publisher is developing a new workplace integrity test and wants to validate it. Which validation strategy provides the strongest evidence for use in pre-employment screening?

Accepted Answer

A predictive validity study correlating pre-hire scores with post-hire counterproductive work behavior outcomes

Answer

Predictive validity using actual hire-and-follow-up designs provides the strongest criterion-related validity evidence for employment selection instruments.

Question 2

A high-stakes exam vendor discovers that a prep company has reconstructed approximately 30% of the live item pool through systematic examinee debriefing. What is the most appropriate response?

Accepted Answer

Retire the compromised items immediately, build replacement items, investigate legal remedies under copyright law, and review exam security protocols

Answer

Immediate item retirement, replacement, legal action, and security review are the required steps to restore exam integrity when significant item exposure is confirmed.

Question 3

An examinee with a documented anxiety disorder requests extended time as a testing accommodation. The testing organization must balance accommodation validity with fairness. Which principle guides this decision?

Accepted Answer

Accommodations should remove barriers created by the disability without altering the construct being measured

Answer

The purpose of accommodations is to provide access by removing disability-related barriers while preserving the integrity of the construct the test measures.

Question 4

A test developer notices that item difficulty values from the field test are substantially higher (easier) than expected based on expert difficulty estimates. What is the most likely explanation?

Accepted Answer

The field test sample was unrepresentative of the target examinee population, skewing toward higher-ability individuals

Answer

If field test p-values are systematically higher than expert predictions, the field test sample likely overrepresents higher-ability examinees relative to the target population.

Question 5

A credentialing organization wants to implement continuous testing (year-round administration) with a large rotating item pool. What is the primary psychometric challenge this introduces?

Accepted Answer

Maintaining score comparability across different item sets and administration windows requires robust equating and item parameter stability over time

Answer

Continuous testing with rotating item banks requires ongoing equating to ensure that scores from different forms and windows remain on a common, comparable scale.

ATP - Association of Test Publishers Practice Test

ATP - Association of Test Publishers Practice Test

ATP Case Studies & Practical Application 4