ATP Case Studies & Practical Application 2

Question 1

A testing organization discovers mid-administration that a subset of test items had a technical error causing ambiguous wording. What is the most appropriate immediate response?

Accepted Answer

Flag the affected items for post-hoc analysis and consider score adjustments or item removal

Answer

Flagging affected items for post-hoc analysis allows the psychometric team to evaluate impact and apply appropriate scoring adjustments while minimizing disruption.

Question 2

A credentialing body finds that its passing standard was set without formal job task analysis. Which remediation step is most critical to ensure defensibility?

Accepted Answer

Commission a new job task analysis and conduct a standard-setting study to establish a defensible cut score

Answer

A new job task analysis followed by a formal standard-setting study establishes the empirical and content linkage required for defensible credentialing decisions.

Question 3

An employer uses a cognitive ability test that shows adverse impact against a protected group. Under the Uniform Guidelines, which action best addresses this situation?

Accepted Answer

Conduct a validity study to determine whether the test predicts job performance and explore alternative tests with less adverse impact

Answer

The Uniform Guidelines require employers to demonstrate validity for any selection procedure causing adverse impact or to seek less adverse alternatives.

Question 4

A test developer is adapting an English-language licensure exam for Spanish-speaking candidates. Which process is most critical to ensure linguistic equivalence?

Accepted Answer

Perform forward-and-back translation, conduct bilingual review panels, and run pilot studies to verify item performance

Answer

Forward-and-back translation with bilingual expert review and pilot testing is the standard methodology for ensuring linguistic and construct equivalence in test adaptation.

Question 5

A large-scale K-12 assessment program is under scrutiny because schools are 'teaching to the test.' What is the most appropriate psychometric and policy response?

Accepted Answer

Distinguish between coaching on the construct versus coaching on specific items, and periodically refresh item pools to maintain construct validity

Answer

Differentiating construct-relevant preparation from item-specific coaching, combined with item pool renewal, protects validity while acknowledging that legitimate test preparation is acceptable.

ATP - Association of Test Publishers Practice Test

ATP - Association of Test Publishers Practice Test

ATP Case Studies & Practical Application 2