ATP Case Studies & Practical Application 3

Question 1

A testing company receives a subpoena requesting examinee score records and test booklets for a legal case. What is the first step the company should take?

Accepted Answer

Consult legal counsel to evaluate the scope of the subpoena and determine what can be released under applicable confidentiality agreements and test security requirements

Answer

Legal counsel must review any subpoena to balance compliance obligations with confidentiality agreements, test security, and relevant state and federal privacy laws.

Question 2

During a validity study for a police officer selection test, researchers find the test predicts performance well overall but shows differential prediction for female officers. What is the correct psychometric response?

Accepted Answer

Conduct separate regression analyses by group and examine whether differential slopes or intercepts indicate biased prediction

Answer

Differential prediction analysis using separate regression lines by subgroup is the standard method for detecting and reporting potential test bias in predictive validity studies.

Question 3

A state department of education wants to use a vendor's nationally normed achievement test for statewide accountability. What is the primary validity concern?

Accepted Answer

Whether the test's content aligns with state academic standards and whether national norms are appropriate for state accountability inferences

Answer

Content alignment with state standards is essential for accountability validity, and national norms may not reflect state-specific performance expectations.

Question 4

A test security investigator identifies a cluster of examinees with near-identical incorrect responses in a proctored testing center. What does this pattern most likely indicate?

Accepted Answer

Potential proxy testing or collusion, warranting further forensic analysis including seat proximity and registration data review

Answer

Identical incorrect response patterns (answer copying indices) in a specific testing center are a recognized statistical signal of potential collusion or proxy testing requiring forensic follow-up.

Question 5

A medical licensing board wants to raise its passing standard to address public safety concerns. Which approach best balances rigor with defensibility?

Accepted Answer

Convene a new standard-setting panel using a recognized method (e.g., modified Angoff or Bookmark), document the process, and validate the new standard against competency evidence

Answer

Standard-setting with a recognized method, expert panelists, and documented validation is the professionally and legally defensible way to revise a passing standard.

ATP - Association of Test Publishers Practice Test

ATP - Association of Test Publishers Practice Test

ATP Case Studies & Practical Application 3