ATP Professional Standards & Competencies 3 — Questions and Answers
Question 1: When a test publisher claims score-based inferences are valid, that claim must be supported by:
- Favorable customer testimonials
- An accumulated body of validity evidence (Correct answer)
- High sales volume
- Government endorsement
Correct answer: An accumulated body of validity evidence
Validity is an evidence-based argument; ATP and professional standards require publishers to accumulate and document evidence that supports intended score interpretations.
Question 2: ATP guidelines on accommodation state that testing accommodations should:
- Alter the construct being measured
- Remove construct-irrelevant barriers without changing what the test measures (Correct answer)
- Be limited to physical disabilities only
- Always extend testing time by 100%
Correct answer: Remove construct-irrelevant barriers without changing what the test measures
Accommodations are designed to level the playing field by eliminating barriers unrelated to the target construct while keeping the measurement construct intact.
Question 3: The principle of 'minimizing construct-irrelevant variance' in testing refers to:
- Keeping test forms short
- Reducing score variation caused by factors unrelated to the ability being measured (Correct answer)
- Using only multiple-choice items
- Restricting testing to high-stakes contexts
Correct answer: Reducing score variation caused by factors unrelated to the ability being measured
Construct-irrelevant variance inflates or deflates scores due to factors like reading difficulty in a math test, undermining the validity of score interpretations.
Question 4: Under ATP's professional standards, a conflict of interest in test development MUST be:
- Ignored if the test developer has domain expertise
- Disclosed and managed through established organizational procedures (Correct answer)
- Resolved by the test developer alone
- Reported directly to the federal government
Correct answer: Disclosed and managed through established organizational procedures
ATP standards require transparent disclosure of conflicts and organizational management of those conflicts to maintain test integrity and public trust.
Question 5: Which standard governs how a test publisher must handle personally identifiable information (PII) collected during test administration?
- Test security standard
- Data privacy and confidentiality standard (Correct answer)
- Equating standard
- Item writing standard
Correct answer: Data privacy and confidentiality standard
ATP's data privacy standards require publishers to collect, store, and share PII in ways that protect test taker privacy and comply with applicable laws.
Question 6: A testing program uses automated essay scoring. According to ATP professional standards, the publisher must:
- Abandon human scoring entirely
- Document validity evidence specific to the automated scoring method and its decision-making context (Correct answer)
- Apply the same standards used for multiple-choice scoring without modification
- Limit automated scoring to low-stakes assessments only
Correct answer: Document validity evidence specific to the automated scoring method and its decision-making context
ATP standards require that evidence supporting validity be gathered specifically for the scoring method employed, including AI or automated scoring engines.
Question 7: The ATP standard requiring that test specifications be documented prior to item development is intended to:
- Increase the cost of test development
- Ensure the test systematically measures the intended content and cognitive domains (Correct answer)
- Comply with copyright law
- Reduce the number of items needed
Correct answer: Ensure the test systematically measures the intended content and cognitive domains
Test specifications (blueprints) define content coverage and cognitive levels in advance, ensuring the final test aligns with its intended measurement purpose.
ATP Professional Standards & Competencies 3
When a test publisher claims score-based inferences are valid, that claim must be supported by: