ATP Professional Standards & Competencies 4 — Questions and Answers
Question 1: According to ATP guidelines, cut scores used for high-stakes credentialing decisions must be established through:
- Publisher preference
- A defensible standard-setting process with documented rationale (Correct answer)
- Averaging raw score distributions
- Regulatory mandate alone
Correct answer: A defensible standard-setting process with documented rationale
ATP and measurement standards require that passing scores be set using a rigorous, transparent standard-setting method with clear documentation of the process and panelist qualifications.
Question 2: Which ATP competency area addresses the obligation to monitor ongoing test use for unintended consequences?
- Item banking
- Consequential validity monitoring (Correct answer)
- Test marketing
- Printing quality control
Correct answer: Consequential validity monitoring
Consequential validity requires publishers to track whether actual uses and outcomes of scores align with intended interpretations and avoid harmful unintended effects.
Question 3: A test publisher is asked by a client to suppress score data unfavorable to the client's program. Under ATP professional standards, the publisher should:
- Comply to maintain the business relationship
- Refuse, as suppressing validity-relevant data violates professional standards (Correct answer)
- Conduct a new study with a different sample
- Remove the unfavorable items from the test
Correct answer: Refuse, as suppressing validity-relevant data violates professional standards
ATP standards prohibit withholding or distorting validity evidence, as doing so undermines the integrity of score interpretations and misleads stakeholders.
Question 4: In the context of ATP standards, test form 'equating' differs from 'linking' in that equating:
- Is only applied to performance-based tests
- Produces scores that are interchangeable across forms measuring the same construct with equal precision (Correct answer)
- Does not require a common-item or common-examinee design
- Adjusts scores for test-taker age
Correct answer: Produces scores that are interchangeable across forms measuring the same construct with equal precision
Equating yields fully interchangeable scores across forms of equal difficulty and reliability measuring the same construct, unlike linking which allows less strict relationships.
Question 5: Which of the following is an ATP-recognized method for gathering validity evidence during test development?
- Increasing item count without piloting
- Conducting cognitive interviews with target examinees to verify item interpretation (Correct answer)
- Consulting only subject-matter experts from within the publishing organization
- Relying solely on post-hoc score distributions
Correct answer: Conducting cognitive interviews with target examinees to verify item interpretation
Cognitive interviews with representative examinees reveal whether items are interpreted as intended, providing response-process validity evidence aligned with ATP standards.
Question 6: ATP guidelines require that a test's technical manual be:
- Kept confidential from all external parties
- Made available to qualified users who need it to evaluate test quality (Correct answer)
- Published only after five years of operational use
- Written exclusively by legal staff
Correct answer: Made available to qualified users who need it to evaluate test quality
Transparency is a core ATP principle; technical manuals must be accessible to qualified stakeholders so they can independently evaluate the psychometric quality of the test.
Question 7: When test items are reused across multiple administrations, ATP standards require that publishers:
- Use items indefinitely without review
- Implement item exposure controls and periodic item refresh strategies (Correct answer)
- Publish all item statistics publicly
- Stop administering the test after 12 months
Correct answer: Implement item exposure controls and periodic item refresh strategies
Controlling item exposure limits security risk, while periodic item refresh maintains test integrity and prevents score inflation from item memorization.
ATP Professional Standards & Competencies 4
According to ATP guidelines, cut scores used for high-stakes credentialing decisions must be established through: