ATP Professional Standards & Competencies 5 — Questions and Answers
Question 1: Under ATP's ethical standards, when a test is no longer psychometrically supported, the publisher's obligation is to:
- Continue selling the test while updating marketing materials
- Retire the test or invest in re-validation (Correct answer)
- Transfer liability to the test users
- Archive the test without notifying users
Correct answer: Retire the test or invest in re-validation
ATP ethics require publishers to retire or re-validate tests that no longer have sufficient technical support rather than continuing to market unsupported instruments.
Question 2: The competency area of 'score reporting design' in ATP standards emphasizes that score reports should:
- Display only percentile ranks
- Present information at the appropriate level of detail for each intended audience (Correct answer)
- Be identical for all stakeholder groups
- Omit confidence intervals to avoid confusion
Correct answer: Present information at the appropriate level of detail for each intended audience
Effective score reports are tailored to their audiences — test takers, educators, and policymakers each need different levels and types of information.
Question 3: Which ATP standard applies when a publisher licenses a test to a third-party administrator?
- The publisher's standards obligations end at the point of licensing
- The publisher must ensure that licensed administrators meet the same test security and quality standards (Correct answer)
- Third-party administrators are solely responsible for all standard compliance
- Licensing agreements supersede professional standards
Correct answer: The publisher must ensure that licensed administrators meet the same test security and quality standards
ATP holds publishers accountable for the proper use of their tests even when administered by licensed third parties; contracts should require adherence to professional standards.
Question 4: In ATP's framework, 'content representativeness' refers to the degree to which:
- Test items use diverse artwork
- The test adequately samples the full domain specified in the test blueprint (Correct answer)
- Items are written at varying difficulty levels
- The test is translated into multiple languages
Correct answer: The test adequately samples the full domain specified in the test blueprint
Content representativeness (content validity) requires that the test systematically covers the breadth and depth of the target domain as defined by the test specifications.
Question 5: ATP professional standards require that a test's reliability be estimated using a method appropriate for the:
- Vendor with the lowest cost
- Test's format, length, and the decisions being made from scores (Correct answer)
- Number of items per page
- Year the test was first developed
Correct answer: Test's format, length, and the decisions being made from scores
Different reliability methods (e.g., test-retest, internal consistency, inter-rater) are appropriate under different conditions, and the choice must be defensibly matched to the test's characteristics and use.
Question 6: A key ATP competency for professionals overseeing large-scale test administration is the ability to:
- Design advertising campaigns
- Develop and execute administration protocols that standardize conditions across testing sites (Correct answer)
- Set printer specifications for test booklets
- Negotiate bulk postage rates
Correct answer: Develop and execute administration protocols that standardize conditions across testing sites
Standardized administration is essential to score comparability; test administration professionals must ensure that all sites follow identical conditions so scores mean the same thing everywhere.
Question 7: According to ATP guidelines, stakeholder feedback collected after a testing program launches should be used to:
- Replace psychometric validation studies
- Inform continuous program improvement and identify potential validity threats (Correct answer)
- Justify price increases
- Modify cut scores without re-validation
Correct answer: Inform continuous program improvement and identify potential validity threats
Post-launch feedback from test takers, users, and score recipients provides ongoing evidence about a program's functioning and can surface fairness or validity concerns requiring remediation.
ATP Professional Standards & Competencies 5
Under ATP's ethical standards, when a test is no longer psychometrically supported, the publisher's obligation is to: