ATP Quality Control & Assurance 2 — Questions and Answers
Question 1: Which statistical method is used to monitor test item performance over time by tracking mean scores across administrations?
- Item response theory calibration
- Statistical process control (SPC) (Correct answer)
- Differential item functioning analysis
- Classical test theory scaling
Correct answer: Statistical process control (SPC)
Statistical process control uses control charts to monitor whether item and test statistics remain within acceptable bounds across administrations.
Question 2: In test development QA, a 'form blueprint' primarily ensures which quality characteristic?
- Adequate test security
- Content validity and coverage (Correct answer)
- Scoring accuracy
- Appropriate item difficulty
Correct answer: Content validity and coverage
A form blueprint specifies the content domains, cognitive levels, and item counts required, ensuring the test adequately covers the intended construct.
Question 3: A QA review finds that 15% of items in a final form have p-values below 0.20. What is the most appropriate action?
- Accept the form since some hard items are expected
- Flag items for content review and possible replacement (Correct answer)
- Lower the passing standard to compensate
- Re-score the test using IRT instead of CTT
Correct answer: Flag items for content review and possible replacement
Extremely low p-values may indicate ambiguous wording, keying errors, or misalignment with instruction, requiring content review before the form is used.
Question 4: Which process ensures that scores from two different test forms are comparable before results are reported?
- Standard setting
- Equating (Correct answer)
- Scaling
- Norming
Correct answer: Equating
Equating adjusts for differences in difficulty between forms so that a given scaled score represents the same level of proficiency regardless of which form was taken.
Question 5: The ATP Code of Fair Testing Practices requires that tests be reviewed for potential bias against subgroups. Which review method evaluates qualitative fairness concerns?
- Mantel-Haenszel DIF analysis
- Sensitivity review by trained panelists (Correct answer)
- Confirmatory factor analysis
- Item characteristic curve comparison
Correct answer: Sensitivity review by trained panelists
Sensitivity reviews use trained reviewers from diverse backgrounds to flag offensive, stereotyping, or culturally biased content before statistical analyses are run.
Question 6: A QA audit reveals that answer keys were entered incorrectly for two items before scoring. What is the immediate corrective action?
- Report results with a disclaimer
- Void the two items and rescore all tests (Correct answer)
- Apply a correction-for-guessing formula
- Archive the error and correct in the next form
Correct answer: Void the two items and rescore all tests
Keying errors require voiding the affected items and rescoring to ensure examinees are not penalized for problems caused by test developer error.
Question 7: In a quality management system for test publishing, which document type records the chain of custody for secure test materials?
- Psychometric technical report
- Material tracking log (Correct answer)
- Examinee score report
- Operational test blueprint
Correct answer: Material tracking log
A material tracking log records who handled secure materials, when, and where, establishing accountability and supporting breach investigations.
ATP Quality Control & Assurance 2
Which statistical method is used to monitor test item performance over time by tracking mean scores across administrations?