ATP Quality Control & Assurance 4 — Questions and Answers
Question 1: A testing vendor uses a 'four-eyes principle' during form assembly. What quality control purpose does this serve?
- Ensuring item exposure rates are equal
- Requiring a second reviewer to verify decisions made by the first (Correct answer)
- Limiting the number of items assembled per day
- Automating the scoring key generation process
Correct answer: Requiring a second reviewer to verify decisions made by the first
The four-eyes principle mandates independent verification by a second person, reducing the risk of undetected errors in form assembly, key entry, or blueprint compliance.
Question 2: Which threat to test quality occurs when examinees' scores are influenced by their familiarity with test-taking strategies rather than the target construct?
- Response bias
- Construct-irrelevant variance (Correct answer)
- Sampling error
- Differential test functioning
Correct answer: Construct-irrelevant variance
Construct-irrelevant variance is introduced when scores reflect factors unrelated to the intended construct, such as test-wiseness, reducing validity of score interpretations.
Question 3: A post-administration QA review reveals a significant drop in mean scores compared to prior administrations. Which should be investigated first?
- Changes in examinee demographics
- Possible form assembly or administration errors (Correct answer)
- Publisher printing quality issues
- Examinee motivation levels
Correct answer: Possible form assembly or administration errors
Sudden score drops most immediately suggest a quality failure such as an incorrect answer key, form misrouting, or administration irregularity that must be ruled out before other explanations.
Question 4: Under the ATP's Standards for Educational and Psychological Testing framework, which term describes the degree to which evidence supports a specific interpretation of test scores?
- Reliability
- Validity (Correct answer)
- Fairness
- Precision
Correct answer: Validity
Validity refers to the degree to which evidence and theory support the interpretations of test scores for proposed uses, making it the central quality concept in the Standards.
Question 5: Which item statistic indicates the degree to which an item discriminates between high- and low-performing examinees?
- P-value (difficulty index)
- Point-biserial correlation (Correct answer)
- Standard error of measurement
- Item information function value
Correct answer: Point-biserial correlation
The point-biserial correlation measures how strongly performance on a single item correlates with total test score, reflecting the item's discriminating power.
Question 6: In quality assurance for online proctored exams, which control addresses the risk of proxy test-taking?
- Randomized item ordering
- Identity verification with photo ID and biometrics (Correct answer)
- Longer testing windows
- Adaptive item selection
Correct answer: Identity verification with photo ID and biometrics
Identity verification using government-issued photo ID combined with biometric or facial recognition checks confirms that the registered candidate is the person actually taking the exam.
Question 7: A QA process flags an item because its correct answer can be determined by test-taking cues from within the test. This flaw is called:
- Clang association
- Testwise cue or inter-item cueing (Correct answer)
- Response set bias
- Fatigue effect
Correct answer: Testwise cue or inter-item cueing
Inter-item cueing occurs when information in one item inadvertently provides the answer to another, allowing examinees to answer correctly without content knowledge.
ATP Quality Control & Assurance 4
A testing vendor uses a 'four-eyes principle' during form assembly.
What quality control purpose does this serve?