ACT Assessment & Evaluation Methods 3 — Questions and Answers
Question 1: A test security breach occurs when an examinee photographs test items. The administrator's FIRST action should be to:
- Confiscate the device and void the examinee's scores immediately
- Follow the testing organization's documented incident response procedures (Correct answer)
- Allow the examinee to finish and report the incident afterward
- Warn the examinee and continue the session without documentation
Correct answer: Follow the testing organization's documented incident response procedures
Administrators must follow the official incident response protocol rather than taking unilateral action that may not align with policy.
Question 2: Content validity evidence is gathered PRIMARILY by:
- Correlating test scores with a job performance measure
- Having subject matter experts review items against the test blueprint (Correct answer)
- Calculating coefficient alpha from examinee response data
- Comparing scores of known groups expected to differ in the construct
Correct answer: Having subject matter experts review items against the test blueprint
Content validity is established through systematic review by experts to confirm that items adequately represent the targeted content domain.
Question 3: When calculating reliability, a Cronbach's alpha of 0.90 indicates:
- The test has low internal consistency
- The test has high internal consistency among items (Correct answer)
- The test is perfectly free of measurement error
- The test should be revised before use
Correct answer: The test has high internal consistency among items
Alpha values closer to 1.0 reflect stronger internal consistency, meaning items are measuring a cohesive construct.
Question 4: Which scenario best illustrates 'construct-irrelevant variance' threatening test validity?
- A chemistry test that includes questions from multiple subtopics
- A reading comprehension test where poor lighting causes eyestrain for all examinees
- A math test where bilingual examinees are disadvantaged by complex English phrasing (Correct answer)
- A history test that samples events from different time periods
Correct answer: A math test where bilingual examinees are disadvantaged by complex English phrasing
When language complexity on a math test disadvantages non-native English speakers, their scores reflect language ability rather than math skill — a construct-irrelevant factor.
Question 5: A cutoff score for a certification exam is BEST described as:
- The average score of all examinees who took the test
- The minimum score required to demonstrate acceptable competency (Correct answer)
- The score at the 50th percentile of the norming group
- The highest score achievable on the assessment
Correct answer: The minimum score required to demonstrate acceptable competency
A cutoff (passing) score represents the minimum level of performance judged sufficient to demonstrate the required knowledge or skill.
Question 6: During testing, an examinee requests clarification on a test question. The administrator should:
- Explain the meaning of the question using their own words
- Refer the examinee to the approved script response (e.g., 'I cannot help with test questions') (Correct answer)
- Allow the examinee extra time to compensate for the confusion
- Mark the item on a log and continue without responding
Correct answer: Refer the examinee to the approved script response (e.g., 'I cannot help with test questions')
Administrators are trained to respond only with approved scripted language to prevent providing unfair advantages on test content.
Question 7: The test blueprint (table of specifications) is primarily used to ensure that a test has adequate:
- Reliability across administrations
- Content coverage across the domain (Correct answer)
- Correlation with external criteria
- Difficulty alignment with examinee ability levels
Correct answer: Content coverage across the domain
The blueprint maps test items to content areas and objectives, ensuring the test representatively samples the intended domain.
ACT Assessment & Evaluation Methods 3
A test security breach occurs when an examinee photographs test items.
The administrator's FIRST action should be to: