CASAS Test Design & Development 3 — Questions and Answers
Question 1: CASAS assessments are best classified as which type of test?
- Norm-referenced tests that rank test-takers against peers
- Criterion-referenced tests that measure performance against defined standards (Correct answer)
- Aptitude tests that predict future learning potential
- Diagnostic tests that identify specific learning disabilities
Correct answer: Criterion-referenced tests that measure performance against defined standards
CASAS assessments are criterion-referenced, meaning scores indicate mastery of specific competencies rather than rank among peers.
Question 2: Which type of validity evidence is most directly supported by having subject matter experts review CASAS items against the competency framework?
- Predictive validity
- Construct validity
- Content validity (Correct answer)
- Concurrent validity
Correct answer: Content validity
Content validity is established when experts confirm that test items adequately represent the full range of skills and competencies the test is designed to measure.
Question 3: The standard error of measurement (SEM) is used in CASAS score interpretation to:
- Convert raw scores to scale scores
- Estimate the range in which a test-taker's true score likely falls (Correct answer)
- Compare the difficulty of two test forms
- Calculate the average score for an instructional group
Correct answer: Estimate the range in which a test-taker's true score likely falls
SEM quantifies measurement error, allowing educators to construct a confidence interval around an observed score to approximate the test-taker's true ability.
Question 4: How does CASAS ensure score comparability across its multiple equivalent test forms?
- By using identical items on every form
- Through an item bank with IRT-based scaled scores (Correct answer)
- By averaging scores from two consecutive test administrations
- By requiring all programs to use a single standardized form
Correct answer: Through an item bank with IRT-based scaled scores
CASAS maintains a calibrated item bank using IRT, so scaled scores are comparable regardless of which form a test-taker completes.
Question 5: Test-retest reliability refers to:
- The correlation between scores on two different versions of the same test
- The consistency of scores when the same group takes the same test at two points in time (Correct answer)
- The agreement between two independent scorers on constructed-response items
- The internal consistency of items within a single test form
Correct answer: The consistency of scores when the same group takes the same test at two points in time
Test-retest reliability measures score stability over time by correlating results from two administrations of the same instrument to the same group.
Question 6: In CASAS test development, what does 'construct validity' specifically require evidence of?
- That items were written by certified educators
- That the test accurately measures the underlying skill or ability it is intended to assess (Correct answer)
- That the test predicts success in a specific job or training program
- That scores are consistent across multiple test administrations
Correct answer: That the test accurately measures the underlying skill or ability it is intended to assess
Construct validity requires demonstrating that test scores reflect the theoretical construct (e.g., reading comprehension) being measured, not irrelevant factors.
Question 7: Which principle of test fairness is central to the CASAS development process?
- All test-takers must receive the same time limit without exception
- Test items must not systematically disadvantage any identifiable subgroup of test-takers (Correct answer)
- All reading passages must be drawn from U.S. government publications
- Only multiple-choice items may be used to eliminate rater bias
Correct answer: Test items must not systematically disadvantage any identifiable subgroup of test-takers
Fairness in test development requires that items be free of differential item functioning (DIF) so no subgroup is systematically penalized by content irrelevant to the skill measured.
CASAS Test Design & Development 3
CASAS assessments are best classified as which type of test?