EQAO EQAO Assessment Development and Validity 2 — Questions and Answers
Question 1: What is the role of the 'Item Response Theory' (IRT) in EQAO assessment development?
- It determines how many teachers are needed to score tests
- It provides a statistical framework for analyzing item difficulty and student ability simultaneously (Correct answer)
- It establishes the physical format of the test booklet
- It sets the passing score for each assessment
Correct answer: It provides a statistical framework for analyzing item difficulty and student ability simultaneously
IRT is a psychometric model used by EQAO to analyze how individual test items perform and to calibrate scores across different assessment forms.
Question 2: Which of the following best describes the concept of 'reliability' in an EQAO assessment?
- The test produces consistent results when administered under similar conditions (Correct answer)
- Every student achieves the same score
- The test is always administered on the same date each year
- The test questions are never changed between years
Correct answer: The test produces consistent results when administered under similar conditions
Reliability refers to the consistency of assessment results, meaning similar students would receive similar scores if the test were repeated under comparable conditions.
Question 3: Why does EQAO include 'common items' that remain the same across different years of an assessment?
- To reduce printing costs
- To enable year-to-year comparisons by anchoring the difficulty scale (Correct answer)
- To allow students to prepare specific answers in advance
- To ensure every school writes identical tests
Correct answer: To enable year-to-year comparisons by anchoring the difficulty scale
Common anchor items allow EQAO to statistically link results across years, making it possible to compare performance trends over time.
Question 4: What type of review panel does EQAO convene specifically to evaluate the cultural appropriateness of assessment materials?
- A financial audit panel
- A sensitivity and bias review panel with diverse community representation (Correct answer)
- A parent council advisory committee
- An international curriculum alignment panel
Correct answer: A sensitivity and bias review panel with diverse community representation
EQAO uses sensitivity and bias review panels composed of educators and community members with diverse backgrounds to flag potentially inappropriate content.
Question 5: What does EQAO mean when it refers to 'standard-setting' in assessment development?
- Deciding which schools must participate in assessments
- The process of determining the cut scores that define each achievement level (Correct answer)
- Writing the curriculum standards that tests are based on
- Establishing rules for test security and administration
Correct answer: The process of determining the cut scores that define each achievement level
Standard-setting is the process by which EQAO convenes expert panels to determine the score thresholds that separate achievement levels 1 through 4.
Question 6: How does EQAO handle test booklets that include both field-test items and scored items?
- Students are told which items are field-test items so they can skip them
- Field-test items are embedded among scored items and are indistinguishable to students (Correct answer)
- Field-test items appear in a separate clearly labelled section at the end
- Only teachers see the field-test items after the assessment
Correct answer: Field-test items are embedded among scored items and are indistinguishable to students
EQAO embeds field-test items within the scored portion of the assessment so that students treat all questions equally, producing authentic response data.
EQAO EQAO Assessment Development and Validity 2
What is the role of the 'Item Response Theory' (IRT) in EQAO assessment development?