ORELA Assessment & Evaluation Techniques 4 — Questions and Answers
Question 1: A teacher observes that a student scores well on unit tests but struggles to transfer skills to new contexts on performance tasks. This pattern MOST likely suggests the student:
- Has mastered the content at the highest level of Bloom's Taxonomy
- Demonstrates surface-level recall but lacks deep conceptual understanding (Correct answer)
- Is experiencing test anxiety only during performance tasks
- Needs more time to complete multiple-choice items
Correct answer: Demonstrates surface-level recall but lacks deep conceptual understanding
High scores on recall-based tests combined with poor transfer suggest the student has memorized content without developing the flexible understanding needed for application.
Question 2: Which of the following statements about portfolio assessment is MOST accurate?
- Portfolios are primarily useful for comparing students to a national norm group
- Portfolios document growth over time and encourage student reflection on their own learning (Correct answer)
- Portfolios eliminate the need for rubrics because work speaks for itself
- Portfolios are only appropriate for arts-based subjects
Correct answer: Portfolios document growth over time and encourage student reflection on their own learning
Portfolios capture a developmental record of student work and are most powerful when paired with student self-reflection about progress and goals.
Question 3: Coefficient alpha (Cronbach's alpha) is used to estimate which aspect of an assessment's quality?
- Content validity by comparing items to curriculum standards
- Internal consistency reliability across items measuring the same construct (Correct answer)
- Predictive validity against a future external criterion
- Inter-rater reliability between two scorers of written work
Correct answer: Internal consistency reliability across items measuring the same construct
Cronbach's alpha measures the degree to which items on a scale correlate with each other, indicating internal consistency of the assessment.
Question 4: A teacher's learning target states: 'Students will analyze how an author's word choice contributes to tone.' Which assessment task BEST aligns with this target?
- Define 10 vocabulary words from the assigned reading in a matching exercise
- Write a paragraph explaining how specific words in the text create a sad or hopeful tone (Correct answer)
- Retell the plot of the story in sequence using a graphic organizer
- Read the text aloud fluently and accurately within a timed period
Correct answer: Write a paragraph explaining how specific words in the text create a sad or hopeful tone
Having students explain the connection between specific word choices and tone directly assesses the analytic skill stated in the learning target.
Question 5: A state education agency wants to determine whether its new science curriculum is causally responsible for improved test scores. Which research design would provide the STRONGEST evidence?
- A correlational study comparing schools that adopted the curriculum and those that did not
- A randomized controlled trial assigning schools to curriculum or control conditions (Correct answer)
- A case study of one high-performing school using the new curriculum
- A time-series analysis of a single district's scores before and after adoption
Correct answer: A randomized controlled trial assigning schools to curriculum or control conditions
Random assignment to conditions is the gold standard for establishing causal relationships because it controls for confounding variables.
Question 6: When developing a summative unit test, a teacher should FIRST:
- Write the test questions and then identify the standards they address
- Review the unit's learning targets and plan items that align to each target's cognitive level (Correct answer)
- Select a commercially available test that covers similar content
- Determine the percentage of students expected to score above 80%
Correct answer: Review the unit's learning targets and plan items that align to each target's cognitive level
Beginning with learning targets ensures the test has content and construct validity by design, rather than retrofitting standards to pre-written items.
Question 7: Which of the following BEST describes the concept of 'washback' in educational assessment?
- The statistical correction applied when converting raw scores to scaled scores
- The influence that a high-stakes test has on curriculum and instructional decisions (Correct answer)
- The process of returning scored assessments to students with teacher feedback
- The practice of re-administering a test to improve its psychometric properties
Correct answer: The influence that a high-stakes test has on curriculum and instructional decisions
Washback (also called backwash) refers to the way high-stakes assessments shape what teachers teach and what students study, for better or worse.
ORELA Assessment & Evaluation Techniques 4
A teacher observes that a student scores well on unit tests but struggles to transfer skills to new contexts on performance tasks.
This pattern MOST likely suggests the student: