ETS ETS Research and Innovation 2 — Questions and Answers
Question 1: What is the purpose of ETS's 'Cognitive Labs' research methodology?
- Measuring brain activity during tests
- Conducting think-aloud protocols to understand test-taker reasoning (Correct answer)
- Developing AI-powered tutoring software
- Analyzing test center environmental factors
Correct answer: Conducting think-aloud protocols to understand test-taker reasoning
Cognitive Labs use think-aloud and interview protocols to understand how test takers interpret and respond to items, informing better question design.
Question 2: Which ETS framework describes a comprehensive approach to designing assessments grounded in cognitive theory and evidence?
- Evidence-Centered Design (ECD) (Correct answer)
- Bloom's Assessment Taxonomy
- Universal Design for Learning (UDL)
- Competency-Based Evaluation (CBE)
Correct answer: Evidence-Centered Design (ECD)
Evidence-Centered Design (ECD) is a principled framework developed at ETS that aligns assessment tasks, evidence, and inferences to the construct being measured.
Question 3: ETS researchers contributed significantly to the development of which measurement theory that models each test taker's ability on a continuous scale?
- Classical Test Theory (CTT)
- Item Response Theory (IRT) (Correct answer)
- Generalizability Theory (G-theory)
- Signal Detection Theory
Correct answer: Item Response Theory (IRT)
Item Response Theory (IRT) is central to ETS test development, allowing ability estimation independent of the particular items administered.
Question 4: What is the focus of ETS's Generalizability Theory (G-theory) research?
- Estimating the generalizability of test scores across different conditions and raters (Correct answer)
- Generalizing test norms to new countries
- Applying scores from one test to predict performance on another
- Making tests available in multiple languages
Correct answer: Estimating the generalizability of test scores across different conditions and raters
G-theory, advanced significantly through ETS research, quantifies how much score variance is attributable to different sources like raters, occasions, and tasks.
Question 5: Which ETS research program focuses specifically on improving assessment for English language learners (ELLs) in K–12 settings?
- TOEFL Academic Program
- Understanding Language Initiative (Correct answer)
- ELL Assessment Equity Project
- Language Assessment Research Collaborative
Correct answer: Understanding Language Initiative
The Understanding Language Initiative, supported by ETS research, focuses on linguistic demands of academic content and assessments for ELL students.
Question 6: ETS's Praxis Series is designed primarily to assess the knowledge and skills of which population?
- Graduate school applicants
- Prospective K–12 teachers (Correct answer)
- Corporate training program participants
- Military personnel seeking promotion
Correct answer: Prospective K–12 teachers
The Praxis Series measures the academic skills and subject-specific content knowledge of candidates entering the teaching profession.
ETS ETS Research and Innovation 2
What is the purpose of ETS's 'Cognitive Labs' research methodology?