MYLS Assessment and Evaluation Methods 3 — Questions and Answers
Question 1: A teacher grades 30 student essays and then re-grades them two weeks later without looking at the original scores. She finds her grades differ significantly. What assessment quality is lacking?
- Validity
- Intra-rater reliability (Correct answer)
- Inter-rater reliability
- Content coverage
Correct answer: Intra-rater reliability
Intra-rater reliability refers to a single evaluator's consistency when scoring the same work at different times.
Question 2: In a numeracy assessment, a student correctly answers all easy and medium items but gets every hard item wrong. Which of these concepts best describes this outcome?
- Regression to the mean
- A floor effect
- A ceiling effect (Correct answer)
- Response bias
Correct answer: A ceiling effect
When a test lacks items difficult enough to challenge high performers, it creates a ceiling effect where top scores cluster at the maximum.
Question 3: A MYLS-style selection test uses multiple-choice questions with four options rather than two. What is the PRIMARY reason for using more answer choices?
- To increase the chance that students guess correctly
- To reduce the probability of getting answers correct by random guessing (Correct answer)
- To make the test longer and more comprehensive
- To align with Australian curriculum standards
Correct answer: To reduce the probability of getting answers correct by random guessing
More options reduce the probability of a correct guess by chance, improving the test's ability to distinguish actual knowledge.
Question 4: A school reports that its students' average score on a literacy test is two standard deviations above the national mean. Approximately what percentage of the national population scored below these students?
- 68%
- 84%
- 95%
- 97.5% (Correct answer)
Correct answer: 97.5%
About 97.5% of scores fall below two standard deviations above the mean in a normal distribution.
Question 5: Which assessment method would be MOST appropriate for evaluating a student's ability to apply mathematical reasoning to real-world situations?
- A timed multiplication drill
- A performance task requiring students to plan a budget (Correct answer)
- A true/false quiz on arithmetic facts
- A multiple-choice vocabulary test
Correct answer: A performance task requiring students to plan a budget
Performance tasks require students to apply skills in realistic contexts, making them ideal for assessing applied mathematical reasoning.
Question 6: A selective entry test uses scaled scores rather than raw scores. What is the main advantage of scaled scores?
- They are easier for parents to interpret without explanation
- They allow fair comparison across different test versions of varying difficulty (Correct answer)
- They eliminate the need for a norm group
- They always range from 0 to 100
Correct answer: They allow fair comparison across different test versions of varying difficulty
Scaling adjusts for differences in difficulty between test forms, ensuring scores are comparable regardless of which version a student took.
Question 7: A teacher observes that students from one demographic group consistently score lower on a reading comprehension item involving sailing terminology. This is MOST likely an example of:
- Low test reliability
- Construct underrepresentation
- Item bias or differential item functioning (Correct answer)
- A floor effect
Correct answer: Item bias or differential item functioning
When an item disadvantages a subgroup due to cultural or background factors unrelated to the skill being tested, it exhibits differential item functioning (DIF).
MYLS Assessment and Evaluation Methods 3
A teacher grades 30 student essays and then re-grades them two weeks later without looking at the original scores.
She finds her grades differ significantly.
What assessment quality is lacking?