MEC Research Methods & Evidence-Based Practice 3 — Questions and Answers
Question 1: An educator reviews three peer-reviewed studies on adaptive quizzing; two show positive effects and one shows no effect. What practice best integrates these findings?
- Conduct or consult a meta-analysis weighing all three studies (Correct answer)
- Accept the majority finding and implement adaptive quizzing
- Discard the null-result study as flawed
- Wait for a single definitive study before acting
Correct answer: Conduct or consult a meta-analysis weighing all three studies
Meta-analysis synthesizes quantitative findings across multiple studies, providing the most rigorous integration of mixed evidence.
Question 2: Which type of validity is threatened when a Moodle quiz contains questions that are too easy for the target population, causing most students to score near the maximum?
- Ceiling effect reducing score variance (Correct answer)
- Content validity
- Construct validity
- Face validity
Correct answer: Ceiling effect reducing score variance
A ceiling effect compresses scores at the top, reducing variance and limiting the quiz's ability to discriminate among high-ability students.
Question 3: A Moodle course designer wants to ensure quiz items measure the intended learning outcomes and nothing else. Which validity type is the primary concern?
- Construct validity (Correct answer)
- Predictive validity
- Concurrent validity
- Ecological validity
Correct answer: Construct validity
Construct validity refers to how well a test measures the specific theoretical construct or learning outcome it is intended to measure.
Question 4: Which of the following best describes a systematic review in the context of online education research?
- A rigorous, pre-specified search and synthesis of all relevant studies on a topic (Correct answer)
- An expert's narrative summary of selected influential papers
- A single large-scale experimental study
- A longitudinal observation of one institution's practices
Correct answer: A rigorous, pre-specified search and synthesis of all relevant studies on a topic
A systematic review uses explicit, reproducible methods to locate, appraise, and synthesize all evidence on a specific question.
Question 5: In Moodle quiz analytics, a high item discrimination index (close to 1.0) indicates what?
- High-scorers on the overall quiz are more likely to answer that item correctly (Correct answer)
- The item is very difficult for all students
- The item measures multiple constructs simultaneously
- The item has low content validity
Correct answer: High-scorers on the overall quiz are more likely to answer that item correctly
A high discrimination index means the item effectively differentiates between students who perform well overall and those who do not.
Question 6: An educator notices that students who complete Moodle H5P interactive lessons score higher on exams. To establish causation rather than correlation, what is the next needed step?
- Random assignment of students to H5P and non-H5P conditions (Correct answer)
- Increasing the sample size of the observational study
- Conducting additional correlational analyses
- Collecting qualitative interviews from top scorers
Correct answer: Random assignment of students to H5P and non-H5P conditions
Random assignment is required to eliminate confounds and establish a causal relationship between the intervention and the outcome.
Question 7: Which Moodle feature provides item-level statistics including facility value and discrimination index after students complete a quiz?
- Quiz statistics report (Correct answer)
- Gradebook report
- Activity completion report
- Logs report
Correct answer: Quiz statistics report
Moodle's quiz statistics report calculates psychometric data such as facility value, standard deviation, and discrimination index for each item.
MEC Research Methods & Evidence-Based Practice 3
An educator reviews three peer-reviewed studies on adaptive quizzing; two show positive effects and one shows no effect.
What practice best integrates these findings?