MTS Evaluation 2 — Questions and Answers
Question 1: Which evaluation design is most appropriate when random assignment to control and experimental groups is not possible?
- True experimental design
- Quasi-experimental design (Correct answer)
- Descriptive design
- Correlational design
Correct answer: Quasi-experimental design
Quasi-experimental designs allow comparison without random assignment, making them practical for many training environments.
Question 2: In Kirkpatrick's model, what does Level 3 (Behavior) evaluation specifically measure?
- Learner satisfaction with the course
- Knowledge retention on post-tests
- Transfer of learned skills to the job (Correct answer)
- Organizational results and ROI
Correct answer: Transfer of learned skills to the job
Level 3 measures the degree to which participants apply what they learned when back on the job.
Question 3: A trainer notices that post-test scores improved significantly but on-the-job performance did not change. Which evaluation gap does this represent?
- Level 1 to Level 2 gap
- Level 2 to Level 3 gap (Correct answer)
- Level 3 to Level 4 gap
- Formative to summative gap
Correct answer: Level 2 to Level 3 gap
A gap between knowledge acquisition (Level 2) and behavior transfer (Level 3) indicates learning is not being applied on the job.
Question 4: What is the primary purpose of a control group in training evaluation?
- To provide a second training cohort
- To establish a baseline for comparison (Correct answer)
- To identify high performers
- To validate the test instrument
Correct answer: To establish a baseline for comparison
A control group that receives no training provides a baseline, allowing evaluators to attribute changes to the training intervention.
Question 5: Which statistical measure best describes the spread of evaluation scores around the mean?
- Mode
- Median
- Standard deviation (Correct answer)
- Percentile rank
Correct answer: Standard deviation
Standard deviation quantifies how much individual scores vary from the mean, indicating score dispersion.
Question 6: When evaluating training effectiveness over time, which method specifically tracks the same learners at multiple points?
- Cross-sectional study
- Longitudinal study (Correct answer)
- Case study
- Focus group
Correct answer: Longitudinal study
A longitudinal study follows the same participants across multiple time points to observe changes and sustained learning.
Question 7: An MTS is selecting an evaluation instrument and wants to ensure it measures what it claims to measure. Which quality is being sought?
- Reliability
- Validity (Correct answer)
- Objectivity
- Sensitivity
Correct answer: Validity
Validity is the degree to which an instrument accurately measures the construct or competency it is intended to assess.
MTS Evaluation 2
Which evaluation design is most appropriate when random assignment to control and experimental groups is not possible?