CT Assessment Methods 3 — Questions and Answers
Question 1: Which Kirkpatrick model level specifically addresses learner satisfaction with the training?
- Level 1 – Reaction (Correct answer)
- Level 2 – Learning
- Level 3 – Behavior
- Level 4 – Results
Correct answer: Level 1 – Reaction
Level 1 (Reaction) measures how participants feel about the training experience.
Question 2: A trainer notices that two different facilitators score the same role-play very differently. This indicates a problem with:
- Content validity
- Inter-rater reliability (Correct answer)
- Test-retest reliability
- Internal consistency
Correct answer: Inter-rater reliability
Inter-rater reliability measures the degree of agreement between different evaluators scoring the same performance.
Question 3: Which of the following is a criterion-referenced assessment?
- Scoring trainees on a bell curve
- Requiring 90% accuracy on a safety checklist to pass (Correct answer)
- Ranking learners from highest to lowest score
- Comparing a group's average to national norms
Correct answer: Requiring 90% accuracy on a safety checklist to pass
Criterion-referenced assessments measure performance against a predetermined standard, not against other learners.
Question 4: Which question format is BEST for assessing higher-order thinking such as analysis and synthesis?
- True/false
- Multiple-choice recall
- Case-study essay (Correct answer)
- Fill-in-the-blank
Correct answer: Case-study essay
Case-study essays require learners to analyze complex situations and synthesize knowledge into coherent responses.
Question 5: A post-training survey shows high satisfaction scores but no improvement in job performance. What does this illustrate?
- Level 1 and Level 3 are always aligned
- Reaction data alone does not predict behavior change (Correct answer)
- The training should be discontinued
- Satisfaction surveys measure skill transfer
Correct answer: Reaction data alone does not predict behavior change
Positive learner reactions (Level 1) do not guarantee on-the-job behavior change (Level 3).
Question 6: What is the advantage of using multiple assessment methods in a single training program?
- It reduces the time needed for evaluation
- It provides a more complete and valid picture of learner performance (Correct answer)
- It eliminates the need for facilitator observation
- It lowers inter-rater reliability concerns
Correct answer: It provides a more complete and valid picture of learner performance
Triangulating data from multiple sources increases the validity and completeness of performance conclusions.
Question 7: When designing a skills observation checklist, the MOST important first step is to:
- Decide on a point scale
- Identify the specific observable behaviors that indicate competence (Correct answer)
- Select the number of raters
- Determine the passing score
Correct answer: Identify the specific observable behaviors that indicate competence
Observable, behavioral anchors must be defined first so the checklist measures concrete, measurable actions.
CT Assessment Methods 3
Which Kirkpatrick model level specifically addresses learner satisfaction with the training?