TS Assessment & Evaluation Strategies 2 — Questions and Answers
Question 1: A trainer wants to measure whether learners can apply new skills on the job three months after training. Which evaluation approach is BEST suited for this?
- Pre-test administered before the program begins
- Level 3 behavior evaluation using supervisor observations (Correct answer)
- End-of-course knowledge test
- Reaction survey distributed immediately after class
Correct answer: Level 3 behavior evaluation using supervisor observations
Level 3 (Behavior) in Kirkpatrick's model assesses transfer of learning to the workplace, typically measured weeks or months post-training.
Question 2: Which of the following BEST describes a norm-referenced assessment?
- Scores are compared against a predetermined mastery standard
- Learners are ranked relative to the performance of their peer group (Correct answer)
- Each question is weighted equally regardless of difficulty
- Results are used solely for formative feedback
Correct answer: Learners are ranked relative to the performance of their peer group
Norm-referenced assessments rank individuals relative to a comparison group rather than against a fixed standard.
Question 3: During a role-play exercise, a trainer evaluates a learner's customer service skills in real time. This is an example of what type of assessment?
- Summative assessment
- Performance-based assessment (Correct answer)
- Criterion-referenced test
- Standardized assessment
Correct answer: Performance-based assessment
Performance-based assessment requires learners to demonstrate skills through direct application in realistic scenarios.
Question 4: A post-training ROI analysis reveals that the cost of the training program exceeded the monetary benefits. What should a trainer specialist do FIRST with this finding?
- Immediately cancel all future sessions of the program
- Investigate which program components drove the cost-benefit gap (Correct answer)
- Increase the number of training hours to improve results
- Report the finding only to senior leadership without further analysis
Correct answer: Investigate which program components drove the cost-benefit gap
Before making decisions, trainers should diagnose root causes of poor ROI to determine whether design, delivery, or business context factors are responsible.
Question 5: Which validity type is MOST concerned with whether a test's content accurately represents the full domain of skills being measured?
- Construct validity
- Predictive validity
- Content validity (Correct answer)
- Concurrent validity
Correct answer: Content validity
Content validity ensures that assessment items adequately sample the full range of knowledge or skills in the target domain.
Question 6: A learning evaluation shows high reaction scores but no measurable change in job performance. Which action BEST addresses this discrepancy?
- Redesign the post-training survey to collect more positive responses
- Conduct a Level 3 analysis to identify transfer barriers in the work environment (Correct answer)
- Add more engaging activities to increase learner satisfaction
- Reduce course length to minimize learner fatigue
Correct answer: Conduct a Level 3 analysis to identify transfer barriers in the work environment
A gap between Level 1 (Reaction) and Level 3 (Behavior) often points to work environment factors such as lack of manager support or opportunity to practice.
Question 7: Which statistical measure is MOST useful for determining whether two raters are scoring performance assessments consistently?
- Mean score comparison
- Inter-rater reliability coefficient (Correct answer)
- Standard deviation of test scores
- Item difficulty index
Correct answer: Inter-rater reliability coefficient
Inter-rater reliability measures the degree of agreement between two or more evaluators scoring the same performance.
TS Assessment & Evaluation Strategies 2
A trainer wants to measure whether learners can apply new skills on the job three months after training.
Which evaluation approach is BEST suited for this?