CTP Evaluation & Measurement of Training Effectiveness 4 — Questions and Answers
Question 1: A training evaluation plan should ideally be created:
- After the training program is delivered
- Before the training program is designed (Correct answer)
- Only when stakeholders request data
- During the final week of training delivery
Correct answer: Before the training program is designed
Evaluation planning should occur during the analysis and design phases so baseline data and measurement strategies are in place before delivery.
Question 2: Which of the following is an example of a tangible benefit that would be included in an ROI calculation?
- Increased employee morale
- Improved supervisor satisfaction ratings
- Reduced scrap and rework costs (Correct answer)
- Greater learner confidence
Correct answer: Reduced scrap and rework costs
Tangible benefits such as reduced scrap costs can be directly converted to monetary value for ROI calculations; morale and confidence are intangible.
Question 3: When designing a Level 3 (Behavior) evaluation, what data source provides the MOST objective evidence of on-the-job transfer?
- Learner self-report surveys
- Manager interviews
- Direct behavioral observation by a trained observer (Correct answer)
- Peer feedback forms
Correct answer: Direct behavioral observation by a trained observer
Direct observation by a trained evaluator provides the most objective and unbiased evidence of behavioral change on the job.
Question 4: Which concept refers to the degree to which an evaluation instrument measures what it is intended to measure?
- Reliability
- Validity (Correct answer)
- Feasibility
- Consistency
Correct answer: Validity
Validity refers to whether an assessment tool actually measures the intended construct, knowledge, or skill.
Question 5: A training professional wants to evaluate whether learners can apply a skill in a realistic scenario. The BEST assessment format is:
- Multiple-choice knowledge test
- True/false quiz
- Performance-based simulation or role-play (Correct answer)
- Fill-in-the-blank recall test
Correct answer: Performance-based simulation or role-play
Performance-based simulations assess application of skills in realistic conditions, aligning with higher-order learning objectives.
Question 6: Which factor MOST threatens the internal validity of a training evaluation?
- Using a pre/post test design
- History effects, where external events influence results (Correct answer)
- Collecting data from all participants
- Using validated assessment instruments
Correct answer: History effects, where external events influence results
History effects occur when external events (e.g., a policy change) coincide with training, making it difficult to attribute results solely to training.
Question 7: In training evaluation, 'benchmarking' refers to:
- Testing learners before the program begins
- Comparing training outcomes to industry standards or best-in-class organizations (Correct answer)
- Setting minimum passing scores for certification
- Timing how long each training module takes
Correct answer: Comparing training outcomes to industry standards or best-in-class organizations
Benchmarking compares an organization's training results or processes to external standards or top-performing peers to identify improvement opportunities.
CTP Evaluation & Measurement of Training Effectiveness 4
A training evaluation plan should ideally be created: