CTP Evaluation & Measurement of Training Effectiveness 3 — Questions and Answers
Question 1: In the Phillips ROI Methodology, which level comes AFTER Kirkpatrick's four levels?
- Level 5 – ROI (Correct answer)
- Level 5 – Impact
- Level 5 – Transfer
- Level 5 – Alignment
Correct answer: Level 5 – ROI
Phillips added a fifth level—ROI—to Kirkpatrick's model, which calculates the financial return on training investment.
Question 2: A trainer collects data DURING a program to make real-time improvements. This is called:
- Summative evaluation
- Formative evaluation (Correct answer)
- Impact evaluation
- Retrospective evaluation
Correct answer: Formative evaluation
Formative evaluation occurs during program development or delivery to guide improvements before the program is finalized.
Question 3: Which method of isolating training effects relies on asking participants and their managers to estimate what portion of improvement is attributable to training?
- Trend line analysis
- Control group comparison
- Participant estimation (Correct answer)
- Pre/post testing
Correct answer: Participant estimation
Participant and manager estimation asks stakeholders to assign a percentage of performance improvement to the training program.
Question 4: Learner retention of training content is BEST supported by which evaluation approach?
- Conducting a single post-test immediately after training
- Spacing assessments over time using delayed retention tests (Correct answer)
- Averaging pre-test and post-test scores
- Measuring only Level 4 outcomes
Correct answer: Spacing assessments over time using delayed retention tests
Delayed retention tests, administered weeks or months after training, better measure long-term knowledge retention than immediate post-tests.
Question 5: Which statistic is MOST appropriate for determining whether a difference between pre-test and post-test scores is statistically significant?
- Mean score average
- Paired t-test (Correct answer)
- Frequency distribution
- Standard deviation only
Correct answer: Paired t-test
A paired t-test compares means from the same group at two different times, making it ideal for pre/post-test training evaluations.
Question 6: A company's training evaluation shows high Level 2 scores but poor Level 3 transfer. The MOST likely cause is:
- The learning objectives were too difficult
- The post-test was poorly designed
- The work environment does not support applying new skills (Correct answer)
- Learners lacked prerequisite knowledge
Correct answer: The work environment does not support applying new skills
Transfer of learning (Level 3) is often blocked by environmental factors like lack of manager support, opportunity to practice, or reinforcing systems.
Question 7: What is a 'balanced scorecard' approach when applied to training evaluation?
- Using only financial ROI to justify training
- Measuring training effectiveness across multiple perspectives including financial, customer, internal processes, and learning (Correct answer)
- Scoring each quiz question equally across competencies
- Averaging all Kirkpatrick levels into one composite score
Correct answer: Measuring training effectiveness across multiple perspectives including financial, customer, internal processes, and learning
A balanced scorecard approach evaluates training impact across multiple dimensions—financial, customer satisfaction, operational, and learning—providing a holistic view.
CTP Evaluation & Measurement of Training Effectiveness 3
In the Phillips ROI Methodology, which level comes AFTER Kirkpatrick's four levels?