CPT Evaluation & Measurement of Performance Outcomes 3 — Questions and Answers
Question 1: A company implements a new onboarding process and wants to assess whether new hires are applying learned skills 90 days on the job. This corresponds to which Kirkpatrick level?
- Level 1 – Reaction
- Level 2 – Learning
- Level 3 – Behavior (Correct answer)
- Level 4 – Results
Correct answer: Level 3 – Behavior
Level 3 (Behavior) evaluates whether participants transfer and apply learned skills in the workplace after training.
Question 2: Which type of validity confirms that a performance assessment tool actually measures the construct it claims to measure?
- Face validity
- Construct validity (Correct answer)
- Content validity
- Predictive validity
Correct answer: Construct validity
Construct validity ensures the assessment instrument measures the theoretical construct it is intended to measure.
Question 3: A performance analyst collects data showing that call center error rates dropped 20% after an intervention. However, a new quality management system was also introduced simultaneously. This situation illustrates which evaluation challenge?
- Instrumentation bias
- Confounding variables (Correct answer)
- Selection bias
- Regression to the mean
Correct answer: Confounding variables
Confounding variables (the new quality system) make it difficult to attribute the error rate improvement solely to the intervention.
Question 4: When converting soft data (e.g., improved teamwork) to monetary value, which approach is considered most credible by CPT practitioners?
- Using the lowest available estimate to be conservative
- Linking the soft measure to a hard data item with a known value (Correct answer)
- Ignoring soft data because it cannot be quantified
- Multiplying employee count by an arbitrary dollar figure
Correct answer: Linking the soft measure to a hard data item with a known value
Converting soft data by linking it to a hard measure with a known monetary value provides the most defensible and credible conversion.
Question 5: In performance evaluation, what does 'reliability' of a measurement instrument refer to?
- The extent to which the instrument measures the correct variable
- The consistency of results across repeated administrations (Correct answer)
- The degree to which results can be generalized to other populations
- The cost-effectiveness of administering the instrument
Correct answer: The consistency of results across repeated administrations
Reliability refers to the consistency and repeatability of a measurement tool's results under the same conditions.
Question 6: Which data source would BEST provide evidence of behavior change (Level 3) after a conflict resolution training program?
- Post-training quiz scores
- Manager observation checklists completed 60 days post-training (Correct answer)
- Trainee satisfaction ratings
- Training attendance records
Correct answer: Manager observation checklists completed 60 days post-training
Manager observation checklists collected after training directly capture whether participants are applying conflict resolution behaviors on the job.
Question 7: A performance improvement team reports that productivity increased by 15% following an intervention. Stakeholders ask how much of that gain is attributable to the program. The team should apply which analytical step?
- Trend line analysis or isolation techniques to estimate program contribution (Correct answer)
- Simply credit all 15% to the intervention to maximize perceived ROI
- Remove the productivity data from the final report
- Repeat the intervention before attributing any results
Correct answer: Trend line analysis or isolation techniques to estimate program contribution
Isolation techniques such as trend line analysis, control groups, or expert estimation help attribute specific portions of an improvement to the intervention.
CPT Evaluation & Measurement of Performance Outcomes 3
A company implements a new onboarding process and wants to assess whether new hires are applying learned skills 90 days on the job.
This corresponds to which Kirkpatrick level?