CIT Assessment & Evaluation Techniques 4 — Questions and Answers
Question 1: Which data collection method is MOST appropriate for capturing nuanced learner perceptions about a training program's relevance?
- Multiple-choice post-test
- Likert-scale reaction survey
- Focus group interview (Correct answer)
- Timed performance drill
Correct answer: Focus group interview
Focus groups allow in-depth, open-ended exploration of learner perceptions that quantitative surveys cannot capture.
Question 2: A training evaluation reveals high Level 2 scores but no change in Level 3 behavior. The MOST likely explanation is:
- The assessment lacked reliability
- Transfer barriers exist in the work environment (Correct answer)
- The pre-test was too easy
- Learners guessed on test items
Correct answer: Transfer barriers exist in the work environment
When learning occurs but behavior doesn't change, environmental factors such as lack of managerial support or opportunity to practice are usually responsible.
Question 3: Which term describes the degree to which an assessment consistently produces the same results under the same conditions?
- Validity
- Reliability (Correct answer)
- Alignment
- Authenticity
Correct answer: Reliability
Reliability refers to the consistency and repeatability of an assessment across different administrations or raters.
Question 4: When evaluating a simulation exercise, which assessment approach BEST captures complex problem-solving ability?
- True/false checklist
- Holistic scoring rubric (Correct answer)
- Standardized multiple-choice test
- Peer nomination survey
Correct answer: Holistic scoring rubric
Holistic rubrics evaluate the overall quality of a complex performance rather than isolated subskills.
Question 5: In the Phillips ROI Methodology, which step comes AFTER calculating net program benefits?
- Identifying intangible benefits
- Isolating the effects of training
- Comparing benefits to program costs (Correct answer)
- Collecting reaction data
Correct answer: Comparing benefits to program costs
After calculating net benefits (benefits minus costs), the ROI percentage is calculated by dividing net benefits by costs.
Question 6: A trainer uses the same test before and after instruction to calculate learning gain. This design is called a:
- Criterion-referenced evaluation
- Pre-test/post-test design (Correct answer)
- Time-series analysis
- Solomon four-group design
Correct answer: Pre-test/post-test design
A pre-test/post-test design measures the difference in scores before and after instruction to quantify learning gain.
Question 7: Which of the following is an example of an unobtrusive measure in training evaluation?
- Administering a knowledge quiz
- Conducting a structured interview
- Reviewing production error rates from existing records (Correct answer)
- Distributing a satisfaction survey
Correct answer: Reviewing production error rates from existing records
Unobtrusive measures use existing organizational data without requiring learners to respond directly to an instrument.
CIT Assessment & Evaluation Techniques 4
Which data collection method is MOST appropriate for capturing nuanced learner perceptions about a training program's relevance?