CAE Evaluation Strategies 2 — Questions and Answers
Question 1: A CAE practitioner wants to measure whether adult learners can apply newly learned CPR skills in a real scenario. Which evaluation type BEST assesses this?
- Written multiple-choice test
- Performance-based assessment (Correct answer)
- Self-reported confidence survey
- Pre-program needs assessment
Correct answer: Performance-based assessment
Performance-based assessment directly measures skill application in realistic conditions, making it the most valid measure for procedural competencies like CPR.
Question 2: Which of the following BEST describes the purpose of formative evaluation in adult education?
- To assign final grades to participants
- To provide ongoing feedback that improves instruction during delivery (Correct answer)
- To determine program cost-effectiveness after completion
- To compare outcomes across different cohorts
Correct answer: To provide ongoing feedback that improves instruction during delivery
Formative evaluation occurs during instruction and is used to make real-time adjustments that improve learning while the program is still in progress.
Question 3: An adult educator uses exit tickets at the end of each session. This practice is an example of:
- Summative evaluation
- Needs assessment
- Formative evaluation (Correct answer)
- Impact evaluation
Correct answer: Formative evaluation
Exit tickets collected during instruction are a formative evaluation tool that provides immediate feedback on session comprehension.
Question 4: When designing a rubric for adult learner assessment, which element is MOST critical to ensure inter-rater reliability?
- Using open-ended criteria descriptions
- Defining explicit, observable performance indicators for each level (Correct answer)
- Limiting the rubric to only two performance levels
- Allowing instructors to modify criteria per learner
Correct answer: Defining explicit, observable performance indicators for each level
Explicit, observable indicators reduce subjective interpretation, enabling different raters to score the same performance consistently.
Question 5: A trainer suspects that a post-test score increase is due to practice effects rather than learning. The BEST design to control for this threat is a:
- Single-group pretest-posttest design
- Solomon four-group design (Correct answer)
- Time-series design without a control group
- Retrospective post-only design
Correct answer: Solomon four-group design
The Solomon four-group design controls for both testing effects and maturation by including groups that do and do not receive a pretest.
Question 6: Kirkpatrick's Level 3 evaluation focuses on:
- Learner satisfaction with instruction
- Knowledge gained during training
- Transfer of learning to the workplace (Correct answer)
- Organizational results and ROI
Correct answer: Transfer of learning to the workplace
Kirkpatrick's Level 3 (Behavior) measures whether learners actually apply what they learned on the job after training.
Question 7: Which statistical measure is MOST appropriate for summarizing the central tendency of ordinal Likert-scale survey data from adult learners?
- Mean
- Median (Correct answer)
- Standard deviation
- Variance
Correct answer: Median
The median is the preferred measure of central tendency for ordinal data because it does not assume equal intervals between scale points.
CAE Evaluation Strategies 2
A CAE practitioner wants to measure whether adult learners can apply newly learned CPR skills in a real scenario.
Which evaluation type BEST assesses this?