PSI Assessment & Evaluation Methods 3 — Questions and Answers
Question 1: When constructing a multiple-choice test item, which practice BEST improves item quality?
- Including 'all of the above' as the correct answer frequently
- Making all distractors clearly wrong to reduce confusion
- Keeping all answer choices similar in length and grammatical structure (Correct answer)
- Writing the correct answer in a different format than distractors
Correct answer: Keeping all answer choices similar in length and grammatical structure
Parallel structure and similar length among all answer choices prevents test-takers from identifying the correct answer through format cues rather than knowledge.
Question 2: A training officer reviews end-of-course evaluations to determine whether a recruits' academy curriculum should be revised. This is an example of:
- Learner self-assessment
- Program evaluation (Correct answer)
- Competency mapping
- Affective domain testing
Correct answer: Program evaluation
Program evaluation examines the effectiveness of an entire training program rather than individual student performance.
Question 3: Which evaluation model focuses on measuring context, input, process, and product to guide program decision-making?
- Kirkpatrick's Four Levels
- CIPP Model (Correct answer)
- Bloom's Taxonomy
- ADDIE Framework
Correct answer: CIPP Model
Stufflebeam's CIPP Model evaluates Context, Input, Process, and Product to support continuous program improvement decisions.
Question 4: An EMS instructor allows students to grade each other's patient assessment documentation using a provided rubric. This technique is called:
- Authentic assessment
- Peer assessment (Correct answer)
- Portfolio evaluation
- Criterion-referenced testing
Correct answer: Peer assessment
Peer assessment involves students evaluating one another's work, promoting critical thinking and collaborative learning.
Question 5: In a skill performance evaluation, inter-rater reliability is best improved by:
- Allowing evaluators to use personal judgment without a checklist
- Training all evaluators on a standardized rubric before testing (Correct answer)
- Having only one evaluator score all students
- Reducing the number of criteria in the checklist
Correct answer: Training all evaluators on a standardized rubric before testing
Standardizing evaluator training and rubric interpretation reduces subjective variance and improves consistency across raters.
Question 6: Which level of Kirkpatrick's Four-Level Evaluation Model measures whether trained skills are being applied on the job?
- Level 1 – Reaction
- Level 2 – Learning
- Level 3 – Behavior (Correct answer)
- Level 4 – Results
Correct answer: Level 3 – Behavior
Level 3 (Behavior) assesses the degree to which participants apply what they learned when back in their work environment.
Question 7: A hazmat instructor assigns a final practical exam worth 40% of the course grade at the end of training. This is classified as:
- Formative assessment
- Diagnostic assessment
- Summative assessment (Correct answer)
- Embedded assessment
Correct answer: Summative assessment
Summative assessment evaluates cumulative learning at the conclusion of an instructional period and typically contributes to a final grade.
PSI Assessment & Evaluation Methods 3
When constructing a multiple-choice test item, which practice BEST improves item quality?