CHSE Assessment and Evaluation Methods 3 — Questions and Answers
Question 1: During debriefing, a faculty member notices a learner consistently underestimates her own performance. Which debriefing technique BEST addresses this self-assessment gap?
- Directive feedback without learner input
- Advocacy-inquiry to explore the gap between self-assessment and observed performance (Correct answer)
- Immediate correction of every error during the scenario
- Delaying feedback until the next simulation session
Correct answer: Advocacy-inquiry to explore the gap between self-assessment and observed performance
Advocacy-inquiry combines the facilitator's observation with a genuine question, inviting the learner to reflect on and explain the gap in their self-assessment.
Question 2: A simulation center wishes to compare its trainees' performance against a national benchmark. Which assessment approach is MOST appropriate?
- Criterion-referenced assessment using local cut scores
- Norm-referenced assessment using national standardization data (Correct answer)
- Formative peer assessment within the cohort
- Self-assessment reflective journals
Correct answer: Norm-referenced assessment using national standardization data
Norm-referenced assessment ranks learner performance relative to a broader reference group, making it suitable for comparison against national benchmarks.
Question 3: Which statistical measure BEST describes the consistency of an assessment tool when administered on two separate occasions to the same group of learners?
- Content validity index
- Test-retest reliability coefficient (Correct answer)
- Inter-rater reliability coefficient
- Cronbach's alpha
Correct answer: Test-retest reliability coefficient
Test-retest reliability measures the stability of scores over time when the same instrument is administered twice to the same participants.
Question 4: In simulation-based assessment, 'deliberate practice' is MOST closely associated with which outcome?
- Single exposure to a scenario leading to competency
- Repeated, focused practice with immediate feedback to mastery (Correct answer)
- Observing expert performance without active participation
- Completing written examinations between simulation sessions
Correct answer: Repeated, focused practice with immediate feedback to mastery
Deliberate practice involves structured, repetitive practice with targeted feedback aimed at achieving a defined mastery standard.
Question 5: A residency program implements simulation checkpoints where trainees must meet a minimum performance standard before advancing. This BEST exemplifies:
- Norm-referenced gatekeeping
- Mastery learning with criterion-referenced standards (Correct answer)
- Summative peer evaluation
- Reflective portfolio assessment
Correct answer: Mastery learning with criterion-referenced standards
Mastery learning requires all learners to achieve a predefined criterion before advancement, regardless of how their peers perform.
Question 6: Which of the following is the PRIMARY limitation of using global rating scales compared to task-specific checklists in simulation assessment?
- Global rating scales cannot be used for procedural skills
- Global rating scales may lack the granularity needed to identify specific skill deficits (Correct answer)
- Global rating scales are too time-consuming for practical use
- Global rating scales have poor face validity for clinical educators
Correct answer: Global rating scales may lack the granularity needed to identify specific skill deficits
While global rating scales capture overall performance efficiently, they may miss discrete skill errors that a detailed task checklist would identify.
Question 7: When a simulation-based assessment is described as 'authentic,' it PRIMARILY means the assessment:
- Uses the highest-fidelity simulator available
- Reflects real-world tasks and contexts relevant to professional practice (Correct answer)
- Is administered by certified CHSE professionals only
- Produces scores that are statistically reliable
Correct answer: Reflects real-world tasks and contexts relevant to professional practice
Authentic assessment requires learners to perform tasks that closely mirror actual professional responsibilities and real clinical contexts.
CHSE Assessment and Evaluation Methods 3
During debriefing, a faculty member notices a learner consistently underestimates her own performance.
Which debriefing technique BEST addresses this self-assessment gap?