OSCE Evaluation and Feedback 3 — Questions and Answers
Question 1: An OSCE examiner notices a student is visibly anxious and performs well below their expected level. What is the most appropriate evaluator response?
- Adjust the score upward to account for anxiety
- Score objectively using the marking scheme regardless of affect (Correct answer)
- Stop the station early and reschedule
- Offer reassurance mid-station that will not affect the score
Correct answer: Score objectively using the marking scheme regardless of affect
Examiners must score objectively using the predetermined marking scheme; adjusting scores based on perceived anxiety undermines validity.
Question 2: A faculty member reviews OSCE item analysis data and finds one question has a negative point-biserial correlation. This indicates that:
- High-scoring students answered the item correctly more often than low-scoring students
- Low-scoring students answered the item correctly more often than high-scoring students (Correct answer)
- The item was too easy for all students
- The item had excellent discriminability
Correct answer: Low-scoring students answered the item correctly more often than high-scoring students
A negative point-biserial means better-performing candidates are getting the item wrong more than weaker ones, flagging the item for review or removal.
Question 3: Which of the following feedback types is LEAST useful for helping a student improve clinical performance?
- Specific: 'You missed asking about medication allergies before prescribing'
- Timely: given within 24 hours of the station
- Vague: 'You need to do better overall' (Correct answer)
- Actionable: 'Practice the SBAR handover framework before your next station'
Correct answer: Vague: 'You need to do better overall'
Vague feedback lacks specificity and actionability, making it impossible for the student to know what to change.
Question 4: In high-stakes OSCE evaluation, standardized patients (SPs) are asked to complete rating forms after each encounter. The PRIMARY advantage of SP ratings over examiner checklists is that SPs can assess:
- Technical procedural accuracy
- The patient's subjective experience of communication and empathy (Correct answer)
- Correct diagnosis formulation
- Appropriate medication dosing
Correct answer: The patient's subjective experience of communication and empathy
SPs are uniquely positioned to rate interpersonal and communication qualities from the patient's perspective, which examiners may not fully capture.
Question 5: A student asks whether their OSCE score will be used for formative or summative purposes. What is the key distinction?
- Formative scores count toward graduation; summative do not
- Summative scores contribute to pass/fail decisions; formative scores guide learning without high-stakes consequences (Correct answer)
- Both types of scores are identical in their impact on the student
- Formative assessments are only used in the first year of training
Correct answer: Summative scores contribute to pass/fail decisions; formative scores guide learning without high-stakes consequences
Summative assessment determines competence for progression, while formative assessment provides feedback to direct future learning.
Question 6: During an OSCE feedback session, a student disagrees with the examiner's assessment and becomes defensive. The BEST approach is to:
- Abandon the feedback session to avoid conflict
- Change the score to reduce tension
- Acknowledge the student's perspective, then return to evidence from the marking criteria (Correct answer)
- Escalate immediately to the course director
Correct answer: Acknowledge the student's perspective, then return to evidence from the marking criteria
Acknowledging feelings validates the student while grounding the discussion in objective criteria maintains the integrity of the evaluation.
Question 7: What does a high Cronbach's alpha (e.g., 0.85) for an OSCE examination indicate?
- The exam is too easy and needs harder stations
- The stations are internally consistent and reliably measure a similar construct (Correct answer)
- Students performed uniformly across all stations
- The pass mark is set too high
Correct answer: The stations are internally consistent and reliably measure a similar construct
Cronbach's alpha measures internal consistency; a value of 0.85 indicates that stations are reliably measuring a coherent underlying construct.
OSCE Evaluation and Feedback 3
An OSCE examiner notices a student is visibly anxious and performs well below their expected level.
What is the most appropriate evaluator response?