MSRA Assessment and Evaluation 3 — Questions and Answers
Question 1: A trainee's 360-degree feedback reveals that nursing staff consistently rate their communication as poor, while consultants rate it as satisfactory. What does this discrepancy most likely indicate?
- The nursing staff ratings are invalid and should be discarded
- The trainee may communicate differently depending on the perceived seniority of colleagues (Correct answer)
- The consultants are lenient raters and introduce positive bias
- The 360-degree tool has poor inter-rater reliability
Correct answer: The trainee may communicate differently depending on the perceived seniority of colleagues
Discrepant ratings across professional groups often indicate that a trainee's interpersonal behaviour varies by hierarchy, a clinically important insight for professional development.
Question 2: Which validity threat is introduced when an assessor gives uniformly high scores to avoid uncomfortable feedback conversations?
- Halo effect
- Leniency bias (Correct answer)
- Hawthorne effect
- Recency bias
Correct answer: Leniency bias
Leniency bias occurs when an assessor inflates scores to avoid difficult feedback, undermining the discriminative validity of the assessment.
Question 3: A trainee asks their educational supervisor to sign off a clinical procedure they have not directly observed. What should the supervisor do?
- Sign off based on the trainee's verbal account to save time
- Decline and arrange a directly observed assessment before signing off (Correct answer)
- Ask a nurse who was present to confirm competence verbally, then sign off
- Sign off with a note that the procedure was not directly observed
Correct answer: Decline and arrange a directly observed assessment before signing off
Workplace-based assessments require direct observation to maintain validity; signing off unseen procedures compromises assessment integrity and patient safety.
Question 4: According to the MSRA framework, which factor most strongly predicts a trainee's future professional performance?
- Written examination scores alone
- A combination of clinical and professional assessments over time (Correct answer)
- Single high-stakes OSCE performance
- Self-reported reflective logs
Correct answer: A combination of clinical and professional assessments over time
Longitudinal assessment combining clinical skills and professional behaviour across multiple tools provides the most predictive validity for future performance.
Question 5: A trainee appeals their ARCP outcome, claiming the panel was biased. What is the standard first stage of the appeals process?
- External review by the GMC
- Internal review by the deanery/HEE without the original panel members (Correct answer)
- Re-sitting all workplace-based assessments
- Referral to a fitness-to-practise panel
Correct answer: Internal review by the deanery/HEE without the original panel members
The first stage of an ARCP appeal is typically an internal review conducted by individuals not on the original panel, ensuring procedural fairness.
Question 6: What is the primary purpose of setting a 'standard' before conducting a structured clinical assessment?
- To reduce the number of questions in the assessment
- To ensure assessors share a common benchmark for performance expectations (Correct answer)
- To make the assessment easier for borderline trainees
- To limit the range of clinical scenarios used
Correct answer: To ensure assessors share a common benchmark for performance expectations
Pre-defined standards align assessor expectations and reduce inter-rater variability, improving the reliability and fairness of the assessment.
Question 7: A trainee in difficulty is referred to an educational supervisor for an enhanced learning agreement. Which element is most essential in this document?
- A list of all previous assessment scores
- Specific, measurable objectives with a clear timeline and review date (Correct answer)
- A statement from the trainee acknowledging their deficiencies
- Details of all clinical cases seen in the last three months
Correct answer: Specific, measurable objectives with a clear timeline and review date
An enhanced learning agreement must contain SMART objectives with a timeline and review date to provide a structured pathway for improvement.
MSRA Assessment and Evaluation 3
A trainee's 360-degree feedback reveals that nursing staff consistently rate their communication as poor, while consultants rate it as satisfactory.
What does this discrepancy most likely indicate?