Free CHSE Assessment and Evaluation Methods Questions and Answers 1 — Questions and Answers
Question 1: A simulation educator runs a 'mock code' scenario for residents halfway through their rotation. The educator provides immediate, targeted feedback on performance using a checklist but does not assign a grade. The primary goal is to identify areas for improvement before the end of the rotation. This is an example of which type of assessment?
- Summative assessment
- Ipsative assessment
- Formative assessment (Correct answer)
- Diagnostic assessment
Correct answer: Formative assessment
Formative assessment is conducted during the learning process to monitor progress and provide ongoing feedback to improve learning. [13, 21, 28] Since the goal is to identify areas for improvement without assigning a final grade, it is formative. Summative assessment evaluates learning at the end of an instructional unit. [21, 27]
Question 2: An educator is designing an assessment for a complex skill, such as breaking bad news, which involves empathy, rapport-building, and adapting to emotional cues. Which assessment tool is most appropriate for capturing the nuances of a learner's performance in these domains?
- A global rating scale with descriptive anchors (Correct answer)
- A binary (done/not done) procedural checklist
- A post-simulation multiple-choice question test
- A structured peer-evaluation survey
Correct answer: A global rating scale with descriptive anchors
A global rating scale (GRS) is better suited for assessing complex skills and behaviors like communication and professionalism. [8, 12, 17] Unlike a binary checklist that focuses on discrete actions, a GRS with descriptive anchors allows raters to make a holistic judgment about the quality of a performance across a continuum. [12]
Question 3: A simulation program is implementing a high-stakes Objective Structured Clinical Examination (OSCE) to credential learners for a specific procedure. To ensure fairness and consistency in scoring across all participants, which of the following is the most critical element to implement?
- Using the newest high-fidelity manikin available
- Providing learners with the scoring tool beforehand
- Ensuring the moulage is identical for every learner
- Conducting robust rater training to establish inter-rater reliability (Correct answer)
Correct answer: Conducting robust rater training to establish inter-rater reliability
For any performance-based assessment, especially a high-stakes one, ensuring that different raters score the same performance consistently is paramount for fairness and validity. [18, 19, 23] Robust rater training is the primary method to achieve high inter-rater reliability, minimizing variability that comes from examiner judgment. [19, 23]
Question 4: A simulation program director is using the CIPP (Context, Input, Process, Product) model to conduct a comprehensive evaluation of a new curriculum. Which of the following questions would be addressed during the 'Input' evaluation phase?
- Did the program achieve its intended learning outcomes for the participants?
- Are the available faculty, funding, and equipment adequate to implement the curriculum as planned? (Correct answer)
- Were the simulation scenarios and debriefings delivered consistently and according to the design?
- What are the key institutional needs and priorities that this curriculum is designed to address?
Correct answer: Are the available faculty, funding, and equipment adequate to implement the curriculum as planned?
The 'Input' phase of the CIPP model focuses on assessing the adequacy of resources, the feasibility of the plan, and the potential strategies. [2, 11, 16] It answers the question, 'How should it be done?' [11] 'Product' evaluation addresses outcomes (A), 'Process' evaluation addresses implementation fidelity (C), and 'Context' evaluation addresses needs (D). [2, 16]
Question 5: A CHSE convenes a panel of subject matter experts to establish a defensible pass/fail score for a new summative assessment. The experts review each item on the scoring tool and estimate the probability that a minimally competent practitioner would perform the item correctly. This process is a hallmark of which standard-setting method?
- The Angoff method (Correct answer)
- The Kirkpatrick method
- The Plus-Delta method
- The BARS method
Correct answer: The Angoff method
The Angoff method is a widely used, evidence-based procedure where a panel of experts evaluates assessment items to determine a cut-score based on the expected performance of a 'minimally competent' individual. [1, 4, 6] This approach bases the passing standard on the content of the exam, not on the performance of the group taking it. [4, 5]
Question 6: An educator develops a new assessment checklist for a central venous catheter insertion scenario. To help establish the tool's validity, the educator asks a group of experienced intensivists to review the checklist to ensure it includes all critical steps and excludes any irrelevant ones. This activity is primarily focused on gathering what type of validity evidence?
- Predictive validity
- Concurrent validity
- Content validity (Correct answer)
- Construct validity
Correct answer: Content validity
Content validity refers to the extent to which an assessment tool adequately represents all facets of the concept or skill being measured. [29] Using subject matter experts to review the instrument's items for relevance and completeness is a fundamental step in establishing evidence of content validity. [3, 25]
Free CHSE Assessment and Evaluation Methods Questions and Answers 1
A simulation educator runs a 'mock code' scenario for residents halfway through their rotation.
The educator provides immediate, targeted feedback on performance using a checklist but does not assign a grade.
The primary goal is to identify areas for improvement before the end of the rotation.
This is an example of which type of assessment?