Structured Oral Examination Design Flashcards

Before administering a structured oral examination, rater training should specifically include which activity?

Answer: Reviewing scoring anchors and practicing rating sample responses

Calibration practice with scoring anchors before the exam reduces inter-rater variance.

Which of the following is a common rater error that structured oral examination design seeks to minimize?

Answer: Halo effect, where a strong first impression inflates ratings across all dimensions

The halo effect causes raters to let one positive trait unduly raise scores on all other dimensions.

After a structured oral examination, which statistical measure is most useful for evaluating how consistently raters scored candidates?

Answer: Inter-rater reliability coefficient

Inter-rater reliability quantifies the degree of agreement among raters, indicating scoring consistency.

In a structured oral exam for a public agency, the number of dimensions rated should ideally be determined by which factor?

Answer: The number of critical KSAs identified in the job analysis

Each rated dimension should correspond to a critical KSA so the exam measures what the job actually requires.

What is the main risk of allowing raters to take extensive personal notes about candidate appearance during a structured oral examination?

Answer: It can introduce bias based on non-job-related characteristics and create legal liability

Notes about appearance or non-job factors can be evidence of discriminatory consideration in legal challenges.

When is it appropriate to use a structured oral examination as the sole selection tool for a public agency position?

Answer: When the job's critical KSAs are primarily interpersonal, communication-based, and best demonstrated orally

An oral exam is best suited as a sole tool when the essential job competencies are oral in nature and well-captured through structured questioning.