CODESP Structured Oral Examination Design 2

Question 1

Before administering a structured oral examination, rater training should specifically include which activity?

Accepted Answer

Reviewing scoring anchors and practicing rating sample responses

Answer

Calibration practice with scoring anchors before the exam reduces inter-rater variance.

Question 2

Which of the following is a common rater error that structured oral examination design seeks to minimize?

Accepted Answer

Halo effect, where a strong first impression inflates ratings across all dimensions

Answer

The halo effect causes raters to let one positive trait unduly raise scores on all other dimensions.

Question 3

After a structured oral examination, which statistical measure is most useful for evaluating how consistently raters scored candidates?

Accepted Answer

Inter-rater reliability coefficient

Answer

Inter-rater reliability quantifies the degree of agreement among raters, indicating scoring consistency.

Question 4

In a structured oral exam for a public agency, the number of dimensions rated should ideally be determined by which factor?

Accepted Answer

The number of critical KSAs identified in the job analysis

Answer

Each rated dimension should correspond to a critical KSA so the exam measures what the job actually requires.

Question 5

What is the main risk of allowing raters to take extensive personal notes about candidate appearance during a structured oral examination?

Accepted Answer

It can introduce bias based on non-job-related characteristics and create legal liability

Answer

Notes about appearance or non-job factors can be evidence of discriminatory consideration in legal challenges.

CODESP - Cooperative Organization for the Development of Employee Selection Procedures Practice Test