CODESP Structured Oral Examination Design 5

Question 1

A jurisdiction wants to convert its structured oral examination scores to a rank-ordered list. Which psychometric consideration is most important?

Accepted Answer

Verifying that score differences are statistically meaningful before treating ranks as distinct

Answer

Rank-ordering requires that score differences exceed measurement error; candidates with statistically indistinguishable scores should be treated as tied.

Question 2

In a structured oral examination, 'situational questions' differ from 'experience-based questions' primarily in that situational questions:

Accepted Answer

Ask candidates what they would do in a hypothetical future scenario

Answer

Situational (or hypothetical) questions present a future scenario and ask intended behavior, while experience-based (behavioral) questions ask for past actual behavior.

Question 3

Which documentation practice is most essential for defending a structured oral examination against a legal challenge?

Accepted Answer

Keeping records of the job analysis, question development rationale, rater training, and individual scores

Answer

Comprehensive documentation of the entire development and administration process provides the evidentiary trail needed to demonstrate job-relatedness and fairness.

Question 4

When a structured oral examination panel includes a subject matter expert (SME), what is their primary role during scoring?

Accepted Answer

To evaluate response quality against job-relevant technical and competency benchmarks

Answer

SMEs provide domain expertise to judge whether candidate responses reflect the knowledge, skills, and abilities actually required on the job.

Question 5

A CODESP oral examination yields an inter-rater reliability coefficient of 0.55. What action is most appropriate?

Accepted Answer

Investigate rater training gaps and scoring anchor clarity before using the exam operationally

Answer

A reliability coefficient of 0.55 is below the generally accepted threshold of approximately 0.70, signaling that rater calibration and anchor definition need improvement.

CODESP - Cooperative Organization for the Development of Employee Selection Procedures Practice Test

CODESP - Cooperative Organization for the Development of Employee Selection Procedures Practice Test

CODESP Structured Oral Examination Design 5