CODESP Structured Oral Examination Design 3 — Questions and Answers
Question 1: A public agency is designing a structured oral exam for a supervisory position. Which type of question is most appropriate for assessing leadership judgment?
- Situational questions presenting realistic supervisory dilemmas (Correct answer)
- Questions about the candidate's favorite management books
- Questions confirming the candidate's knowledge of the agency's organizational chart
- Open-ended questions about the candidate's career aspirations
Correct answer: Situational questions presenting realistic supervisory dilemmas
Situational questions directly test how candidates apply supervisory judgment in realistic scenarios drawn from the job.
Question 2: Which practice best ensures that structured oral exam questions remain legally defensible under Title VII and the Uniform Guidelines?
- Documenting the link between each question and a specific KSA from a formal job analysis (Correct answer)
- Having an attorney approve the general topic of each question
- Using only questions that have been used by other agencies in the past
- Letting candidates choose which questions they prefer to answer
Correct answer: Documenting the link between each question and a specific KSA from a formal job analysis
The Uniform Guidelines require that selection procedures be job-related, which demands documented KSA linkage from job analysis.
Question 3: A candidate asks for a copy of the oral examination scoring sheet after learning they did not pass. Under California public records principles, the agency should generally:
- Consult with legal counsel regarding what materials are releasable while protecting the exam's future integrity (Correct answer)
- Provide the full scoring sheet with all rater notes immediately upon request
- Refuse all disclosure because exam materials are always confidential
- Delete all scoring records to avoid future requests
Correct answer: Consult with legal counsel regarding what materials are releasable while protecting the exam's future integrity
Agencies must balance public records transparency with test security and consult counsel before releasing exam content.
Question 4: If one rater in a three-person oral exam panel consistently scores significantly higher than the other two raters, the recommended corrective action is to:
- Conduct post-exam rater calibration and, if necessary, statistically normalize or flag the outlier scores (Correct answer)
- Accept all scores as-is to avoid challenging the rater's professional judgment
- Dismiss the outlier rater's scores entirely from the final average
- Require all candidates to retake the oral exam with a new panel
Correct answer: Conduct post-exam rater calibration and, if necessary, statistically normalize or flag the outlier scores
Calibration and statistical review are appropriate responses to systematic inter-rater discrepancy without invalidating the exam outright.
Question 5: To protect against order effects in a structured oral exam, which scheduling strategy is most effective?
- Randomly assigning the order in which candidates are interviewed (Correct answer)
- Scheduling the most experienced candidates first to set a benchmark
- Interviewing internal candidates before external candidates
- Scheduling all candidates on the same day in alphabetical order
Correct answer: Randomly assigning the order in which candidates are interviewed
Random interview order prevents the first or last candidate position from consistently advantaging or disadvantaging applicants.
Question 6: Which element is essential to include in the written instructions provided to candidates before a structured oral examination?
- The format, time allotted per question, and how responses will be scored (Correct answer)
- The names and credentials of each rater on the panel
- The identity of other candidates competing for the same position
- The agency's preferred answer to each question
Correct answer: The format, time allotted per question, and how responses will be scored
Clear pre-exam instructions on format and scoring criteria ensure all candidates can perform to their best ability under equal conditions.
CODESP Structured Oral Examination Design 3
A public agency is designing a structured oral exam for a supervisory position.
Which type of question is most appropriate for assessing leadership judgment?