KSA Assessment Design & Evaluation Methods 3

Question 1

The standard error of measurement (SEM) is used in assessment to:

Accepted Answer

Estimate the range of error around an observed score

Answer

The SEM quantifies measurement error and is used to build a confidence interval around an examinee's observed score.

Question 2

A structured behavioral interview differs from a traditional interview primarily because it:

Accepted Answer

Uses standardized questions and a scoring rubric based on past behavior

Answer

Structured behavioral interviews use consistent, pre-determined questions tied to job KSAs and score responses using an anchored rating scale.

Question 3

Which analysis identifies whether a test item functions differently for subgroups of similar ability, potentially indicating bias?

Accepted Answer

Differential Item Functioning (DIF) analysis

Answer

DIF analysis detects items where examinees of equal ability from different demographic groups have different probabilities of answering correctly, signaling potential bias.

Question 4

A summative evaluation of a training program is conducted:

Accepted Answer

After the program to judge overall effectiveness

Answer

Summative evaluation assesses the overall impact and outcomes of a completed program, unlike formative evaluation which informs ongoing improvements.

Question 5

In the Angoff method for setting cut scores, SMEs are asked to:

Accepted Answer

Estimate the probability that a minimally competent candidate answers each item correctly

Answer

The Angoff method has SMEs judge the probability that a borderline-competent candidate would correctly answer each test item, and these probabilities are summed to establish the cut score.

(KSA) Certified Knowledge, Skills, and Abilities Assessor Practice Test

(KSA) Certified Knowledge, Skills, and Abilities Assessor Practice Test

KSA Assessment Design & Evaluation Methods 3