CODESP - Cooperative Organization for the Development of Employee Selection Procedures Test Validation and Reliability Questions and Answers

Question 1

A personnel commission uses a panel of three subject matter experts (SMEs) to score candidates' oral interview responses for a "Fire Captain" promotion.
After the interviews, a review of the scores reveals a very low correlation among the ratings provided by the three SMEs for the same candidates.
This low correlation indicates a problem with which type of reliability?

Accepted Answer

Inter-rater reliability

Answer

Test-retest reliability

Answer

Internal consistency

Answer

Parallel forms reliability

Question 2

A public agency wants to conduct a criterion-related validity study for a new pre-employment test for budget analysts. They administer the test to all current budget analysts and simultaneously collect their most recent performance appraisal scores. The agency then correlates the test scores with the performance scores. What specific type of validation strategy is this?

Accepted Answer

Concurrent validity

Answer

Predictive validity

Answer

Content validity

Answer

Construct validity

Question 3

A public works department uses a test to select equipment operators. The test consistently produces the same scores for individuals who take it multiple times (high test-retest reliability). However, there is no correlation between scores on the test and subsequent measures of on-the-job performance. Which statement best describes this selection test?

Accepted Answer

The test is reliable, but not valid.

Answer

The test is valid, but not reliable.

Answer

The test is both reliable and valid.

Answer

The test is neither reliable nor valid.

Question 4

A police department wants to develop a selection test to measure the abstract characteristic of "integrity," which is difficult to observe directly. To validate this test, they show that scores on it are highly correlated with scores on other established integrity tests and are not correlated with measures of unrelated traits like cognitive ability. This approach is primarily focused on establishing which type of validity?

Accepted Answer

Construct validity

Answer

Content validity

Answer

Predictive validity

Answer

Concurrent validity

Question 5

An analyst develops a new 100-item multiple-choice test for a clerical position and calculates a reliability coefficient of .55 using Cronbach's alpha. What is the most accurate interpretation of this result?

Accepted Answer

The test is not sufficiently reliable for making selection decisions, as scores contain a large amount of measurement error.

Answer

The test is highly reliable and ready for operational use.

Answer

The test scores are strongly correlated with job performance.

Answer

55% of the applicants will pass the test.

Question 6

Which of the following is the most critical step in establishing the content validity of a written examination for a journey-level electrician position?

Accepted Answer

Ensuring the test items are a representative sample of the important knowledge, skills, and abilities (KSAs) identified in a thorough job analysis.

Answer

Correlating test scores with scores on a different, already-validated electrician test.

Answer

Administering the test to applicants and later correlating their scores with their on-the-job performance ratings.

Answer

Demonstrating that the test measures the theoretical construct of "electrical aptitude."