CBT - Computer Based Testing Psychometric Principles and Analysis Questions and Answers

Question 1

A test developer for a new certification exam calculates an item difficulty index (p-value) of 0.92 for a multiple-choice question. What does this value indicate about the item?

Accepted Answer

The item is very easy, as 92% of test-takers answered it correctly.

Answer

The item difficulty index (p-value) represents the proportion of test-takers who answered the item correctly. A value of 0.92 means that 92% of the sample answered correctly, indicating that the item is very easy. While easy items can be appropriate for certain goals, this one would likely have low discrimination because most candidates, regardless of ability, answered it correctly.

Question 2

In the context of Computerized Adaptive Testing (CAT), which psychometric theory is most essential for the item selection algorithm?

Accepted Answer

Item Response Theory (IRT)

Answer

Item Response Theory (IRT) is the foundational psychometric model for CAT. IRT places both test-taker ability and item difficulty on the same scale, allowing the algorithm to select items that are optimally informative for a candidate's estimated ability level. This enables the test to be tailored in real-time.

Question 3

Which of the following metrics provides an estimate of the consistency of a test's results if a person were to take the same test multiple times?

Accepted Answer

Standard Error of Measurement (SEM)

Answer

The Standard Error of Measurement (SEM) quantifies the amount of error in an individual's test score. It is used to create a confidence interval around the observed score, indicating the likely range where the individual's 'true score' lies. A smaller SEM indicates higher reliability and more confidence that the observed score is close to the true score upon retesting.

Question 4

A psychometrician is analyzing a new test item and finds it has a discrimination index of -0.15. What is the most appropriate action to take?

Accepted Answer

Review the item for flaws, as high-scorers are getting it wrong more often than low-scorers.

Answer

A negative discrimination index indicates that lower-scoring test-takers are answering the item correctly more often than higher-scoring test-takers. This is a significant red flag, suggesting the item may be poorly worded, have a confusing key, or be otherwise flawed. The item should be reviewed and likely removed or substantially revised.

Question 5

A certification body is transitioning its exam from a linear, fixed-form test to a Computer Based Test (CBT). They want to ensure that scores are comparable across different versions of the exam administered on different days. This psychometric property is known as:

Accepted Answer

Equating

Answer

Equating is the statistical process used to adjust scores on different forms of a test to make them comparable. This is crucial in CBT environments where different candidates receive different sets of items or test forms, ensuring that a score of, for example, 85 on one form is equivalent in meaning to a score of 85 on another, more or less difficult, form.

CBT - Computer Based Testing Practice Test

CBT - Computer Based Testing Practice Test

CBT - Computer Based Testing Psychometric Principles and Analysis Questions and Answers