CBT - Computer Based Testing Psychometric Principles and Analysis Questions and Answers

Question 1

A test developer for a new certification exam calculates an item difficulty index (p-value) of 0.92 for a multiple-choice question. What does this value indicate about the item?

Accepted Answer

The item is very easy, as 92% of test-takers answered it correctly.

Answer

The item is very difficult, as only 8% of test-takers answered it correctly.

Answer

The item has high discrimination, effectively differentiating between high and low performers.

Answer

The item may be flawed because the p-value is outside the acceptable range of -1.0 to +1.0.

Question 2

In the context of Computerized Adaptive Testing (CAT), which psychometric theory is most essential for the item selection algorithm?

Accepted Answer

Item Response Theory (IRT)

Answer

Generalizability Theory

Answer

Classical Test Theory (CTT)

Answer

Factor Analysis Theory

Question 3

Which of the following metrics provides an estimate of the consistency of a test's results if a person were to take the same test multiple times?

Accepted Answer

Standard Error of Measurement (SEM)

Answer

Item Discrimination Index

Answer

Content Validity Ratio

Answer

Cronbach's Alpha

Question 4

A psychometrician is analyzing a new test item and finds it has a discrimination index of -0.15. What is the most appropriate action to take?

Accepted Answer

Review the item for flaws, as high-scorers are getting it wrong more often than low-scorers.

Answer

Keep the item as it is, as it is functioning well.

Answer

Increase the difficulty of the item to improve discrimination.

Answer

Keep the item, but only for low-stakes practice exams.

Question 5

A certification body is transitioning its exam from a linear, fixed-form test to a Computer Based Test (CBT). They want to ensure that scores are comparable across different versions of the exam administered on different days. This psychometric property is known as:

Accepted Answer

Equating

Answer

Normalization

Answer

Standardization

Answer

Calibration

Question 6

When comparing Classical Test Theory (CTT) and Item Response Theory (IRT), which statement is most accurate?

Accepted Answer

IRT models the probability of a specific response based on person and item parameters, allowing for more sophisticated analysis than CTT.

Answer

CTT provides item statistics that are independent of the test-taker sample, while IRT statistics are sample-dependent.

Answer

IRT focuses on the total test score as the primary unit of analysis, whereas CTT focuses on individual item responses.

Answer

CTT is a more modern and complex framework, making it better suited for computer adaptive testing than IRT.

A test developer for a new certification exam calculates an item difficulty index (p-value) of 0.92 for a multiple-choice question.What does this value indicate about the item?

A test developer for a new certification exam calculates an item difficulty index (p-value) of 0.92 for a multiple-choice question.
What does this value indicate about the item?