CBT - Computer Based Testing Test Assembly and Delivery Questions and Answers — Questions and Answers
Question 1: A certification body is implementing Linear-On-the-Fly Testing (LOFT) for its high-stakes exam. Which of the following is the primary advantage of this test assembly model over traditional fixed-form exams?
- It adapts the difficulty of each question to the candidate's ability level.
- It significantly reduces the required size of the item bank compared to fixed forms.
- It generates a unique, psychometrically equivalent test form for each candidate in real-time. (Correct answer)
- It eliminates the need for subject matter experts (SMEs) to review items before they are used.
Correct answer: It generates a unique, psychometrically equivalent test form for each candidate in real-time.
Linear-On-the-Fly Testing (LOFT) assembles a unique, full-length test for each candidate just before the exam begins. The algorithm pulls items from a large bank based on a blueprint to ensure each test form is equivalent in content and statistical difficulty, which greatly enhances security by minimizing item overlap between candidates.
Question 2: During the delivery of a remotely proctored CBT exam, the proctoring software flags a candidate for frequently looking away from the screen. The software uses AI to analyze webcam footage, gaze direction, and background noise. This type of security measure is primarily designed to prevent which threat?
- Item harvesting by screen-capturing questions.
- Use of unauthorized physical materials or assistance from another person. (Correct answer)
- Candidate impersonation or proxy testing.
- Use of unauthorized software or web browsing on the testing computer.
Correct answer: Use of unauthorized physical materials or assistance from another person.
AI-powered proctoring systems that monitor a candidate's physical behavior, such as eye movements and background activity, are specifically designed to detect cheating attempts involving unauthorized aids like notes, textbooks, or receiving help from someone else in the room.
Question 3: Which of the following is a critical prerequisite for successfully implementing a Linear-On-the-Fly Testing (LOFT) model?
- A large and well-calibrated item bank with detailed metadata. (Correct answer)
- A small, highly secure pool of items that are reused frequently.
- An algorithm that presents easier questions first to build candidate confidence.
- A delivery system that allows candidates to skip questions and return to them later.
Correct answer: A large and well-calibrated item bank with detailed metadata.
The LOFT model relies on an extensive, robust item bank where each item is calibrated and tagged with metadata (e.g., content area, difficulty, cognitive level). This allows the assembly algorithm to construct unique forms that are statistically equivalent and conform to the test blueprint. Without a large pool, item exposure would be too high, defeating the security purpose of the model.
Question 4: A test development organization uses an automated test assembly (ATA) algorithm to build multiple parallel forms of a certification exam. A key feature of this algorithm is its ability to recognize and avoid pairing certain items together on the same form because they contain overlapping content or one gives away the answer to another. These items are best known as:
- Anchor Items
- Calibrated Items
- Enemy Items (Correct answer)
- Distractor Items
Correct answer: Enemy Items
Enemy items (or item enemies) are pairs or sets of items that should not appear on the same test form due to content overlap, where one item may provide clues or the direct answer to another. Automated test assembly algorithms are programmed with rules to prevent these pairings, ensuring test fairness and validity.
Question 5: In a Computerized Adaptive Test (CAT), an item exposure control strategy is implemented where items are grouped into strata based on their discrimination parameter ('a' value). The algorithm selects items from lower-discrimination strata early in the test and from higher-discrimination strata as the ability estimate becomes more precise. What is this strategy called?
- Sympson-Hetter Method
- a-Stratification (Correct answer)
- Linear-On-the-Fly Testing (LOFT)
- Maximum Information Selection
Correct answer: a-Stratification
The a-Stratification method is an item exposure control strategy used in CAT. It involves dividing the item bank into several levels (strata) based on the items' 'a' (discrimination) parameter. The test then pulls items from different strata at different stages of the exam to balance measurement precision with item exposure control, preventing the most discriminating (and often most exposed) items from being used too frequently.
Question 6: After a new CBT form has been administered to its first cohort of candidates, the testing organization conducts an analysis of item performance data, overall test reliability, and candidate comments. The primary purpose of this activity, known as post-test analysis, is to:
- Set the final passing score for the examination.
- Assign final scores to each individual candidate.
- Validate the test assembly process and identify flawed items for future revision. (Correct answer)
- Generate unique test forms for the next group of candidates.
Correct answer: Validate the test assembly process and identify flawed items for future revision.
Post-test analysis involves reviewing all available data after a test administration to evaluate its performance. This includes analyzing item statistics (like difficulty and discrimination), checking the test's overall reliability, and reviewing candidate feedback. This process is crucial for quality control, validating that the test form functioned as intended, and identifying poorly performing items that need to be removed from the active pool or revised.
CBT - Computer Based Testing Test Assembly and Delivery Questions and Answers
A certification body is implementing Linear-On-the-Fly Testing (LOFT) for its high-stakes exam.
Which of the following is the primary advantage of this test assembly model over traditional fixed-form exams?