WeBWork Quality Assurance and Improvement 2 — Questions and Answers
Question 1: In WeBWorK quality assurance, which metric best measures whether a problem set's difficulty is appropriately calibrated?
- Average score across all students (Correct answer)
- Percentage of students who attempt the problem
- Standard deviation of scores across students
- Time to first correct submission
Correct answer: Average score across all students
Average score across all students directly reflects whether the problem set difficulty is appropriately calibrated for the target population.
Question 2: When a WeBWorK problem returns an error for a specific student input but not others, what is the most likely cause?
- Server overload
- Unhandled edge case in the problem code (Correct answer)
- Browser incompatibility
- Incorrect student login
Correct answer: Unhandled edge case in the problem code
Errors triggered by specific inputs typically indicate unhandled edge cases in the problem's Perl code, such as division by zero or invalid domain values.
Question 3: Which WeBWorK feature allows instructors to test a problem with randomized seeds before publishing it to students?
- Problem Editor preview (Correct answer)
- Library Browser search
- Hardcopy generator
- Achievement editor
Correct answer: Problem Editor preview
The Problem Editor's preview feature lets instructors render a problem with any seed to check randomized variants before assigning them.
Question 4: An instructor notices students consistently answer a true/false WeBWorK problem correctly on the first try at a 95% rate. What QA concern does this raise?
- The problem is too difficult
- The problem may be too easy or non-discriminating (Correct answer)
- Answer checking is broken
- The due date is too far in the future
Correct answer: The problem may be too easy or non-discriminating
A near-perfect first-attempt rate suggests the problem may be trivial or guessable, making it a poor discriminator of student understanding.
Question 5: What is the purpose of the 'showMeAnother' feature in WeBWorK from a QA perspective?
- To display the correct answer immediately
- To regenerate a problem with a new random seed for practice (Correct answer)
- To email the instructor a copy of the problem
- To extend the due date automatically
Correct answer: To regenerate a problem with a new random seed for practice
ShowMeAnother regenerates the problem with a new seed so students can practice the same concept with different values, which also helps QA teams verify multiple randomization variants.
Question 6: Which approach is most effective for QA-testing WeBWorK answer checkers that evaluate mathematical expressions?
- Testing only the canonical correct answer
- Testing algebraically equivalent forms of the correct answer (Correct answer)
- Checking that any numeric input is accepted
- Verifying only wrong answers are rejected
Correct answer: Testing algebraically equivalent forms of the correct answer
Effective QA for expression-based answers requires verifying that algebraically equivalent forms (e.g., x+1 and 1+x) are all accepted as correct.
Question 7: In WeBWorK, a problem set's 'reduced scoring period' is a QA-relevant feature because it:
- Prevents students from submitting after the due date entirely
- Encourages early completion and discourages last-minute mass submissions that stress servers (Correct answer)
- Randomly selects which students receive partial credit
- Automatically emails warnings to low-performing students
Correct answer: Encourages early completion and discourages last-minute mass submissions that stress servers
Reduced scoring incentivizes earlier submission, distributing server load and reducing the risk of performance degradation during peak submission times.
WeBWork Quality Assurance and Improvement 2
In WeBWorK quality assurance, which metric best measures whether a problem set's difficulty is appropriately calibrated?