User Experience Design Quality Control & Assurance 2 — Questions and Answers
Question 1: During a heuristic evaluation of a mobile app, an evaluator finds that error messages say 'Error 404' with no further context. Which Nielsen heuristic does this violate?
- Visibility of system status
- Help users recognize, diagnose, and recover from errors (Correct answer)
- Consistency and standards
- Aesthetic and minimalist design
Correct answer: Help users recognize, diagnose, and recover from errors
Nielsen's 'Help users recognize, diagnose, and recover from errors' heuristic requires error messages to be expressed in plain language and suggest a solution.
Question 2: A QA team uses the System Usability Scale (SUS) and obtains a score of 51. How should this be interpreted?
- Above average usability
- Marginal usability that likely needs improvement (Correct answer)
- Excellent usability
- The score is too low to be meaningful
Correct answer: Marginal usability that likely needs improvement
A SUS score of 51 falls below the average benchmark of 68, indicating marginal usability that warrants investigation and improvement.
Question 3: What is the primary purpose of conducting a cognitive walkthrough in UX QA?
- To measure task completion time
- To evaluate learnability by simulating a new user's thought process (Correct answer)
- To identify visual design inconsistencies
- To benchmark performance against competitors
Correct answer: To evaluate learnability by simulating a new user's thought process
A cognitive walkthrough evaluates learnability by having evaluators step through tasks as a first-time user would, identifying points of confusion.
Question 4: In UX quality assurance, what does 'regression testing' specifically refer to?
- Testing older user segments to see if they find the interface harder over time
- Verifying that new changes have not broken previously working features or flows (Correct answer)
- Rolling back a design to an earlier version after negative feedback
- Testing the design on outdated browsers only
Correct answer: Verifying that new changes have not broken previously working features or flows
Regression testing ensures that updates or new features have not introduced defects in parts of the product that were previously functioning correctly.
Question 5: A UX team wants to assess whether their checkout flow meets WCAG 2.1 Level AA standards. Which tool category is MOST appropriate for an initial automated check?
- Eye-tracking software
- Axe or WAVE accessibility checkers (Correct answer)
- A/B testing platforms
- Card sorting tools
Correct answer: Axe or WAVE accessibility checkers
Automated accessibility checkers like Axe or WAVE scan pages for WCAG violations such as missing alt text, low contrast, and missing form labels.
Question 6: What is the key distinction between formative and summative usability evaluation?
- Formative uses quantitative data; summative uses qualitative data
- Formative is done during design to inform improvements; summative judges the final product against benchmarks (Correct answer)
- Formative involves external users; summative uses internal team members
- Formative tests accessibility; summative tests performance
Correct answer: Formative is done during design to inform improvements; summative judges the final product against benchmarks
Formative evaluation is diagnostic and iterative (done during design), while summative evaluation measures the finished product against predefined quality benchmarks.
Question 7: A product team discovers that 3 out of 5 usability test participants cannot complete the account registration task. According to standard UX QA practice, what should be the immediate next step?
- Increase the sample size to 20 participants before acting
- Prioritize the registration flow for redesign and root-cause analysis (Correct answer)
- Archive the finding and revisit it in the next annual review
- Switch to a different testing method such as surveys
Correct answer: Prioritize the registration flow for redesign and root-cause analysis
A 60% failure rate on a critical task is a severe usability issue that warrants immediate prioritization for root-cause analysis and redesign.
User Experience Design Quality Control & Assurance 2
During a heuristic evaluation of a mobile app, an evaluator finds that error messages say 'Error 404' with no further context.
Which Nielsen heuristic does this violate?