User Testing Exam User Testing Analysis and Synthesis 2 — Questions and Answers
Question 1: What is the primary purpose of a 'debrief session' after usability testing?
- To give participants extended time to complete unfinished tasks
- For the research team to share observations, identify patterns, and align on key findings before analysis (Correct answer)
- To notify participants of compensation disbursement
- To present final recommendations to the product team
Correct answer: For the research team to share observations, identify patterns, and align on key findings before analysis
A post-session debrief while memory is fresh helps the team capture impressions, identify emerging themes, and align before formal analysis begins.
Question 2: What does 'inter-rater reliability' mean in qualitative usability research?
- The consistency of task completion rates across different participant groups
- The degree to which two or more analysts independently arrive at the same codes or conclusions (Correct answer)
- The reliability of session recording software across different devices
- The agreement between participant self-reported ratings and observed behavior
Correct answer: The degree to which two or more analysts independently arrive at the same codes or conclusions
High inter-rater reliability indicates that findings are not idiosyncratic to one analyst but are consistently identifiable by independent reviewers.
Question 3: What is 'behavioral data' in the context of usability testing?
- Self-reported attitudes collected through post-test questionnaires
- Directly observed actions participants take while using the product (Correct answer)
- Participant demographic information collected during screening
- Eye-tracking heat maps generated after the session
Correct answer: Directly observed actions participants take while using the product
Behavioral data captures what users actually do—clicks, navigation paths, errors, hesitations—as opposed to what they say they do.
Question 4: What does the 'say-do gap' refer to in user research analysis?
- The delay between a participant speaking and the moderator responding
- The discrepancy between what participants say they do and what is actually observed (Correct answer)
- The gap in data caused by participants who go silent during think-aloud
- The difference between pre-test and post-test satisfaction scores
Correct answer: The discrepancy between what participants say they do and what is actually observed
The say-do gap highlights that self-reported behavior often differs from observed behavior, making direct observation essential alongside surveys.
Question 5: What is 'top task analysis' as an input to usability test design?
- Ranking interface tasks by visual hierarchy on the screen
- Identifying the small set of tasks users perform most frequently that should be prioritized in testing (Correct answer)
- Analyzing the most error-prone tasks found in a previous study
- Evaluating tasks based on how much development effort they require
Correct answer: Identifying the small set of tasks users perform most frequently that should be prioritized in testing
Top task analysis uses surveys or analytics to identify the ~20% of tasks users perform ~80% of the time, ensuring testing effort targets what matters most.
Question 6: What is a 'customer journey map' and how does it relate to usability testing?
- A map of geographic locations where users test the product
- A visualization of user steps, emotions, and touchpoints that helps scope usability test tasks (Correct answer)
- A technical diagram of server requests during a user session
- A chart showing how many users visited each page before testing
Correct answer: A visualization of user steps, emotions, and touchpoints that helps scope usability test tasks
Journey maps outline user stages, goals, and pain points across a product experience, helping researchers identify which stages most need usability investigation.
User Testing Exam User Testing Analysis and Synthesis 2
What is the primary purpose of a 'debrief session' after usability testing?