User Experience Design Research & Evidence-Based Practice 4 â Questions and Answers
Question 1: A product team has limited budget and must choose between a tree test and a first-click test. The primary question is whether users can find a specific feature. Which method is more appropriate?
- Tree test
- First-click test (Correct answer)
- Eye-tracking study
- Guerrilla usability test
Correct answer: First-click test
A first-click test specifically measures whether users' initial click is correct, which directly answers navigation findability questions.
Question 2: What does 'satisficing' mean in the context of UX research and user behavior?
- Measuring user satisfaction with a 5-point scale
- Users accepting a 'good enough' solution rather than searching for the optimal one (Correct answer)
- Designing interfaces that satisfy all usability heuristics
- Combining satisfaction and efficiency metrics
Correct answer: Users accepting a 'good enough' solution rather than searching for the optimal one
Satisficing, coined by Herbert Simon, describes users stopping their search once they find an option that meets a minimum threshold rather than seeking the best option.
Question 3: A researcher wants to evaluate three design concepts with the same participants across three sessions. What threat to validity must they control for?
- Construct validity
- Order effects / learning bias (Correct answer)
- Ecological validity
- Inter-rater reliability
Correct answer: Order effects / learning bias
When the same participants evaluate multiple designs, order effects can bias results â counterbalancing the presentation order mitigates this.
Question 4: Which statistical measure is most useful for reporting the central tendency of task completion times when the data contains extreme outliers?
- Mean
- Median (Correct answer)
- Mode
- Standard deviation
Correct answer: Median
The median is resistant to outliers and better represents typical task time when a few extreme values would skew the mean.
Question 5: During moderated remote testing, a participant shares their screen but their audio cuts out. The best facilitator response is to:
- End the session and reschedule
- Ask the participant to type their thoughts in the chat or switch to text-based think-aloud (Correct answer)
- Continue and rely solely on screen recording
- Ask the observer to infer the participant's thoughts
Correct answer: Ask the participant to type their thoughts in the chat or switch to text-based think-aloud
Switching to a text-based think-aloud maintains data collection when audio fails, preserving the session's value.
Question 6: What is the key difference between formative and summative usability evaluation?
- Formative uses quantitative data; summative uses qualitative data
- Formative informs design improvements during development; summative measures final product performance (Correct answer)
- Formative is done remotely; summative is done in a lab
- There is no meaningful difference between the two
Correct answer: Formative informs design improvements during development; summative measures final product performance
Formative evaluation is diagnostic and iterative, shaping design during development, while summative evaluation assesses the finished product against benchmarks.
Question 7: A researcher uses multiple data sourcesâinterviews, analytics logs, and field observationsâto validate a finding. This practice is known as:
- Saturation
- Triangulation (Correct answer)
- Affinity diagramming
- Rapid iterative testing and evaluation (RITE)
Correct answer: Triangulation
Triangulation increases confidence in findings by cross-validating them across multiple independent data sources or methods.
User Experience Design Research & Evidence-Based Practice 4
A product team has limited budget and must choose between a tree test and a first-click test.
The primary question is whether users can find a specific feature.
Which method is more appropriate?