CIA User Research and Analysis 3 — Questions and Answers
Question 1: A researcher notices that participants behave differently when they know they are being observed during usability testing. This phenomenon is known as:
- Confirmation bias
- The Hawthorne effect (Correct answer)
- Social desirability bias
- Observer effect
Correct answer: The Hawthorne effect
The Hawthorne effect describes how people modify their behavior when aware of being watched, potentially skewing research results.
Question 2: Which sampling method is most appropriate when the goal is to ensure all key user segments (novice, intermediate, expert) are represented in a small usability study?
- Random sampling
- Stratified sampling (Correct answer)
- Convenience sampling
- Snowball sampling
Correct answer: Stratified sampling
Stratified sampling divides the population into subgroups and samples from each, ensuring all key segments are represented.
Question 3: A persona that is based entirely on team assumptions without supporting research data is called a:
- Proto-persona (Correct answer)
- Primary persona
- Secondary persona
- Anti-persona
Correct answer: Proto-persona
Proto-personas (or provisional personas) are assumption-based placeholders created before research data is available.
Question 4: In tree testing, a task is considered successfully completed when:
- The user reaches any page within the correct section
- The user selects the correct destination node (Correct answer)
- The user navigates back fewer than two times
- The user completes the task in under 30 seconds
Correct answer: The user selects the correct destination node
Tree testing measures whether users correctly identify the destination node in the hierarchy for a given task.
Question 5: Which research method is best suited for exploring why users abandon a multi-step online form?
- Benchmark usability testing
- Analytics funnel analysis combined with exit interviews (Correct answer)
- Remote unmoderated testing
- Competitive analysis
Correct answer: Analytics funnel analysis combined with exit interviews
Combining funnel analytics to locate the drop-off point with exit interviews to understand the 'why' gives the most complete picture.
Question 6: A UX researcher wants to measure how efficiently users complete tasks over multiple sessions as they gain experience. This is a measure of:
- Learnability (Correct answer)
- Memorability
- Satisfaction
- Error rate
Correct answer: Learnability
Learnability measures how quickly users improve their efficiency and effectiveness with a system over repeated use.
Question 7: When conducting remote unmoderated usability testing, which is the most significant limitation compared to moderated sessions?
- Smaller participant pool available
- Inability to probe or follow up on unexpected behavior (Correct answer)
- Higher cost per participant
- Lower ecological validity
Correct answer: Inability to probe or follow up on unexpected behavior
Without a moderator present, researchers cannot ask follow-up questions when participants do something unexpected or confusing.
CIA User Research and Analysis 3
A researcher notices that participants behave differently when they know they are being observed during usability testing.
This phenomenon is known as: