Mobile App Design Research & Evidence-Based Practice 3 — Questions and Answers
Question 1: A researcher notices that participants in a mobile usability study perform significantly better in the second half of the session than the first. This is most likely due to which effect?
- Learning effect (Correct answer)
- Hawthorne effect
- Primacy effect
- Confirmation bias
Correct answer: Learning effect
The learning effect occurs when participants improve performance over time simply by becoming familiar with the interface during the study itself.
Question 2: Which of the following best describes the purpose of a 'cognitive walkthrough' in mobile app design research?
- Evaluating whether a design supports user goal achievement step-by-step without formal user testing (Correct answer)
- Mapping neural pathways activated during mobile app use
- Recording eye movement patterns across the screen
- Testing app performance under heavy server load
Correct answer: Evaluating whether a design supports user goal achievement step-by-step without formal user testing
A cognitive walkthrough is an expert inspection method that evaluates each step of a task to determine if users can understand and act on the interface without prior training.
Question 3: In mobile UX research, what is the key distinction between formative and summative evaluation?
- Formative evaluation improves design during development; summative evaluates final product quality (Correct answer)
- Formative uses qualitative data; summative uses quantitative data exclusively
- Formative is done by users; summative is done by experts
- Formative measures satisfaction; summative measures task completion
Correct answer: Formative evaluation improves design during development; summative evaluates final product quality
Formative evaluation occurs during development to guide design decisions, while summative evaluation benchmarks the completed product against defined standards or competitors.
Question 4: A mobile app researcher wants to understand why users abandon checkout flows. Which combination of research methods would yield the most complete picture?
- Funnel analytics to find drop-off points, then qualitative interviews to understand reasons (Correct answer)
- Survey alone sent to all registered users
- Heuristic evaluation by three experts
- A/B test of two checkout layouts
Correct answer: Funnel analytics to find drop-off points, then qualitative interviews to understand reasons
Combining quantitative analytics to identify where abandonment occurs with qualitative interviews to explain why provides both the scope and the context needed for actionable insights.
Question 5: What is the primary risk of relying solely on app store reviews as a source of user research data?
- Reviews over-represent extreme experiences and are not a representative sample of the user base (Correct answer)
- Reviews are too technical for designers to interpret
- Review data cannot be exported for analysis
- App store reviews are removed after 90 days
Correct answer: Reviews over-represent extreme experiences and are not a representative sample of the user base
App store reviews suffer from self-selection bias — users with very negative or very positive experiences are far more likely to leave reviews than average users.
Question 6: Which accessibility research practice ensures that mobile app designs are evaluated for users with a range of abilities, not just average users?
- Inclusive usability testing that recruits participants with diverse disabilities (Correct answer)
- Automated accessibility scanning only
- Expert review using WCAG guidelines exclusively
- Color contrast ratio checks
Correct answer: Inclusive usability testing that recruits participants with diverse disabilities
Inclusive usability testing with participants who have visual, motor, cognitive, and other disabilities reveals real barriers that automated tools and expert reviews alone cannot detect.
Question 7: A UX researcher reports that 'users said they loved the new onboarding.' Why should this finding be treated with caution before acting on it?
- Self-reported satisfaction often diverges from actual behavior, and social desirability bias may inflate positive responses (Correct answer)
- Users are never reliable sources of design feedback
- Love is not a measurable UX metric
- Onboarding research always produces positive results
Correct answer: Self-reported satisfaction often diverges from actual behavior, and social desirability bias may inflate positive responses
Social desirability bias causes participants to give positive feedback to please the researcher, while behavioral data — like completion rates — may tell a different story.
Mobile App Design Research & Evidence-Based Practice 3
A researcher notices that participants in a mobile usability study perform significantly better in the second half of the session than the first.
This is most likely due to which effect?