Mobile App Design Research & Evidence-Based Practice 2 — Questions and Answers
Question 1: Which research method is best suited for understanding the emotional journey a user experiences when completing a complex task in a mobile app?
- Experience sampling method (Correct answer)
- Heuristic evaluation
- A/B testing
- Card sorting
Correct answer: Experience sampling method
Experience sampling method captures in-the-moment emotional states and experiences as users complete tasks in their natural context.
Question 2: A mobile app team wants to validate that their navigation redesign reduces task completion time. Which metric should be the PRIMARY outcome measure?
- Time on task (Correct answer)
- App store rating
- Daily active users
- Session length
Correct answer: Time on task
Time on task directly measures how quickly users can complete specific tasks, making it the primary metric for evaluating navigation efficiency.
Question 3: What does a high 'rage tap' rate on a mobile element typically indicate in analytics data?
- Users expect the element to be interactive but it is not responding (Correct answer)
- The app has a memory leak
- Users are highly engaged with the content
- The element loads too slowly
Correct answer: Users expect the element to be interactive but it is not responding
Rage taps — repeated rapid taps — indicate user frustration when an element appears tappable but fails to respond or respond quickly enough.
Question 4: In a moderated usability test for a mobile app, when should a facilitator intervene during a participant's session?
- Only when the participant is in genuine distress or asks a direct question about the task (Correct answer)
- Whenever the participant makes a wrong turn
- After every completed task to clarify observations
- At the 10-minute mark to check on progress
Correct answer: Only when the participant is in genuine distress or asks a direct question about the task
Facilitators should intervene minimally — only for safety, genuine distress, or direct questions — to avoid biasing participant behavior and preserve ecological validity.
Question 5: Which statistical concept explains why running 20 A/B tests simultaneously increases the likelihood of finding a 'significant' result by chance?
- Multiple comparisons problem (Correct answer)
- Selection bias
- Regression to the mean
- Survivorship bias
Correct answer: Multiple comparisons problem
The multiple comparisons problem (or p-hacking) means that running many simultaneous tests inflates the false positive rate well beyond the nominal significance level.
Question 6: A diary study for a mobile banking app runs for two weeks. What is the PRIMARY advantage over a single lab usability session?
- It captures behavioral patterns and context across varied real-world situations over time (Correct answer)
- It requires fewer participants to reach statistical significance
- It eliminates observer effect entirely
- It produces quantitative data faster
Correct answer: It captures behavioral patterns and context across varied real-world situations over time
Diary studies reveal longitudinal patterns, habit formation, and situational variability that a single lab session cannot capture.
Question 7: When evaluating competing mobile app design concepts, which technique allows researchers to identify which specific features users value most without directly asking them?
- Conjoint analysis (Correct answer)
- Semantic differential scale
- Think-aloud protocol
- Net Promoter Score
Correct answer: Conjoint analysis
Conjoint analysis presents trade-off scenarios that reveal underlying feature preferences indirectly through forced choices rather than self-reported priorities.
Mobile App Design Research & Evidence-Based Practice 2
Which research method is best suited for understanding the emotional journey a user experiences when completing a complex task in a mobile app?