Icon Design Research & Evidence-Based Practice 4 — Questions and Answers
Question 1: A designer receives contradictory results from a survey (positive ratings) and a task-based test (poor performance). Which data source should be prioritized?
- Survey data, because larger samples are more reliable
- Task-based test data, because behavioral evidence outweighs self-report (Correct answer)
- Average both equally
- Discard both and repeat
Correct answer: Task-based test data, because behavioral evidence outweighs self-report
Behavioral performance data is more objective than self-reported attitudes, which are subject to social desirability bias and inaccurate recall.
Question 2: When testing icon comprehension across cultures, which research design ensures cultural factors are isolated as the variable?
- Within-subjects design with the same participants tested in two languages
- Between-subjects design with matched samples from each target culture (Correct answer)
- Longitudinal design tracking the same users over time
- Case study of one international user
Correct answer: Between-subjects design with matched samples from each target culture
A between-subjects design with demographically matched groups isolates culture as the independent variable while controlling for individual differences.
Question 3: Which icon design heuristic is directly supported by research showing that labels increase recognition rates by over 30%?
- Icons should never use text
- Pairing icons with text labels significantly improves usability (Correct answer)
- Labels should only appear on hover
- Icon size is more important than labeling
Correct answer: Pairing icons with text labels significantly improves usability
Research consistently shows that icon-label pairings outperform standalone icons in recognition and task completion speed.
Question 4: A product team wants to track whether a redesigned icon set improves over time after release. What research approach is most appropriate?
- Retrospective interview
- Longitudinal usability tracking with periodic measurements (Correct answer)
- Single-session benchmark test
- Diary study of one user
Correct answer: Longitudinal usability tracking with periodic measurements
Longitudinal tracking captures how icon performance and user familiarity evolve over time after deployment.
Question 5: What does a confidence interval communicate in icon recognition study results?
- How confident the designer feels about the icon
- The range within which the true population parameter likely falls (Correct answer)
- The percentage of users who liked the icon
- The exact number of errors made
Correct answer: The range within which the true population parameter likely falls
A confidence interval provides a range of plausible values for the true population parameter, conveying the precision of the study's estimate.
Question 6: In a card-sorting study for icon taxonomy, participants group icon concepts rather than physical cards. What does this method primarily inform?
- Color preferences for the icon set
- The mental models users have for categorizing icon functions (Correct answer)
- Technical file format decisions
- Animation timing preferences
Correct answer: The mental models users have for categorizing icon functions
Card sorting reveals users' natural categorization logic, which should align with the icon set's organizational structure and naming.
Question 7: A researcher reports that a new icon set has p = 0.03 for improved recognition over the baseline. What does this mean?
- 3% of users preferred the new icons
- There is a 3% probability of observing this result if there were no real effect (Correct answer)
- The icons are 3% more colorful
- The study had a 3% response rate
Correct answer: There is a 3% probability of observing this result if there were no real effect
A p-value of 0.03 means there is only a 3% chance of obtaining this result by random variation if the null hypothesis (no effect) were true.
Icon Design Research & Evidence-Based Practice 4
A designer receives contradictory results from a survey (positive ratings) and a task-based test (poor performance).
Which data source should be prioritized?