GMC Data Analysis & Decision Making 3 — Questions and Answers
Question 1: You run two simultaneous A/B tests on the same landing page. What is the primary risk of this approach?
- Increased server load
- Interaction effects between experiments that contaminate results (Correct answer)
- Insufficient statistical power
- Higher cost per click
Correct answer: Interaction effects between experiments that contaminate results
Simultaneous tests on the same page can interact, making it impossible to attribute performance changes to a single experiment.
Question 2: Which of the following is an example of a vanity metric in growth marketing?
- Revenue per user
- Total registered users without accounting for active users (Correct answer)
- 30-day retention rate
- Customer acquisition cost
Correct answer: Total registered users without accounting for active users
Total registered users looks impressive but ignores engagement and activity, making it a vanity metric that doesn't predict business health.
Question 3: A company's DAU/MAU ratio is 0.15. What does this indicate?
- 15% of monthly users engage daily, suggesting low stickiness (Correct answer)
- The product has 15 times more monthly users than daily users
- Daily revenue is 15% of monthly revenue
- 15% of users have churned in the past month
Correct answer: 15% of monthly users engage daily, suggesting low stickiness
A DAU/MAU ratio of 0.15 means only 15% of monthly active users return daily, indicating low product stickiness.
Question 4: When should a growth team use qualitative data alongside quantitative analytics?
- Only when quantitative data is unavailable
- When they need to understand the 'why' behind the numbers (Correct answer)
- Qualitative data is never appropriate for growth decisions
- Only during early-stage product development
Correct answer: When they need to understand the 'why' behind the numbers
Qualitative data (interviews, session recordings) explains user motivations and context that quantitative metrics alone cannot reveal.
Question 5: A growth experiment shows a 5% lift in revenue but a 3% decline in NPS. How should the team proceed?
- Ship the change immediately since revenue increased
- Discard the change since NPS declined
- Evaluate both metrics against long-term LTV and retention projections before deciding (Correct answer)
- Run the test for another week
Correct answer: Evaluate both metrics against long-term LTV and retention projections before deciding
Both revenue lift and NPS decline have long-term implications; the decision requires weighing trade-offs against retention and LTV data.
Question 6: What is 'sample ratio mismatch' (SRM) in A/B testing?
- When the test group is larger than intended due to a bug in traffic allocation (Correct answer)
- When the sample size is too small to detect a meaningful effect
- When test and control groups have different conversion goals
- When two audiences overlap in targeting
Correct answer: When the test group is larger than intended due to a bug in traffic allocation
SRM occurs when the observed traffic split between variants doesn't match the intended split, indicating a technical bug that invalidates results.
Question 7: Which segmentation dimension is most useful for identifying high-value users to inform a referral program?
- Device type
- Acquisition channel combined with LTV tier (Correct answer)
- Browser preference
- Time zone
Correct answer: Acquisition channel combined with LTV tier
Combining acquisition channel with LTV tier identifies which sources bring the most valuable users, allowing the referral program to target and replicate that pattern.
GMC Data Analysis & Decision Making 3
You run two simultaneous A/B tests on the same landing page.
What is the primary risk of this approach?