CPM Email Segmentation & A/B Testing 2 — Questions and Answers
Question 1: A marketer wants to segment an email list based on how recently subscribers have made a purchase. Which segmentation model best supports this?
- Psychographic segmentation
- RFM (Recency, Frequency, Monetary) segmentation (Correct answer)
- Geographic segmentation
- Firmographic segmentation
Correct answer: RFM (Recency, Frequency, Monetary) segmentation
RFM segmentation classifies customers by how recently, how often, and how much they purchase, making it ideal for recency-based targeting.
Question 2: When A/B testing email subject lines, what is the most important principle for maintaining test validity?
- Test multiple variables simultaneously to save time
- Send variant A in the morning and variant B in the afternoon
- Randomly split the audience and change only one variable (Correct answer)
- Use the smaller segment to test the riskier subject line
Correct answer: Randomly split the audience and change only one variable
Valid A/B tests require a random split and isolation of a single variable so results can be attributed to that change alone.
Question 3: A retail brand segments subscribers by past purchase category (shoes, apparel, accessories). This is an example of which segmentation type?
- Demographic segmentation
- Behavioral segmentation (Correct answer)
- Psychographic segmentation
- Lifecycle segmentation
Correct answer: Behavioral segmentation
Behavioral segmentation groups customers by their past actions, such as product categories they have purchased.
Question 4: An A/B test on email CTA button color yields a p-value of 0.08. At a 95% confidence threshold, what should the marketer conclude?
- The result is statistically significant and the winner should be deployed
- The result is not statistically significant and more data is needed (Correct answer)
- The red button always outperforms blue, so deploy it anyway
- A p-value above 0.05 means the test was run incorrectly
Correct answer: The result is not statistically significant and more data is needed
A p-value of 0.08 exceeds the 0.05 threshold for 95% confidence, meaning the result is not statistically significant.
Question 5: Which metric is MOST appropriate to use as the primary success metric when A/B testing email send time?
- Unsubscribe rate
- Spam complaint rate
- Open rate (Correct answer)
- List growth rate
Correct answer: Open rate
Open rate directly measures whether recipients are responding to the email at the time it arrives, making it the most relevant metric for send-time tests.
Question 6: A SaaS company segments its email list into free-trial users, paying subscribers, and churned customers. This is an example of:
- Geographic segmentation
- Firmographic segmentation
- Customer lifecycle segmentation (Correct answer)
- Engagement-based segmentation
Correct answer: Customer lifecycle segmentation
Lifecycle segmentation groups contacts by where they are in the customer journey, such as trial, active, or churned status.
Question 7: What is 'winner takes all' deployment in the context of A/B testing?
- Sending the test to the entire list at once without a holdout
- Automatically deploying the winning variant to the remaining audience after the test period (Correct answer)
- Running the test indefinitely until one variant reaches 100% open rate
- Giving the marketing director final say over which variant wins
Correct answer: Automatically deploying the winning variant to the remaining audience after the test period
Winner-takes-all deployment means the statistically superior variant is automatically sent to the remainder of the list once the test concludes.
CPM Email Segmentation & A/B Testing 2
A marketer wants to segment an email list based on how recently subscribers have made a purchase.
Which segmentation model best supports this?