Website Performance Optimization & Reporting Flashcards

Which KPI best measures how well a landing page turns visits into desired outcomes?

Answer: Conversion rate

Conversion rate is conversions divided by visits and directly measures goal achievement.

An A/B test shows variant B with a 4% lift but p = 0.38. What is the correct interpretation?

Answer: The result is not statistically significant

A p-value of 0.38 is far above common thresholds like 0.05, so the lift could easily be random noise.

Why should an A/B test generally run for at least one or two full business cycles?

Answer: To account for weekday and weekend behavior differences

Full weekly cycles smooth out day-of-week variation that could bias results.

A report shows overall conversion rate flat, but mobile is down while desktop is up. This is an example of what?

Answer: Simpson's-paradox-style masking that segmentation reveals

Opposing segment trends can cancel out in aggregate, so segmenting exposes the hidden change.

Which metric is most useful to find the step where users abandon a checkout process?

Answer: Funnel visualization drop-off rate

Funnel reports show the percentage lost at each step, pinpointing the abandonment point.

What is a key risk of 'peeking' at A/B test results and stopping as soon as significance appears?

Answer: Inflated false-positive rate

Repeatedly checking and stopping early inflates the chance of declaring a false winner.

Which change is a valid way to improve a form's completion rate based on analytics evidence?

Answer: Remove fields where users most often abandon

Field-level abandonment data suggests removing or simplifying friction-heavy fields.