Data Analysis & Interpretation Techniques Flashcards

An A/B test reaches 95% significance after two days, before the planned sample size is collected. What is the best practice?

Answer: Continue until the predetermined sample size or duration is met

Stopping early on a significance peek inflates false positives; pre-set sample size avoids this.

Which situation best justifies using a median instead of a mean for order value?

Answer: A few very large wholesale orders skew the average

The median is robust to extreme values and better represents the typical order in skewed data.

Traffic from a channel increases while its conversions stay flat. Which interpretation is most reasonable?

Answer: The channel is now bringing lower-intent visitors

More traffic without more conversions often means lower quality or less relevant visitors, though tracking should be checked too.

What does a data sampling warning in an analytics report mean for interpretation?

Answer: Results are based on a subset of data and may be less precise

Sampling applies calculations to a subset of sessions, which can introduce estimation error.

Which is an example of a leading indicator for e-commerce revenue?

Answer: Add-to-cart rate

Add-to-cart rate tends to change before purchases and revenue, signaling future performance.

Why should analysts apply a consistent date range when comparing two segments?

Answer: To avoid time-based differences confounding the comparison

Different periods introduce seasonality and event effects that distort a like-for-like comparison.

A high average session duration but a very low conversion rate on a content page most likely suggests what?

Answer: Visitors engage with the content but are not being guided toward a conversion action

Engagement and conversion are separate; strong engagement without conversion points to weak calls to action or intent mismatch.