CWA CWA Conversion Rate Optimization & A/B Testing 2

Question 1

What is a 'hypothesis' in A/B testing?

Accepted Answer

A testable prediction about how a change will affect user behavior

Answer

An A/B test hypothesis is a structured, testable prediction stating that a specific change will produce a measurable improvement in a defined metric.

Question 2

What is 'multivariate testing' (MVT) in CRO?

Accepted Answer

Testing multiple changes across multiple page elements at the same time

Answer

Multivariate testing simultaneously tests multiple variations of multiple page elements to identify which combination produces the best results.

Question 3

What is the 'control' in an A/B test?

Accepted Answer

The original, unchanged version of the page

Answer

The control is the existing, unmodified version of the page that serves as the baseline against which the new variant is compared.

Question 4

What does a 'p-value' represent in A/B testing?

Accepted Answer

The probability of seeing results as extreme as observed if there is no real difference between variants

Answer

A p-value measures the probability that the observed test results occurred by chance; a low p-value (typically <0.05) suggests statistical significance.

Question 5

Which is a best practice when running A/B tests?

Accepted Answer

Test one element at a time to isolate the variable's effect

Answer

Testing one element at a time ensures you can attribute any performance difference directly to that specific change without confounding variables.

CWA - Certified Web Analyst Practice Test

CWA - Certified Web Analyst Practice Test

CWA CWA Conversion Rate Optimization & A/B Testing 2