Web Traffic Research & Evidence-Based Practice 5 — Questions and Answers
Question 1: A traffic analyst wants to compare the effectiveness of three different content strategies across 12 months. Which statistical test is most appropriate for this repeated-measures design?
- Independent samples t-test
- Repeated-measures ANOVA with Greenhouse-Geisser correction (Correct answer)
- Mann-Whitney U test
- Pearson correlation
Correct answer: Repeated-measures ANOVA with Greenhouse-Geisser correction
Repeated-measures ANOVA accounts for the within-subject correlation across time points, and the Greenhouse-Geisser correction addresses violations of the sphericity assumption.
Question 2: Which practice best ensures reproducibility when publishing a web traffic analysis based on Google Search Console data?
- Sharing the raw exported CSV files and all analysis scripts publicly (Correct answer)
- Describing the methodology in one paragraph in the paper
- Using proprietary software only available to the research team
- Rounding all figures to the nearest thousand
Correct answer: Sharing the raw exported CSV files and all analysis scripts publicly
Sharing raw data exports and analysis scripts enables other researchers to exactly replicate every step of the analysis, the gold standard for reproducibility.
Question 3: An analyst is building a causal model of paid search traffic using directed acyclic graphs (DAGs). What is the primary purpose of identifying collider variables in this context?
- To increase the number of control variables in the model
- To avoid conditioning on variables that open non-causal backdoor paths (Correct answer)
- To maximize R-squared of the regression
- To identify which variables to include in the ad budget
Correct answer: To avoid conditioning on variables that open non-causal backdoor paths
Conditioning on a collider variable in a DAG opens a spurious association between its parents, introducing bias into the causal estimate of paid traffic effects.
Question 4: A 12-month traffic dataset shows a strong upward trend for both newsletter subscribers and organic visits. Before concluding that subscribers drive organic traffic, which test should be applied?
- Granger causality test after differencing to achieve stationarity (Correct answer)
- Pearson correlation of raw series
- Simple linear regression with month as the predictor
- Cross-tabulation of weekly visits
Correct answer: Granger causality test after differencing to achieve stationarity
Granger causality tests whether past values of subscribers improve predictions of future organic visits beyond the visit series alone, after removing non-stationary trends.
Question 5: In evidence-based web optimization, which hierarchy of evidence should guide decisions about implementing a new site architecture?
- Expert opinion → case studies → experiments → meta-analyses
- Meta-analyses → systematic reviews → randomized experiments → observational studies → expert opinion (Correct answer)
- Observational studies → expert opinion → randomized experiments
- Case studies → randomized experiments → expert opinion → meta-analyses
Correct answer: Meta-analyses → systematic reviews → randomized experiments → observational studies → expert opinion
The standard evidence hierarchy places meta-analyses and systematic reviews at the top, followed by controlled experiments, then observational studies, and expert opinion at the bottom.
Question 6: A researcher notices that high-traffic pages in their dataset have disproportionately more backlinks. Which sampling strategy should be used to avoid overrepresenting popular pages in a study of backlink impact?
- Probability proportional to size sampling
- Random sampling stratified by traffic quintile (Correct answer)
- Convenience sampling of the top 100 pages
- Systematic sampling by alphabetical URL order
Correct answer: Random sampling stratified by traffic quintile
Stratified sampling by traffic quintile ensures equal representation across all traffic levels, preventing high-traffic outliers from dominating the analysis.
Question 7: Which form of bias occurs when an analyst only investigates traffic campaigns that were deemed successful by the marketing team, ignoring failed ones?
- Attrition bias
- Survivorship bias (Correct answer)
- Information bias
- Lead-time bias
Correct answer: Survivorship bias
Survivorship bias occurs when only successful cases are studied, leading to overestimation of strategy effectiveness because failures are systematically excluded.
Web Traffic Research & Evidence-Based Practice 5
A traffic analyst wants to compare the effectiveness of three different content strategies across 12 months.
Which statistical test is most appropriate for this repeated-measures design?