Risk Assessment & Mitigation Flashcards

A brand launches a new ad campaign and sees a spike in direct traffic the following week. The analytics team attributes all direct traffic growth to brand awareness. What risk does this interpretation carry?

Answer: Confirmation bias leading to over-crediting the campaign without ruling out alternative causes

Attributing all direct traffic growth to the campaign without controlling for seasonality or other concurrent activities is confirmation bias.

Which approach best mitigates the risk of campaign decisions being made on statistically insignificant A/B test results?

Answer: Pre-specifying minimum detectable effect size and required sample size before launching the test

Pre-specifying MDE and sample size ensures the test is adequately powered before launch, preventing premature conclusions from underpowered results.

A media mix model (MMM) trained on three years of data is used to allocate next year's budget. Model performance degrades significantly in the second quarter. Which risk is MOST likely responsible?

Answer: Model staleness due to changed market dynamics post-training period

MMMs trained on historical data degrade when market conditions, competitive activity, or consumer behavior shift, a form of model drift.

An analyst uses the same dataset to both build and evaluate a predictive churn model. What data risk does this create?

Answer: Data leakage causing overly optimistic performance estimates

Evaluating a model on its training data inflates performance metrics because the model has already 'seen' those examples.

A marketing team relies on third-party cookie data for audience segmentation. Which strategic risk is MOST critical to address given current browser privacy changes?

Answer: Dependency on a data source that is being deprecated, threatening segmentation accuracy

Third-party cookie deprecation directly eliminates the data source the segmentation relies on, requiring migration to first-party or cookieless alternatives.

Which scenario represents a data quality risk caused by schema drift in a marketing data pipeline?

Answer: An upstream CRM adds a new field format, breaking downstream joins and causing null values in reports

Schema drift occurs when upstream data structure changes unexpectedly, breaking downstream transformations and introducing silent data errors.

A DTC brand's conversion rate model shows high accuracy in backtests but fails to predict sales during a flash sale event. Which mitigation would BEST address this for future models?

Answer: Incorporating promotional event flags as a feature in the model training data

Including a promotional event indicator variable allows the model to account for the behavioral shift that occurs during sales, reducing forecast error.