Analytics Leadership & Innovation Flashcards

In the 'Three Horizons' innovation framework, where would a CDO place an exploratory generative AI research initiative with uncertain returns?

Answer: Horizon 3

Horizon 3 covers long-term, exploratory bets that may create future business models.

Which U.S. framework, published by a federal standards body, provides voluntary guidance for managing risks of AI systems through Govern, Map, Measure, and Manage functions?

Answer: NIST AI Risk Management Framework

The NIST AI RMF 1.0 organizes AI risk management into Govern, Map, Measure, and Manage functions.

A bank's model risk management program in the U.S. is primarily guided by which supervisory guidance?

Answer: SR 11-7

Federal Reserve/OCC SR 11-7 sets expectations for model development, validation, and governance in banks.

Which metric best demonstrates the business value of an analytics program to the board?

Answer: Incremental revenue or cost savings attributable to analytics use cases

Boards care about measurable financial outcomes rather than activity or output metrics.

What is the primary purpose of a champion/challenger approach in production analytics?

Answer: To compare a new model against the current production model on live traffic

Champion/challenger tests whether a new model outperforms the incumbent before full replacement.

Which discipline applies DevOps principles specifically to the deployment, monitoring, and retraining of machine learning models?

Answer: MLOps

MLOps automates and governs the ML lifecycle from training through deployment and monitoring.

A CDO observes that a treatment appears better in every customer segment but worse in the combined data. What statistical phenomenon should the team investigate?

Answer: Simpson's paradox

Simpson's paradox occurs when a trend in subgroups reverses after the groups are combined.