Risk Assessment & Management Flashcards

What is 'adversarial risk' in the context of deployed ML models, and which domain is most affected?

Answer: The risk that malicious actors craft inputs to deliberately fool the model, most critically affecting security and safety systems

Adversarial risk arises when intentional perturbations—often imperceptible to humans—cause a model to misclassify inputs, posing critical threats in domains like autonomous vehicles, malware detection, and face recognition.

Which statistical test is most commonly used to detect whether two data distributions are significantly different, supporting data drift monitoring?

Answer: Kolmogorov-Smirnov (KS) test

The KS test is a non-parametric test that measures the maximum difference between two empirical cumulative distribution functions, making it well-suited for detecting distributional shifts in continuous features.

An ML model is integrated into an automated lending decision system. Which risk control is specifically required under SR 11-7 guidance from the Federal Reserve?

Answer: Independent model validation conducted by a team separate from model developers

SR 11-7 mandates that model validation be performed by qualified staff independent of model development to provide an unbiased assessment of model soundness.

What role does a 'kill switch' or 'circuit breaker' play in ML model risk management?

Answer: It automatically halts or reverts model predictions when performance drops below a defined threshold

A kill switch or circuit breaker provides an automated safeguard that takes the model offline or reverts to a fallback rule-based system when real-time metrics signal unacceptable degradation.

When assessing third-party ML model risk (e.g., a vendor-supplied model), which practice is most important?

Answer: Conducting due diligence including independent validation, documentation review, and contractual SLAs

Third-party model risk requires the same validation rigor as internally developed models, including independent performance testing, data governance review, and clear service level agreements.

What is 'tail risk' in the context of ML model predictions, and why is it critical for risk management?

Answer: The risk of rare but extreme prediction errors that can cause disproportionately large negative outcomes

Tail risk refers to low-probability but high-severity errors at the extremes of the prediction distribution, which standard average-based metrics obscure but which can dominate real-world losses.

Which approach best mitigates the risk of a model perpetuating historical biases present in training data?

Answer: Applying fairness constraints or reweighting techniques during training to balance outcomes across groups

Fairness constraints (e.g., equalized odds) and reweighting penalize or correct for disparate outcomes during optimization, directly targeting the mechanism by which historical bias propagates into predictions.