AML Risk Assessment & Management 2 — Questions and Answers
Question 1: Which technique is most appropriate for quantifying uncertainty in a neural network's predictions to support risk-aware decision making?
- Dropout at inference time (MC Dropout) (Correct answer)
- Increasing the learning rate during training
- Applying L1 regularization to all layers
- Using a larger mini-batch size
Correct answer: Dropout at inference time (MC Dropout)
MC Dropout approximates Bayesian inference by running multiple stochastic forward passes at test time, yielding a distribution over predictions that reflects model uncertainty.
Question 2: In a high-stakes ML deployment, what does 'model risk' primarily refer to?
- The financial cost of retraining the model
- Adverse outcomes arising from incorrect or misused model outputs (Correct answer)
- The compute resources consumed during inference
- The complexity of the model's architecture
Correct answer: Adverse outcomes arising from incorrect or misused model outputs
Model risk encompasses the potential for financial loss, regulatory penalty, or harm caused by errors in model development, implementation, or inappropriate use of predictions.
Question 3: A credit scoring model shows strong performance on historical data but degrades rapidly after deployment. Which risk category best describes this phenomenon?
- Sampling bias risk
- Data drift risk (Correct answer)
- Hyperparameter sensitivity risk
- Label noise risk
Correct answer: Data drift risk
Data drift occurs when the statistical properties of input features change over time post-deployment, causing models trained on historical distributions to underperform.
Question 4: When assessing risks of an ML model in a regulated industry, which document type formally captures identified risks and corresponding mitigations?
- A confusion matrix report
- A model risk register (Correct answer)
- A hyperparameter tuning log
- A data pipeline DAG
Correct answer: A model risk register
A model risk register systematically catalogs identified risks, their likelihood, potential impact, and the mitigations or controls applied to each.
Question 5: What is the primary purpose of stress-testing an ML model during risk assessment?
- To measure GPU utilization under heavy load
- To evaluate model behavior under extreme or adversarial input conditions (Correct answer)
- To benchmark training speed across hardware
- To optimize the model's F1 score on the test set
Correct answer: To evaluate model behavior under extreme or adversarial input conditions
Stress testing exposes the model to edge cases, outliers, and adversarial inputs to reveal failure modes that may not appear in standard evaluation.
Question 6: Which risk mitigation strategy involves training an ML model on multiple diverse datasets to reduce vulnerability to any single distribution shift?
- Ensemble diversification (Correct answer)
- Early stopping
- Weight decay scheduling
- Gradient clipping
Correct answer: Ensemble diversification
Ensemble diversification trains models on varied data sources or with different architectures so that no single distribution shift degrades all ensemble members simultaneously.
Question 7: In the context of ML risk management, 'operational risk' most directly refers to:
- The chance that the chosen loss function is suboptimal
- Failures in people, processes, or systems supporting the model's operation (Correct answer)
- Overfitting caused by insufficient regularization
- The risk of choosing the wrong evaluation metric
Correct answer: Failures in people, processes, or systems supporting the model's operation
Operational risk in ML covers failures stemming from inadequate internal processes, human error, system failures, or external events affecting the model's production pipeline.
AML Risk Assessment & Management 2
Which technique is most appropriate for quantifying uncertainty in a neural network's predictions to support risk-aware decision making?