AML Model Evaluation & Optimization Techniques 3 — Questions and Answers
Question 1: Which regularization technique explicitly prevents co-adaptation of neurons by randomly zeroing activations during training?
- L2 weight decay
- Batch normalization
- Dropout (Correct answer)
- Label smoothing
Correct answer: Dropout
Dropout randomly deactivates neurons during each training step, forcing the network to learn redundant representations and preventing co-adaptation.
Question 2: In the context of learning curves, what does a large gap between training and validation loss with both curves converging indicate?
- Underfitting requiring a more complex model
- Optimal model capacity with good generalization
- High variance (overfitting) needing more regularization or data (Correct answer)
- Label noise causing inconsistent gradients
Correct answer: High variance (overfitting) needing more regularization or data
A large persistent gap between training and validation loss with convergence indicates the model has memorized training data and is not generalizing well (high variance/overfitting).
Question 3: What does the Brier Score measure in probabilistic classification?
- The log-likelihood of the predicted probability distribution
- The mean squared error between predicted probabilities and actual binary outcomes (Correct answer)
- The area under the calibration curve
- The KL divergence between predicted and true distributions
Correct answer: The mean squared error between predicted probabilities and actual binary outcomes
The Brier Score is the mean squared error between predicted class probabilities and the true binary (0/1) outcomes, rewarding well-calibrated probability estimates.
Question 4: Which cross-validation strategy is most appropriate when time-series data exhibits temporal dependencies?
- Standard k-fold with random shuffling
- Leave-one-out cross-validation
- Time series split with expanding window (Correct answer)
- Group k-fold by timestamp
Correct answer: Time series split with expanding window
Time series split uses an expanding or sliding window where the validation set always comes after the training set chronologically, preserving temporal causality.
Question 5: What is the effect of using a very small batch size in stochastic gradient descent?
- Smoother gradient estimates with faster convergence
- Higher gradient noise that can escape local minima but slower wall-clock convergence (Correct answer)
- Reduced memory usage with no effect on convergence quality
- Elimination of the need for learning rate scheduling
Correct answer: Higher gradient noise that can escape local minima but slower wall-clock convergence
Small batch sizes produce noisy gradient estimates that can help escape sharp local minima but typically require more iterations and wall-clock time to converge.
Question 6: In multi-objective hyperparameter optimization, what does the Pareto frontier represent?
- The single best configuration across all objectives
- The set of configurations where no objective can be improved without degrading another (Correct answer)
- The average performance weighted by objective importance
- The region of hyperparameter space with highest variance
Correct answer: The set of configurations where no objective can be improved without degrading another
The Pareto frontier contains all non-dominated solutions — configurations where improving one objective (e.g., accuracy) necessarily worsens another (e.g., inference latency).
Question 7: Which technique decomposes model predictions into feature contributions that satisfy both efficiency and local accuracy properties inspired by cooperative game theory?
- LIME (Local Interpretable Model-agnostic Explanations)
- SHAP (SHapley Additive exPlanations) (Correct answer)
- Integrated Gradients
- Partial Dependence Plots
Correct answer: SHAP (SHapley Additive exPlanations)
SHAP uses Shapley values from cooperative game theory to fairly attribute each feature's contribution, satisfying efficiency (contributions sum to prediction) and local accuracy.
AML Model Evaluation & Optimization Techniques 3
Which regularization technique explicitly prevents co-adaptation of neurons by randomly zeroing activations during training?