AML Model Evaluation & Optimization Techniques 2

Question 1

Which technique addresses class imbalance during model evaluation by computing metrics on a resampled dataset that reflects equal class distribution?

Accepted Answer

Balanced accuracy metric on original data

Answer

Balanced accuracy averages recall across classes, directly accounting for imbalance without resampling the dataset itself.

Question 2

In Bayesian hyperparameter optimization, what probabilistic model is most commonly used to approximate the objective function?

Accepted Answer

Gaussian Process

Answer

Gaussian Processes provide a posterior distribution over the objective function, enabling uncertainty-aware acquisition functions like Expected Improvement.

Question 3

What does a DET (Detection Error Tradeoff) curve plot that distinguishes it from an ROC curve?

Accepted Answer

False Negative Rate vs. False Positive Rate on normal deviate axes

Answer

DET curves plot FNR vs. FPR using normal deviate (probit) scales, which linearize performance curves for easier comparison of classifiers.

Question 4

When using nested cross-validation for model selection, what does the outer loop estimate?

Accepted Answer

The generalization error of the model selection procedure

Answer

The outer loop in nested cross-validation provides an unbiased estimate of the generalization error of the entire model selection pipeline, not just the selected model.

Question 5

Which optimizer adapts per-parameter learning rates using exponentially weighted moving averages of both gradients and their squares?

Accepted Answer

Adam

Answer

Adam combines first-moment (gradient) and second-moment (squared gradient) exponential moving averages with bias correction, making it more adaptive than RMSprop or AdaGrad alone.

(AML) Advanced Machine Learning Certification Practice Test

(AML) Advanced Machine Learning Certification Practice Test

AML Model Evaluation & Optimization Techniques 2