Deep Learning Training Techniques and Optimization 3

Question 1

What is hyperparameter tuning and which approach searches the space most efficiently?

Accepted Answer

Searching for optimal training configuration values; Bayesian optimization generally outperforms random or grid search

Answer

Hyperparameter tuning searches for the best learning rate, batch size, architecture choices, etc.; Bayesian optimization uses a probabilistic surrogate model to guide the search efficiently.

Question 2

What is cross-validation and how is it used for model evaluation in deep learning?

Accepted Answer

Partitioning data into k folds, training on k-1 and evaluating on 1 repeatedly to estimate generalization

Answer

k-fold cross-validation trains and evaluates the model k times with different folds as validation, providing a lower-variance estimate of generalization performance than a single split.

Question 3

What does momentum in SGD with momentum do during optimization?

Accepted Answer

Accumulates a velocity vector in the gradient direction, smoothing updates and accelerating progress in consistent directions

Answer

Momentum adds a fraction of the previous update to the current gradient step, building up speed in consistent gradient directions and dampening oscillations in high-curvature directions.

Question 4

What is label smoothing and why is it used in training classification models?

Accepted Answer

Replacing hard one-hot labels with soft distributions to prevent overconfident predictions and improve calibration

Answer

Label smoothing replaces the target probability of 1.0 with 1-ε and distributes ε across other classes, preventing the model from becoming overconfident and improving generalization.

Question 5

What is the purpose of a confusion matrix in evaluating a classification model?

Accepted Answer

Showing the counts of correct and incorrect predictions per class to reveal patterns of misclassification

Answer

A confusion matrix displays true positives, true negatives, false positives, and false negatives for each class, revealing which classes the model confuses with one another.

Deep Learning Practice Test

Deep Learning Practice Test

Deep Learning Training Techniques and Optimization 3