Deep Learning Training Techniques and Optimization 3

Question 1

What is hyperparameter tuning and which approach searches the space most efficiently?

Accepted Answer

Searching for optimal training configuration values; Bayesian optimization generally outperforms random or grid search

Answer

Manually adjusting weights; manual search is most efficient

Answer

Adjusting model architecture during training; random search is always best

Answer

Tuning only the learning rate; grid search suffices

Question 2

What is cross-validation and how is it used for model evaluation in deep learning?

Accepted Answer

Partitioning data into k folds, training on k-1 and evaluating on 1 repeatedly to estimate generalization

Answer

Training on multiple datasets and averaging weights

Answer

Using cross-entropy loss for validation metric only

Answer

Evaluating the model on data from different geographic regions

Question 3

What does momentum in SGD with momentum do during optimization?

Accepted Answer

Accumulates a velocity vector in the gradient direction, smoothing updates and accelerating progress in consistent directions

Answer

Scales the learning rate by the gradient magnitude

Answer

Randomly perturbs weights to escape local minima

Answer

Applies second-order gradient information to the update

Question 4

What is label smoothing and why is it used in training classification models?

Accepted Answer

Replacing hard one-hot labels with soft distributions to prevent overconfident predictions and improve calibration

Answer

Filtering noisy labels from training data

Answer

Smoothing class distributions by oversampling minority classes

Answer

Applying moving average to training labels over epochs

Question 5

What is the purpose of a confusion matrix in evaluating a classification model?

Accepted Answer

Showing the counts of correct and incorrect predictions per class to reveal patterns of misclassification

Answer

Visualizing the loss surface around the current weights

Answer

Displaying the distribution of training examples per class

Answer

Plotting gradient magnitudes during training

Question 6

What is class imbalance and how does oversampling address it?

Accepted Answer

Having far more examples of some classes than others; oversampling duplicates minority class examples to balance the dataset

Answer

Having too many classes; oversampling removes rare classes

Answer

Having overlapping classes; oversampling adds noise to class boundaries

Answer

Having too few total examples; oversampling generates synthetic majority examples