CNN Overfitting and Regularization 5

Question 1

Why is it problematic to use the test set to choose the best regularization hyperparameters in a CNN experiment?

Accepted Answer

It causes information leakage, making the model appear to generalize better than it truly does

Answer

Using the test set for hyperparameter selection causes data leakage, resulting in over-optimistic performance estimates that do not reflect true generalization.

Question 2

In CNN regularization, what is 'stochastic depth' and how does it reduce overfitting in deep networks?

Accepted Answer

Randomly shortcircuiting (skipping) entire residual blocks during training

Answer

Stochastic depth randomly bypasses entire residual layers during training, effectively training an ensemble of networks with different depths.

Question 3

How does adding Gaussian noise to the input images during CNN training act as a regularizer?

Accepted Answer

It forces the model to learn robust features invariant to small perturbations

Answer

Input noise augmentation forces the CNN to learn features that are stable under small corruptions, improving robustness and generalization.

Question 4

Which metric directly measures whether a CNN is overfitting during a training run?

Accepted Answer

The ratio of training loss to validation loss (or training accuracy gap)

Answer

The gap between training and validation loss (or accuracy) is the direct operational metric for detecting overfitting during training.

Question 5

A CNN uses L2 regularization with coefficient λ. What happens to the gradient update rule for a weight w?

Accepted Answer

An additive penalty λ is added to the loss, and the gradient gains a +2λw term

Answer

L2 regularization adds λ‖w‖² to the loss, so its gradient contribution is +2λw, which is added to the usual gradient, effectively shrinking weights.

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Practice Test

CNN Overfitting and Regularization 5