Overfitting and Regularization Flashcards

Which regularization technique randomly zeroes out feature maps during training in CNNs rather than individual neurons?

Answer: Spatial Dropout

Spatial Dropout drops entire feature maps (channels) instead of individual activations, which is more effective for spatially correlated CNN features.

A CNN trained on 10,000 images achieves 99% training accuracy but only 62% validation accuracy. Which symptom does this describe?

Answer: Overfitting

A large gap between training accuracy (99%) and validation accuracy (62%) is the classic symptom of overfitting.

What effect does L1 regularization have on CNN weight vectors compared to L2?

Answer: It produces sparse weights with many zeros

L1 regularization adds an absolute-value penalty that drives many weights exactly to zero, producing sparse weight vectors.

During CNN training, which data augmentation technique generates new training samples by combining two images and their labels linearly?

Answer: Mixup

Mixup creates training samples as convex combinations of pairs of training examples and their labels.

What is weight decay in the context of CNN optimization, and how does it relate to L2 regularization?

Answer: It is equivalent to L2 regularization for SGD but differs for Adam

Weight decay and L2 regularization are equivalent under SGD but diverge under adaptive optimizers like Adam due to interaction with moment estimates.

A deep CNN with many layers is tested without any regularization on a small dataset. Which problem is MOST likely?

Answer: Overfitting due to excess model capacity

A high-capacity model (many parameters) on a small dataset will memorize training examples, causing overfitting.

Which technique involves training the CNN with the true label replaced by a soft distribution over all classes to reduce overconfidence?

Answer: Label smoothing

Label smoothing replaces hard one-hot targets with soft targets (e.g., 0.9 for correct class, 0.1/K for others) to prevent over-confident predictions.