CNN Overfitting and Regularization 2 — Questions and Answers
Question 1: Which regularization technique randomly zeroes out feature maps during training in CNNs rather than individual neurons?
- Spatial Dropout (Correct answer)
- L2 regularization
- Batch Normalization
- Weight decay
Correct answer: Spatial Dropout
Spatial Dropout drops entire feature maps (channels) instead of individual activations, which is more effective for spatially correlated CNN features.
Question 2: A CNN trained on 10,000 images achieves 99% training accuracy but only 62% validation accuracy. Which symptom does this describe?
- Underfitting
- Overfitting (Correct answer)
- High bias
- Gradient vanishing
Correct answer: Overfitting
A large gap between training accuracy (99%) and validation accuracy (62%) is the classic symptom of overfitting.
Question 3: What effect does L1 regularization have on CNN weight vectors compared to L2?
- It produces sparse weights with many zeros (Correct answer)
- It produces uniformly small weights
- It increases the learning rate
- It doubles the gradient magnitude
Correct answer: It produces sparse weights with many zeros
L1 regularization adds an absolute-value penalty that drives many weights exactly to zero, producing sparse weight vectors.
Question 4: During CNN training, which data augmentation technique generates new training samples by combining two images and their labels linearly?
- CutMix
- Mixup (Correct answer)
- Cutout
- Random Erasing
Correct answer: Mixup
Mixup creates training samples as convex combinations of pairs of training examples and their labels.
Question 5: What is weight decay in the context of CNN optimization, and how does it relate to L2 regularization?
- It is equivalent to L2 regularization for SGD but differs for Adam (Correct answer)
- It is always identical to L1 regularization
- It only applies to bias terms
- It increases weight magnitudes over time
Correct answer: It is equivalent to L2 regularization for SGD but differs for Adam
Weight decay and L2 regularization are equivalent under SGD but diverge under adaptive optimizers like Adam due to interaction with moment estimates.
Question 6: A deep CNN with many layers is tested without any regularization on a small dataset. Which problem is MOST likely?
- Underfitting due to too many parameters
- Overfitting due to excess model capacity (Correct answer)
- Vanishing gradients only
- Exploding activations only
Correct answer: Overfitting due to excess model capacity
A high-capacity model (many parameters) on a small dataset will memorize training examples, causing overfitting.
Question 7: Which technique involves training the CNN with the true label replaced by a soft distribution over all classes to reduce overconfidence?
- Label smoothing (Correct answer)
- Temperature scaling
- Knowledge distillation
- Mixup
Correct answer: Label smoothing
Label smoothing replaces hard one-hot targets with soft targets (e.g., 0.9 for correct class, 0.1/K for others) to prevent over-confident predictions.
CNN Overfitting and Regularization 2
Which regularization technique randomly zeroes out feature maps during training in CNNs rather than individual neurons?