CNN Overfitting and Regularization 3 — Questions and Answers
Question 1: In a CNN, early stopping halts training when which condition is met?
- Training loss reaches zero
- Validation loss stops improving or starts increasing (Correct answer)
- Learning rate becomes very small
- All weights converge to the same value
Correct answer: Validation loss stops improving or starts increasing
Early stopping monitors validation loss and terminates training when it plateaus or increases, indicating the model is beginning to overfit.
Question 2: How does max pooling contribute to a CNN's resistance to overfitting?
- It adds noise to feature maps
- It reduces spatial resolution, decreasing parameter count in subsequent layers (Correct answer)
- It applies L2 penalties to filters
- It randomly drops feature map values
Correct answer: It reduces spatial resolution, decreasing parameter count in subsequent layers
Max pooling downsamples feature maps, which reduces the number of inputs to subsequent layers and thus the total number of learned parameters.
Question 3: Which statement about Dropout during inference (test time) in a CNN is correct?
- Dropout is applied with the same rate as training
- Dropout is disabled and weights are scaled by the keep probability (Correct answer)
- Dropout is replaced by Batch Normalization
- Dropout rate is doubled at test time
Correct answer: Dropout is disabled and weights are scaled by the keep probability
At inference, Dropout is turned off and weights are scaled by the keep probability (or equivalently, outputs are scaled at training) to maintain expected activation magnitudes.
Question 4: What is the 'bias-variance tradeoff' as it applies to CNN design?
- Increasing model capacity reduces bias but can increase variance (overfitting) (Correct answer)
- Larger datasets always reduce both bias and variance equally
- Adding more layers always reduces both bias and variance
- Regularization increases both bias and variance
Correct answer: Increasing model capacity reduces bias but can increase variance (overfitting)
Larger CNN capacity reduces training error (bias) but increases sensitivity to training data fluctuations (variance), risking overfitting.
Question 5: CutOut augmentation reduces CNN overfitting by which mechanism?
- Blending two images together
- Randomly masking out rectangular regions of training images (Correct answer)
- Flipping the image horizontally
- Adding Gaussian noise to pixel values
Correct answer: Randomly masking out rectangular regions of training images
CutOut randomly removes square patches from training images, forcing the CNN to use multiple regions rather than relying on any single discriminative area.
Question 6: Which layer type in modern CNNs implicitly regularizes by normalizing activations and thereby reducing internal covariate shift?
- Fully connected layer
- Batch Normalization (Correct answer)
- Softmax layer
- ReLU activation
Correct answer: Batch Normalization
Batch Normalization normalizes each mini-batch's activations, stabilizing training and acting as a mild regularizer.
Question 7: A practitioner reduces the number of filters in each CNN convolutional layer by half. How does this combat overfitting?
- It increases training data diversity
- It reduces total model parameters, lowering capacity to memorize noise (Correct answer)
- It applies L1 penalty to activations
- It increases the effective learning rate
Correct answer: It reduces total model parameters, lowering capacity to memorize noise
Fewer filters means fewer learnable parameters, directly reducing the model's capacity to overfit.
CNN Overfitting and Regularization 3
In a CNN, early stopping halts training when which condition is met?