CNN Classic CNN Architectures 5 — Questions and Answers
Question 1: LeNet-5 uses which type of pooling in its subsampling layers?
- Max pooling
- Average pooling (Correct answer)
- Global average pooling
- Fractional pooling
Correct answer: Average pooling
LeNet-5 uses average pooling (called 'subsampling') in its pooling layers, which was the standard before max pooling became dominant.
Question 2: What key observation about 3x3 convolutions did VGGNet exploit to justify using only small filters?
- 3x3 filters are faster to compute on GPUs
- Two stacked 3x3 convolutions have the same receptive field as one 5x5 (Correct answer)
- 3x3 filters prevent overfitting better than larger filters
- 3x3 filters can be parallelized across batch dimension
Correct answer: Two stacked 3x3 convolutions have the same receptive field as one 5x5
Two stacked 3x3 conv layers cover the same 5x5 receptive field but with fewer parameters (18 vs 25 per channel) and an extra nonlinearity.
Question 3: GoogLeNet includes auxiliary classifiers during training. What is their primary purpose?
- To increase the number of output classes
- To combat vanishing gradients by injecting gradient signal at intermediate layers (Correct answer)
- To create an ensemble of predictions at inference
- To regularize the network using a multi-task loss
Correct answer: To combat vanishing gradients by injecting gradient signal at intermediate layers
Auxiliary classifiers in GoogLeNet inject gradient signal into earlier layers during backpropagation, helping train the deep network by fighting vanishing gradients.
Question 4: Which classic CNN was specifically designed to fit on embedded systems with a model size under 0.5MB using 'fire modules'?
- MobileNet
- SqueezeNet (Correct answer)
- ShuffleNet
- EfficientNet-B0
Correct answer: SqueezeNet
SqueezeNet (2016) used 'fire modules' with squeeze and expand layers to achieve AlexNet-level accuracy at less than 0.5MB, targeting embedded deployment.
Question 5: What is a 'projection shortcut' in ResNet, as opposed to an 'identity shortcut'?
- A shortcut that applies batch normalization before addition
- A shortcut that uses a 1x1 convolution to match dimensions when channel sizes change (Correct answer)
- A shortcut that skips two blocks instead of one
- A shortcut that averages input and output before addition
Correct answer: A shortcut that uses a 1x1 convolution to match dimensions when channel sizes change
A projection shortcut uses a 1x1 convolution (with stride) to match the spatial and channel dimensions when the skip connection would otherwise have a size mismatch.
Question 6: How does increasing network depth in VGGNet (from VGG-11 to VGG-19) primarily affect training?
- Training becomes faster due to more gradient paths
- Training accuracy improves monotonically without any downside
- Deeper models are harder to train and may plateau or degrade without careful initialization (Correct answer)
- The additional layers are skipped automatically via gating
Correct answer: Deeper models are harder to train and may plateau or degrade without careful initialization
Deeper VGGNet variants face greater optimization difficulty due to vanishing gradients, requiring careful initialization; VGG-19 offers only marginal gains over VGG-16.
Question 7: Which classic CNN architecture introduced the concept of training two parallel GPU pipelines (splitting feature maps across GPUs)?
- LeNet-5
- AlexNet (Correct answer)
- VGGNet
- GoogLeNet
Correct answer: AlexNet
AlexNet split its feature maps across two GPUs to fit within the 3GB VRAM of 2012 GTX 580 GPUs, an engineering constraint that shaped its architecture.
CNN Classic CNN Architectures 5
LeNet-5 uses which type of pooling in its subsampling layers?