CNN Architecture Fundamentals 3 — Questions and Answers
Question 1: In VGGNet, what architectural choice was a key design principle?
- Using a single large 11×11 kernel in the first layer
- Stacking many small 3×3 convolutional filters instead of large kernels (Correct answer)
- Applying global average pooling before the classifier
- Introducing residual skip connections between layers
Correct answer: Stacking many small 3×3 convolutional filters instead of large kernels
VGGNet demonstrated that deep networks using only 3×3 convolutions can achieve strong performance, as two 3×3 layers have the same receptive field as one 5×5 layer with fewer parameters.
Question 2: What problem do residual connections in ResNet primarily solve?
- Overfitting on small datasets
- Vanishing gradients that hinder training of very deep networks (Correct answer)
- Slow inference speed during deployment
- Excessive memory usage from large feature maps
Correct answer: Vanishing gradients that hinder training of very deep networks
Residual (skip) connections allow gradients to flow directly through the network, mitigating vanishing gradient problems that arise in very deep architectures.
Question 3: What is the output size formula for a convolutional layer given input W, filter F, padding P, and stride S?
- (W + 2P - F) / S + 1 (Correct answer)
- (W - F + 2P) * S
- (W * F) / (P + S)
- (W + F - 2P) / S - 1
Correct answer: (W + 2P - F) / S + 1
The spatial output size is computed as (W + 2P − F) / S + 1, which accounts for padding, filter size, and the step size.
Question 4: In the Inception module (GoogLeNet), why are convolutions of different kernel sizes applied in parallel?
- To reduce the number of training epochs needed
- To capture features at multiple scales simultaneously within the same layer (Correct answer)
- To eliminate the need for pooling layers
- To enforce weight sharing across different resolution branches
Correct answer: To capture features at multiple scales simultaneously within the same layer
Parallel branches with 1×1, 3×3, and 5×5 filters let the Inception module learn features at multiple spatial scales and concatenate them along the depth axis.
Question 5: What is global average pooling (GAP) and why is it used before the final classifier in modern CNNs?
- It averages across spatial dimensions to produce a single value per feature map, replacing dense layers (Correct answer)
- It pools across the channel dimension to reduce depth
- It applies max pooling with a kernel equal to the full feature map size
- It normalizes activations globally to prevent covariate shift
Correct answer: It averages across spatial dimensions to produce a single value per feature map, replacing dense layers
GAP collapses each feature map to a scalar by averaging all spatial values, drastically reducing parameters and providing some translation invariance.
Question 6: In a bottleneck residual block (ResNet-50+), what is the purpose of the 1×1 convolutions flanking the 3×3 layer?
- To add non-linearity before the skip connection
- To reduce and then restore channel depth, lowering computation for the 3×3 step (Correct answer)
- To apply spatial attention across the feature map
- To perform batch normalization without an extra layer
Correct answer: To reduce and then restore channel depth, lowering computation for the 3×3 step
The flanking 1×1 convolutions compress channels before the expensive 3×3 convolution and expand them back, making the block computationally efficient.
Question 7: What role does Batch Normalization play in CNN training?
- It clips gradient values to prevent exploding gradients
- It normalizes layer inputs per mini-batch, stabilizing and accelerating training (Correct answer)
- It randomly drops entire feature maps to improve generalization
- It rescales weight matrices to unit norm after each update
Correct answer: It normalizes layer inputs per mini-batch, stabilizing and accelerating training
Batch Normalization normalizes activations to zero mean and unit variance per mini-batch, reducing internal covariate shift and allowing higher learning rates.
CNN Architecture Fundamentals 3
In VGGNet, what architectural choice was a key design principle?