CNN Architecture Fundamentals 5 — Questions and Answers
Question 1: What is the primary advantage of using depthwise convolutions over standard convolutions in terms of computational cost?
- They require more memory but execute faster on GPU
- They reduce multiply-add operations by separating spatial and channel-wise filtering (Correct answer)
- They eliminate the need for activation functions
- They allow arbitrarily large kernel sizes without extra cost
Correct answer: They reduce multiply-add operations by separating spatial and channel-wise filtering
Depthwise convolutions apply one filter per input channel independently, reducing the number of multiply-add operations by a factor roughly equal to the number of output channels.
Question 2: In the context of CNNs, what does 'weight sharing' mean and why is it important?
- Multiple layers share the same weights to save memory
- The same filter weights are applied at every spatial location, drastically reducing parameters (Correct answer)
- Weights are shared between the encoder and decoder paths
- Convolutional and fully connected layers reuse the same weight matrix
Correct answer: The same filter weights are applied at every spatial location, drastically reducing parameters
Weight sharing means one learned filter slides across the entire input, so a layer with a 3×3 filter has only 9 weights per channel regardless of input size.
Question 3: Which architecture introduced the concept of 'network-in-network' using micro neural networks at each convolutional step?
- AlexNet
- VGGNet
- Network-in-Network (NIN) (Correct answer)
- SqueezeNet
Correct answer: Network-in-Network (NIN)
NIN replaced standard convolutional filters with small multi-layer perceptrons (mlpconv layers) at each location, enabling more complex feature extraction per patch.
Question 4: What is the effect of using a very large learning rate when training a CNN with batch normalization?
- It causes gradient values to be clipped by BN, preventing learning
- It can be tolerated better than without BN because BN stabilizes the distribution of layer inputs (Correct answer)
- It immediately causes NaN losses due to BN's variance computation
- It has no effect because BN removes the dependence on learning rate scale
Correct answer: It can be tolerated better than without BN because BN stabilizes the distribution of layer inputs
Batch Normalization reduces sensitivity to learning rate by normalizing activations, allowing higher rates that would cause instability in unnormalized networks.
Question 5: In EfficientNet, what is 'compound scaling' and what does it scale jointly?
- Scaling only depth to add more residual blocks uniformly
- Jointly scaling network width, depth, and input resolution using a fixed ratio (Correct answer)
- Scaling the learning rate and batch size together during training
- Alternating between scaling width and depth in even and odd layers
Correct answer: Jointly scaling network width, depth, and input resolution using a fixed ratio
EfficientNet's compound scaling coefficient φ uniformly scales width (channels), depth (layers), and resolution (input size) with empirically derived ratios.
Question 6: Why does max pooling provide a degree of translation invariance in CNNs?
- It averages activations, smoothing out positional differences
- It selects the peak activation in a region regardless of its exact position within that window (Correct answer)
- It rotates the feature map to align dominant orientations
- It normalizes activations so that their spatial positions become irrelevant
Correct answer: It selects the peak activation in a region regardless of its exact position within that window
Max pooling discards the precise location of a feature within the pooling window, so small shifts in input position produce the same output, yielding local translation invariance.
Question 7: What distinguishes a 'same' padding strategy from a 'valid' padding strategy in convolutional layers?
- Same padding uses larger filters; valid padding uses 1×1 filters
- Same padding pads the input so output spatial size equals input size; valid padding applies no padding, shrinking output (Correct answer)
- Same padding is only used in pooling; valid padding applies to convolutions
- Same padding doubles the input size; valid padding halves it
Correct answer: Same padding pads the input so output spatial size equals input size; valid padding applies no padding, shrinking output
With 'same' padding, zeros are added so the output matches the input's spatial dimensions; with 'valid', no padding is added and the output shrinks based on filter size.
CNN Architecture Fundamentals 5
What is the primary advantage of using depthwise convolutions over standard convolutions in terms of computational cost?