CNN Pooling and Activation Functions 2 — Questions and Answers
Question 1: What is the primary effect of average pooling compared to max pooling in CNNs?
- Produces sharper feature maps by emphasizing dominant activations
- Smooths feature maps by computing the mean of values in each region (Correct answer)
- Increases spatial resolution of the feature map
- Eliminates negative activations from the feature map
Correct answer: Smooths feature maps by computing the mean of values in each region
Average pooling computes the mean of all values in a pooling window, resulting in smoother feature maps compared to max pooling.
Question 2: Which activation function is most commonly used in the hidden layers of modern deep CNNs?
- Sigmoid
- Tanh
- ReLU (Correct answer)
- Softmax
Correct answer: ReLU
ReLU (Rectified Linear Unit) is the dominant activation in hidden layers because it is computationally efficient and mitigates the vanishing gradient problem.
Question 3: Global Average Pooling (GAP) is frequently used at the end of a CNN to replace which traditional layer?
- Batch normalization layer
- Convolutional layer
- Fully connected (dense) layer (Correct answer)
- Dropout layer
Correct answer: Fully connected (dense) layer
GAP reduces each feature map to a single value, replacing large fully connected layers and dramatically reducing parameter count.
Question 4: What is the 'dying ReLU' problem?
- ReLU outputs explode to infinity during training
- Neurons with ReLU permanently output zero because their weights push inputs negative (Correct answer)
- ReLU causes gradients to grow exponentially
- ReLU saturates at large positive values, halting learning
Correct answer: Neurons with ReLU permanently output zero because their weights push inputs negative
When neurons receive consistently negative inputs, ReLU outputs zero with zero gradient, so weights never update—the neuron effectively 'dies.'
Question 5: In a CNN, a 4×4 feature map with a 2×2 max pooling kernel and stride 2 produces an output of what size?
- 4×4
- 3×3
- 2×2 (Correct answer)
- 1×1
Correct answer: 2×2
With a 2×2 kernel and stride 2, each dimension is halved: (4-2)/2 + 1 = 2, so the output is 2×2.
Question 6: The Leaky ReLU activation function was designed to address which limitation of standard ReLU?
- Computational cost of exponential operations
- Inability to handle multi-class outputs
- Zero gradient for negative inputs leading to dead neurons (Correct answer)
- Vanishing gradients for large positive inputs
Correct answer: Zero gradient for negative inputs leading to dead neurons
Leaky ReLU allows a small non-zero slope for negative inputs (e.g., 0.01x), ensuring gradients can still flow and preventing neurons from dying.
Question 7: Which property of pooling operations makes CNNs more robust to small translations in the input image?
- Pooling increases the number of learnable parameters
- Pooling introduces translational invariance by summarizing local regions (Correct answer)
- Pooling amplifies high-frequency details in feature maps
- Pooling normalizes pixel intensities across the input
Correct answer: Pooling introduces translational invariance by summarizing local regions
By taking the max or average over a local region, pooling makes the output insensitive to small shifts of features within that region.
CNN Pooling and Activation Functions 2
What is the primary effect of average pooling compared to max pooling in CNNs?