CNN Pooling and Activation Functions 3 — Questions and Answers
Question 1: What distinguishes the ELU (Exponential Linear Unit) activation from ReLU?
- ELU uses a linear function for all positive inputs and zero for negatives
- ELU outputs negative values for negative inputs using an exponential, giving non-zero mean activations (Correct answer)
- ELU clips activations at a fixed maximum value to prevent explosion
- ELU applies a sigmoid curve across all input values
Correct answer: ELU outputs negative values for negative inputs using an exponential, giving non-zero mean activations
ELU uses an exponential decay for negative inputs, pushing mean activations closer to zero and speeding up learning compared to ReLU.
Question 2: In the context of pooling, what does 'stride' control?
- The depth of the pooling operation across channels
- How much the pooling window moves at each step across the feature map (Correct answer)
- The number of distinct pooling operations applied in parallel
- The activation function applied after pooling
Correct answer: How much the pooling window moves at each step across the feature map
Stride determines the step size of the pooling window as it slides across the feature map, controlling output size and overlap.
Question 3: Which activation function outputs values strictly between 0 and 1 and is commonly used for binary classification output layers?
- ReLU
- Tanh
- Sigmoid (Correct answer)
- SELU
Correct answer: Sigmoid
Sigmoid maps any real-valued input to the (0, 1) range, making it suitable for binary classification where outputs represent probabilities.
Question 4: What is fractional max pooling?
- Max pooling with a non-integer kernel size or stride, producing irregular output dimensions (Correct answer)
- A pooling method that applies different pool sizes per channel
- Pooling that averages only the top fraction of activations in a window
- A technique combining max and average pooling with a learned weight
Correct answer: Max pooling with a non-integer kernel size or stride, producing irregular output dimensions
Fractional max pooling uses random or pseudo-random pooling regions with non-integer reduction ratios, acting as a form of data augmentation.
Question 5: How does the Softmax activation function differ from Sigmoid when used in output layers?
- Softmax outputs sum to 1 across all classes, while Sigmoid outputs each class independently (Correct answer)
- Softmax clips values above 1, while Sigmoid allows unbounded positive outputs
- Softmax is used for binary tasks, while Sigmoid handles multi-class problems
- Softmax introduces sparsity, while Sigmoid ensures dense probability distributions
Correct answer: Softmax outputs sum to 1 across all classes, while Sigmoid outputs each class independently
Softmax normalizes outputs so they form a probability distribution summing to 1, making it ideal for mutually exclusive multi-class classification.
Question 6: A CNN feature map of size 8×8 is processed by 2×2 max pooling with stride 2 applied twice sequentially. What is the final output size?
- 4×4
- 2×2 (Correct answer)
- 1×1
- 6×6
Correct answer: 2×2
After the first pooling: 8→4; after the second pooling: 4→2, so the final feature map is 2×2.
Question 7: Why is the hyperbolic tangent (Tanh) activation function generally preferred over Sigmoid for hidden layers in older architectures?
- Tanh is computationally cheaper to compute than Sigmoid
- Tanh outputs are zero-centered, reducing bias in gradient updates (Correct answer)
- Tanh avoids saturation at extreme input values unlike Sigmoid
- Tanh produces sparse activations useful for feature selection
Correct answer: Tanh outputs are zero-centered, reducing bias in gradient updates
Tanh outputs range from -1 to 1 with mean zero, so activations are zero-centered, which helps gradients flow more symmetrically during backpropagation.
CNN Pooling and Activation Functions 3
What distinguishes the ELU (Exponential Linear Unit) activation from ReLU?