Pooling and Activation Functions Flashcards

What is the primary effect of average pooling compared to max pooling in CNNs?

Answer: Smooths feature maps by computing the mean of values in each region

Average pooling computes the mean of all values in a pooling window, resulting in smoother feature maps compared to max pooling.

Which activation function is most commonly used in the hidden layers of modern deep CNNs?

Answer: ReLU

ReLU (Rectified Linear Unit) is the dominant activation in hidden layers because it is computationally efficient and mitigates the vanishing gradient problem.

Global Average Pooling (GAP) is frequently used at the end of a CNN to replace which traditional layer?

Answer: Fully connected (dense) layer

GAP reduces each feature map to a single value, replacing large fully connected layers and dramatically reducing parameter count.

What is the 'dying ReLU' problem?

Answer: Neurons with ReLU permanently output zero because their weights push inputs negative

When neurons receive consistently negative inputs, ReLU outputs zero with zero gradient, so weights never update—the neuron effectively 'dies.'

In a CNN, a 4×4 feature map with a 2×2 max pooling kernel and stride 2 produces an output of what size?

Answer: 2×2

With a 2×2 kernel and stride 2, each dimension is halved: (4-2)/2 + 1 = 2, so the output is 2×2.

The Leaky ReLU activation function was designed to address which limitation of standard ReLU?

Answer: Zero gradient for negative inputs leading to dead neurons

Leaky ReLU allows a small non-zero slope for negative inputs (e.g., 0.01x), ensuring gradients can still flow and preventing neurons from dying.

Which property of pooling operations makes CNNs more robust to small translations in the input image?

Answer: Pooling introduces translational invariance by summarizing local regions

By taking the max or average over a local region, pooling makes the output insensitive to small shifts of features within that region.