Pooling and Activation Functions Flashcards

What is the primary purpose of a pooling layer in a Convolutional Neural Network?

Answer: To reduce the spatial dimensions of the feature maps.

Pooling layers, such as max pooling or average pooling, are used to downsample the feature maps, which reduces their width and height. This process helps to decrease the computational complexity, control overfitting, and make the network more invariant to small translations in the input image.

A CNN is being designed to classify images where the most prominent feature (e.g., a bright edge) in a local region is the most important for classification. Which pooling strategy would be most appropriate?

Answer: Max Pooling

Max pooling selects the maximum value from each patch of the feature map. This is particularly effective at capturing the most prominent or intense features, such as bright edges or corners, while discarding less relevant information.

Which of the following is a major advantage of using the ReLU (Rectified Linear Unit) activation function over the Sigmoid function in the hidden layers of a CNN?

Answer: It helps mitigate the vanishing gradient problem.

The ReLU function has a constant gradient of 1 for positive inputs, which helps to prevent the gradients from becoming extremely small during backpropagation in deep networks. The Sigmoid function, on the other hand, has gradients that are close to zero for very large or very small inputs, leading to the vanishing gradient problem.

What is the primary role of an activation function in a CNN?

Answer: To introduce non-linearity, allowing the network to learn complex patterns.

Without non-linear activation functions, a neural network, regardless of its depth, would behave like a single-layer linear model. Activation functions introduce non-linear properties that enable the network to learn and approximate complex relationships between inputs and outputs.

In a scenario where a CNN needs to classify images by considering a smoothed-out, generalized representation of features rather than the most dominant ones, which pooling method would be the most suitable choice?

Answer: Average Pooling

Average pooling calculates the average of the elements in a pooling window. This has a smoothing effect on the feature map, providing a more generalized representation and preserving the overall context rather than just the most salient features.

Which of the following activation functions would be most appropriate for the output layer of a CNN designed for a multi-class classification problem (e.g., classifying images into 10 different categories)?

Answer: Softmax

The Softmax function is ideal for multi-class classification because it converts a vector of raw output scores (logits) into a probability distribution over the classes. Each output value is between 0 and 1, and the sum of all output values equals 1, representing the model's confidence for each class.