Visualizing CNN Feature Maps Flashcards

When visualizing the feature maps of a trained CNN, what type of patterns are typically detected by the filters in the initial (early) layers compared to the final (deeper) layers?

Answer: Early layers detect simple features like edges and colors; deeper layers detect more complex patterns and object parts.

CNNs learn features hierarchically. The initial layers, being closest to the raw input image, learn to detect basic, low-level features such as edges, corners, and color blobs. Subsequent, deeper layers combine these simple features to recognize more complex and abstract patterns like textures, shapes, and eventually parts of objects.

A data scientist wants to understand which specific pixels in an input image are most influential in causing a CNN to make a particular classification decision (e.g., classifying an image as a 'dog'). Which visualization technique would be most appropriate for this purpose?

Answer: Saliency Maps (or Gradient-based Attribution).

Saliency maps are designed to solve this exact problem. They work by computing the gradient of the output class score with respect to the input image pixels. The magnitude of the gradient for each pixel indicates how much a small change in that pixel's intensity would affect the class score, thus highlighting the most influential pixels for that specific classification decision.

Which of the following is a key architectural requirement for generating a classic Class Activation Map (CAM) to visualize where a CNN is 'looking'?

Answer: The network must have a Global Average Pooling (GAP) layer followed by a dense output layer.

The original Class Activation Mapping (CAM) technique requires a specific network architecture. The final convolutional layer must be followed by a Global Average Pooling (GAP) layer, which is then connected to a final dense (fully-connected) layer for classification. This structure allows the weights from the dense layer to be used to create a weighted sum of the preceding feature maps, generating the heatmap.

What is the primary goal of the Activation Maximization visualization technique when applied to a specific filter or neuron in a CNN?

Answer: To generate a synthetic image that causes the highest possible activation for that filter.

Activation Maximization, also known as feature visualization, is an optimization process that aims to understand what a filter or neuron has learned to detect. It starts with a random noise image and iteratively modifies it using gradient ascent to find an input that maximally activates the chosen filter. The resulting synthetic image is a representation of the ideal pattern or feature that the filter is tuned to recognize.

A researcher is working with a standard ResNet-50 model and wants to generate a class-specific heatmap showing important image regions for a prediction. Since this architecture doesn't end with a Global Average Pooling (GAP) layer suitable for classic CAM, which popular technique can they use without retraining or modifying the model?

Answer: Gradient-weighted Class Activation Mapping (Grad-CAM).

Grad-CAM is a generalization of CAM designed specifically for this scenario. It overcomes the architectural limitation of requiring a GAP layer by using the gradient information flowing into the final convolutional layer. These gradients are used to weight the feature maps, indicating their importance for a specific class prediction, making it applicable to a wide range of CNN architectures like ResNet without modification.

Upon visualizing the learned filters of the first convolutional layer of a CNN trained on a large dataset of natural images (like ImageNet), which of the following patterns would you most expect to see?

Answer: Gabor-like filters detecting edges at various orientations and color blobs.

The first layer of a CNN processes raw pixel data. To build a hierarchical representation of the visual world, it must first learn to detect the most basic visual primitives. In natural images, these primitives are features like edges, lines, corners, and patches of color. These learned filters often resemble Gabor filters, which are used in image processing for edge and texture detection.