CNN Visualizing CNN Feature Maps 3 — Questions and Answers
Question 1: What is the purpose of 'maximally activating patches' in CNN visualization?
- To find input image regions that produce the highest activation for a specific filter (Correct answer)
- To prune filters with low average activation
- To normalize feature maps across a batch
- To reduce the number of channels in a layer
Correct answer: To find input image regions that produce the highest activation for a specific filter
Maximally activating patches are the input image crops that cause a particular filter or neuron to fire most strongly, revealing what it has learned to detect.
Question 2: Which method synthesizes an input image from scratch to maximize a neuron's activation, rather than searching the training set?
- Feature map subtraction
- Activation maximization (deep dream / feature visualization) (Correct answer)
- Dropout visualization
- Weight histogram analysis
Correct answer: Activation maximization (deep dream / feature visualization)
Activation maximization iteratively updates a random input image via gradient ascent to maximize a target neuron's response, revealing the ideal stimulus.
Question 3: Why are deeper convolutional layers in a CNN said to have larger 'receptive fields'?
- They use larger filters than early layers
- Each neuron integrates information from an increasingly larger region of the original input (Correct answer)
- They have more channels
- Their filters are applied with larger strides
Correct answer: Each neuron integrates information from an increasingly larger region of the original input
As feature maps are stacked, each subsequent neuron indirectly covers a larger spatial region of the original input, giving it a larger effective receptive field.
Question 4: When visualizing CNN feature maps for debugging, dead filters (all-zero activations) often indicate what problem?
- The model is overfitting
- Dying ReLU problem where filters never activate (Correct answer)
- Too much dropout regularization in the output layer
- The learning rate is too low
Correct answer: Dying ReLU problem where filters never activate
All-zero feature maps often result from the dying ReLU phenomenon, where neurons get stuck outputting zero for all inputs after large negative pre-activations.
Question 5: In PyTorch, what is a common approach to capture feature maps from an intermediate layer during a forward pass?
- Modifying the loss function
- Registering a forward hook on the target layer (Correct answer)
- Using torch.autograd.grad() on the layer
- Calling model.eval() before inference
Correct answer: Registering a forward hook on the target layer
PyTorch's register_forward_hook() attaches a callback to a layer that captures its output tensor whenever data passes through it.
Question 6: Feature maps from different channels in the same convolutional layer represent what?
- The same filter applied at different strides
- Different learned detectors applied to the same spatial locations (Correct answer)
- Separate images in the batch
- Different color channels of the output
Correct answer: Different learned detectors applied to the same spatial locations
Each channel in a convolutional layer's output corresponds to a different learned filter, so channels capture different types of patterns at the same spatial resolution.
Question 7: What does it mean for a feature map to be 'sparse'?
- Most activations are high and saturated
- Only a few spatial locations have significant non-zero activations (Correct answer)
- The feature map has fewer channels than the previous layer
- The filter weights contain many large values
Correct answer: Only a few spatial locations have significant non-zero activations
Sparse feature maps have most values near zero, with activation concentrated at a few positions where the filter's learned pattern is present.
CNN Visualizing CNN Feature Maps 3
What is the purpose of 'maximally activating patches' in CNN visualization?