CNN Visualizing CNN Feature Maps 5 — Questions and Answers
Question 1: What is the primary goal of applying t-SNE to CNN feature vectors extracted from an intermediate layer?
- To reduce the number of filters in the layer
- To visualize high-dimensional feature space by projecting similar activations close together in 2D (Correct answer)
- To measure filter redundancy during training
- To convert feature maps into saliency maps
Correct answer: To visualize high-dimensional feature space by projecting similar activations close together in 2D
t-SNE projects high-dimensional CNN feature vectors into 2D, clustering semantically similar images together, revealing the structure of learned representations.
Question 2: Which approach would best reveal whether a CNN's convolutional filters have learned color-selective detectors?
- Visualize feature maps on grayscale inputs
- Visualize maximally activating patches from a color image dataset and inspect their dominant hues (Correct answer)
- Plot the weight distribution histogram of the filters
- Apply dropout and observe which features survive
Correct answer: Visualize maximally activating patches from a color image dataset and inspect their dominant hues
Examining the real image patches that maximally activate each filter exposes their preferred colors, textures, and orientations directly from the data.
Question 3: SHAP (SHapley Additive exPlanations) applied to CNN image classification attributes what to each pixel?
- A gradient value from the loss function
- A contribution score indicating how much that pixel pushed the prediction toward or away from a class (Correct answer)
- The filter weight that responds most strongly to that pixel
- The attention weight from a transformer layer
Correct answer: A contribution score indicating how much that pixel pushed the prediction toward or away from a class
SHAP computes each pixel's Shapley value, quantifying its marginal contribution to the final prediction score relative to a reference (background) image.
Question 4: In CNN feature map analysis, what does 'polysemanticity' of a neuron mean?
- A neuron that only activates for a single, highly specific visual concept
- A neuron that responds to multiple unrelated visual concepts (Correct answer)
- A neuron whose activation depends on batch statistics
- A neuron with a very large receptive field
Correct answer: A neuron that responds to multiple unrelated visual concepts
Polysemantic neurons respond to multiple semantically distinct inputs, making them harder to interpret cleanly — a common finding in large CNN models.
Question 5: When performing activation maximization with total variation (TV) regularization, what does the TV term contribute?
- It encourages the generated image to have high-frequency details
- It penalizes pixel-to-pixel variation, promoting smoother, more natural-looking visualizations (Correct answer)
- It maximizes the contrast between adjacent filter activations
- It constrains the number of active neurons in the generated image
Correct answer: It penalizes pixel-to-pixel variation, promoting smoother, more natural-looking visualizations
Total variation regularization penalizes large differences between neighboring pixels, preventing the gradient-ascent image from becoming a noisy pattern of high-frequency artifacts.
Question 6: What information does a 'mean activation map' (averaging all channels of a feature tensor) convey?
- Which filters have the largest weights
- A spatial summary of overall network attention — where the layer as a whole responds strongly (Correct answer)
- The class probability at each spatial location
- The gradient magnitude at each position
Correct answer: A spatial summary of overall network attention — where the layer as a whole responds strongly
Averaging across all channels of a feature tensor collapses filter-specific detail into a single spatial map showing the regions that collectively activate most across all detectors.
Question 7: Why might feature maps of a batch-normalized CNN layer look different when the model is in training mode vs. inference mode?
- The filter weights are updated only during training mode
- Batch norm uses mini-batch statistics during training but running mean/variance during inference, changing activation distributions (Correct answer)
- Dropout is applied to feature maps only during training
- The learning rate affects activation values directly
Correct answer: Batch norm uses mini-batch statistics during training but running mean/variance during inference, changing activation distributions
During training, batch normalization normalizes using the current mini-batch's mean and variance; during inference it uses accumulated running statistics, which can shift activation values.
CNN Visualizing CNN Feature Maps 5
What is the primary goal of applying t-SNE to CNN feature vectors extracted from an intermediate layer?