Architecture Fundamentals Flashcards

In VGGNet, what architectural choice was a key design principle?

Answer: Stacking many small 3×3 convolutional filters instead of large kernels

VGGNet demonstrated that deep networks using only 3×3 convolutions can achieve strong performance, as two 3×3 layers have the same receptive field as one 5×5 layer with fewer parameters.

What problem do residual connections in ResNet primarily solve?

Answer: Vanishing gradients that hinder training of very deep networks

Residual (skip) connections allow gradients to flow directly through the network, mitigating vanishing gradient problems that arise in very deep architectures.

What is the output size formula for a convolutional layer given input W, filter F, padding P, and stride S?

Answer: (W + 2P - F) / S + 1

The spatial output size is computed as (W + 2P − F) / S + 1, which accounts for padding, filter size, and the step size.

In the Inception module (GoogLeNet), why are convolutions of different kernel sizes applied in parallel?

Answer: To capture features at multiple scales simultaneously within the same layer

Parallel branches with 1×1, 3×3, and 5×5 filters let the Inception module learn features at multiple spatial scales and concatenate them along the depth axis.

What is global average pooling (GAP) and why is it used before the final classifier in modern CNNs?

Answer: It averages across spatial dimensions to produce a single value per feature map, replacing dense layers

GAP collapses each feature map to a scalar by averaging all spatial values, drastically reducing parameters and providing some translation invariance.

In a bottleneck residual block (ResNet-50+), what is the purpose of the 1×1 convolutions flanking the 3×3 layer?

Answer: To reduce and then restore channel depth, lowering computation for the 3×3 step

The flanking 1×1 convolutions compress channels before the expensive 3×3 convolution and expand them back, making the block computationally efficient.

What role does Batch Normalization play in CNN training?

Answer: It normalizes layer inputs per mini-batch, stabilizing and accelerating training

Batch Normalization normalizes activations to zero mean and unit variance per mini-batch, reducing internal covariate shift and allowing higher learning rates.