Architecture Fundamentals Flashcards

What is the primary advantage of using depthwise convolutions over standard convolutions in terms of computational cost?

Answer: They reduce multiply-add operations by separating spatial and channel-wise filtering

Depthwise convolutions apply one filter per input channel independently, reducing the number of multiply-add operations by a factor roughly equal to the number of output channels.

In the context of CNNs, what does 'weight sharing' mean and why is it important?

Answer: The same filter weights are applied at every spatial location, drastically reducing parameters

Weight sharing means one learned filter slides across the entire input, so a layer with a 3×3 filter has only 9 weights per channel regardless of input size.

Which architecture introduced the concept of 'network-in-network' using micro neural networks at each convolutional step?

Answer: Network-in-Network (NIN)

NIN replaced standard convolutional filters with small multi-layer perceptrons (mlpconv layers) at each location, enabling more complex feature extraction per patch.

What is the effect of using a very large learning rate when training a CNN with batch normalization?

Answer: It can be tolerated better than without BN because BN stabilizes the distribution of layer inputs

Batch Normalization reduces sensitivity to learning rate by normalizing activations, allowing higher rates that would cause instability in unnormalized networks.

In EfficientNet, what is 'compound scaling' and what does it scale jointly?

Answer: Jointly scaling network width, depth, and input resolution using a fixed ratio

EfficientNet's compound scaling coefficient φ uniformly scales width (channels), depth (layers), and resolution (input size) with empirically derived ratios.

Why does max pooling provide a degree of translation invariance in CNNs?

Answer: It selects the peak activation in a region regardless of its exact position within that window

Max pooling discards the precise location of a feature within the pooling window, so small shifts in input position produce the same output, yielding local translation invariance.

What distinguishes a 'same' padding strategy from a 'valid' padding strategy in convolutional layers?

Answer: Same padding pads the input so output spatial size equals input size; valid padding applies no padding, shrinking output

With 'same' padding, zeros are added so the output matches the input's spatial dimensions; with 'valid', no padding is added and the output shrinks based on filter size.