CNN Batch Normalization and Dropout

Question 1

What does batch normalization normalize during training?

Accepted Answer

The activations of each layer across a mini-batch

Answer

Batch normalization normalizes the activations (outputs) of a layer across the current mini-batch to have zero mean and unit variance.

Question 2

What are the two learnable parameters introduced by batch normalization?

Accepted Answer

Scale (gamma) and shift (beta)

Answer

Batch normalization introduces gamma (scale) and beta (shift) parameters that allow the network to undo the normalization if needed, learned during training.

Question 3

What problem does batch normalization primarily address?

Accepted Answer

Internal covariate shift across layers

Answer

Batch normalization addresses internal covariate shift, which is the change in the distribution of layer inputs during training as preceding layer parameters change.

Question 4

During inference, how does batch normalization behave differently compared to training?

Accepted Answer

It uses population statistics (running mean and variance) instead of batch statistics

Answer

During inference, batch normalization uses fixed population statistics (running mean and variance accumulated during training) rather than computing batch statistics, ensuring consistent outputs for single samples.

Question 5

Where is batch normalization typically inserted in a convolutional block?

Accepted Answer

Before the activation function, after the convolutional layer

Answer

Batch normalization is typically applied after the convolutional operation and before the activation function (e.g., Conv → BN → ReLU) to normalize pre-activation values.

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Practice Test

CNN Batch Normalization and Dropout