CNN Batch Normalization and Dropout

Question 1

What does batch normalization normalize during training?

Accepted Answer

The activations of each layer across a mini-batch

Answer

The weights of each convolutional filter

Answer

The learning rate at each iteration

Answer

The gradient magnitudes before backpropagation

Question 2

What are the two learnable parameters introduced by batch normalization?

Accepted Answer

Scale (gamma) and shift (beta)

Answer

Mean (mu) and variance (sigma)

Answer

Learning rate (alpha) and momentum (beta)

Answer

Weight decay (lambda) and dropout rate (p)

Question 3

What problem does batch normalization primarily address?

Accepted Answer

Internal covariate shift across layers

Answer

Vanishing gradients in shallow networks

Answer

Overfitting due to excessive parameters

Answer

Exploding activations in pooling layers

Question 4

During inference, how does batch normalization behave differently compared to training?

Accepted Answer

It uses population statistics (running mean and variance) instead of batch statistics

Answer

It disables all normalization and passes activations unchanged

Answer

It uses a larger batch to compute more accurate statistics

Answer

It applies normalization only to the first and last layers

Question 5

Where is batch normalization typically inserted in a convolutional block?

Accepted Answer

Before the activation function, after the convolutional layer

Answer

After the activation function, before the pooling layer

Answer

Before the convolutional layer, after the pooling layer

Answer

Between the pooling layer and the fully connected layer only

Question 6

How does batch normalization generally affect the training speed of a CNN?

Accepted Answer

It allows higher learning rates and accelerates convergence

Answer

It slows training by adding extra computation at each step

Answer

It has no measurable effect on training speed

Answer

It speeds up forward passes but significantly slows backpropagation

Question 7

What is a known limitation of batch normalization when using very small batch sizes?

Accepted Answer

Batch statistics become noisy and unreliable, degrading performance

Answer

Learnable parameters gamma and beta become unstable

Answer

The running mean is reset to zero after each batch

Answer

It cannot be used with convolutional layers, only fully connected layers