CNN Batch Normalization and Dropout 2

Question 1

What does a dropout layer do during training?

Accepted Answer

Randomly sets a fraction of neuron activations to zero

Answer

During training, dropout randomly sets a fraction of activations to zero with probability p (the dropout rate), forcing the network to learn redundant representations.

Question 2

What happens to dropout during model inference/testing?

Accepted Answer

All neurons are active and outputs are scaled by the keep probability

Answer

At inference time, dropout is disabled and all neurons are active; outputs are typically scaled by the keep probability (1 - dropout rate) to match expected values during training.

Question 3

What is 'inverted dropout' and why is it preferred in practice?

Accepted Answer

Scaling kept activations by 1/(1-p) during training so no scaling is needed at test time

Answer

Inverted dropout scales kept activations by 1/(1-p) during training, so the network's expected output magnitude is unchanged and no scaling adjustment is needed at inference time.

Question 4

How does dropout act as a form of regularization?

Accepted Answer

By forcing the network to learn robust features not dependent on specific co-adaptations

Answer

Dropout prevents co-adaptation of neurons by randomly removing them, forcing each neuron to learn useful features independently, which improves generalization.

Question 5

What is spatial dropout (also called 2D dropout) and when is it used?

Accepted Answer

Dropping entire feature maps (channels) rather than individual activations in CNNs

Answer

Spatial dropout drops entire feature maps (channels) at once rather than individual elements, which is more effective for CNNs because adjacent pixels in a feature map are highly correlated.

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Practice Test

CNN Batch Normalization and Dropout 2