CNN Training and Optimization 2

Question 1

What does the learning rate scheduler 'ReduceLROnPlateau' do when validation loss stops improving?

Accepted Answer

Reduces the learning rate by a factor

Answer

ReduceLROnPlateau monitors a metric and reduces the learning rate by a specified factor when no improvement is seen for a patience number of epochs.

Question 2

Which technique adds Gaussian noise to the gradients during CNN training to help escape sharp local minima?

Accepted Answer

Stochastic gradient noise

Answer

Stochastic gradient noise injects Gaussian noise into gradients during optimization, helping the model escape sharp local minima and find flatter, more generalizable optima.

Question 3

In CNN training, what does 'warm restarts' refer to in the context of learning rate schedules?

Accepted Answer

Periodically resetting the learning rate to a high value then annealing it

Answer

Warm restarts (SGDR) periodically resets the learning rate to a maximum value and then follows a cosine annealing schedule, helping the model explore different loss landscape regions.

Question 4

What is the primary benefit of using mixed-precision training (FP16 + FP32) in CNN optimization?

Accepted Answer

Reduced training time and memory usage with minimal accuracy loss

Answer

Mixed-precision training uses 16-bit floats for most operations and 32-bit for critical accumulations, significantly reducing memory footprint and speeding up computation on modern GPUs.

Question 5

During CNN training, which regularization method randomly drops entire feature maps rather than individual neurons?

Accepted Answer

DropBlock

Answer

DropBlock drops contiguous regions of feature maps, which is more effective for CNNs than standard dropout because adjacent units in feature maps tend to be correlated.

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Practice Test

CNN Training and Optimization 2