CNN Training and Optimization 4

Question 1

What is 'cosine annealing' in CNN learning rate scheduling?

Accepted Answer

Decreasing the learning rate following a cosine curve from maximum to minimum

Answer

Cosine annealing smoothly reduces the learning rate following half a cosine cycle from an initial maximum to a minimum value, enabling gradual and smooth convergence.

Question 2

During CNN training, what is the purpose of the 'learning rate warmup' phase?

Accepted Answer

To gradually increase the learning rate from near-zero before using the main schedule

Answer

Warmup slowly increases the learning rate from a very small value, stabilizing early training when weights and gradients are poorly scaled before the main learning rate schedule begins.

Question 3

What is 'knowledge distillation' in the context of training CNNs?

Accepted Answer

Training a smaller student model to mimic the outputs of a larger teacher model

Answer

Knowledge distillation trains a compact student CNN to reproduce the soft probability outputs of a larger pretrained teacher, transferring generalization capability to the smaller model.

Question 4

Which data augmentation technique mixes two training images and their labels by linear interpolation to improve CNN generalization?

Accepted Answer

Mixup

Answer

Mixup creates virtual training samples by interpolating pixel values and labels of two random images, encouraging the model to learn more linear behavior between classes.

Question 5

What does 'gradient accumulation' allow during CNN training with limited GPU memory?

Accepted Answer

Using a larger effective batch size by accumulating gradients over multiple mini-batches before updating

Answer

Gradient accumulation sums gradients over several small mini-batches before performing a weight update, simulating a larger batch size without requiring additional GPU memory.

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Practice Test

CNN Training and Optimization 4