CNN Training and Optimization 3

Question 1

In transfer learning for CNNs, what is 'fine-tuning' as opposed to 'feature extraction'?

Accepted Answer

Updating all or some pretrained weights on the new task

Answer

Fine-tuning unfreezes some or all pretrained layers and continues training on the new dataset, allowing the model to adapt its representations to the target task.

Question 2

What is the vanishing gradient problem in deep CNNs and which activation function helps mitigate it?

Accepted Answer

Gradients shrink toward zero in early layers; ReLU helps

Answer

Vanishing gradients occur when gradients diminish exponentially during backpropagation through many layers; ReLU avoids saturation for positive inputs, preserving gradient magnitude.

Question 3

What does the 'momentum' hyperparameter do in SGD with momentum for CNN training?

Accepted Answer

Accumulates a fraction of past gradients to accelerate optimization

Answer

Momentum accumulates an exponentially decaying moving average of past gradients, helping accelerate SGD in relevant directions and dampening oscillations.

Question 4

Which CNN training technique uses multiple GPUs where each device holds a copy of the model and processes a different mini-batch?

Accepted Answer

Data parallelism

Answer

Data parallelism replicates the model on each GPU and splits mini-batches across devices, synchronizing gradients after each forward-backward pass.

Question 5

What is label smoothing in CNN training and what problem does it address?

Accepted Answer

Augmenting labels with noise to prevent overfitting to overconfident predictions

Answer

Label smoothing replaces hard 0/1 targets with soft values (e.g., 0.9/0.1), preventing the model from becoming overconfident and improving generalization.

CNN - Convolutional Neural Networks Practice Test

CNN - Convolutional Neural Networks Practice Test

CNN Training and Optimization 3