Neural Network Backpropagation and Training

Question 1

What is the vanishing gradient problem in deep neural networks?

Accepted Answer

Gradients become extremely small and fail to update early layers

Answer

Gradients become too large during backpropagation

Answer

The loss function produces zero gradients

Answer

Weight updates cause NaN values

Question 2

Which technique clips gradient magnitudes to prevent exploding gradients during training?

Accepted Answer

Gradient clipping

Answer

Dropout

Answer

Weight decay

Answer

Batch normalization

Question 3

In mini-batch gradient descent, what happens to the gradient estimate as batch size increases?

Accepted Answer

It becomes a more accurate estimate of the true gradient

Answer

It becomes noisier

Answer

It always leads to faster convergence

Answer

It eliminates the need for learning rate tuning

Question 4

What does the learning rate control in neural network training?

Accepted Answer

The step size taken in the direction of the negative gradient

Answer

The number of training epochs

Answer

The proportion of neurons dropped out

Answer

The ratio of training to validation data

Question 5

What is the purpose of momentum in gradient descent optimization?

Accepted Answer

To accumulate past gradients to accelerate and smooth updates

Answer

To reduce the learning rate over time

Answer

To randomly perturb weights during training

Answer

To normalize gradients before applying updates

Question 6

What does the chain rule of calculus enable in backpropagation?

Accepted Answer

Computing gradients through composite functions layer by layer

Answer

Parallel computation across layers

Answer

Storing activations for forward pass reuse

Answer

Regularizing weight magnitudes automatically