Machine Learning Deep Learning 2

Question 1

What is the vanishing gradient problem in deep neural networks?

Accepted Answer

Gradients shrink exponentially as they propagate backward through layers

Answer

In deep networks, gradients can shrink exponentially during backpropagation, making it difficult to update weights in early layers.

Question 2

Which activation function is most commonly used in hidden layers of modern deep neural networks to mitigate the vanishing gradient problem?

Accepted Answer

ReLU

Answer

ReLU (Rectified Linear Unit) outputs zero for negative inputs and the input itself for positives, avoiding gradient saturation in most cases.

Question 3

What does batch normalization do in a neural network?

Accepted Answer

Normalizes activations within each mini-batch to stabilize training

Answer

Batch normalization normalizes layer inputs within each mini-batch, which reduces internal covariate shift and speeds up training.

Question 4

In a convolutional neural network, what is the purpose of a pooling layer?

Accepted Answer

To downsample feature maps and reduce spatial dimensions

Answer

Pooling layers reduce the spatial size of feature maps, decreasing computation and helping the network become more spatially invariant.

Question 5

What is a skip connection (residual connection) in deep learning?

Accepted Answer

A shortcut path that adds a layer's input directly to its output

Answer

Skip connections allow gradients to flow directly through the network, enabling training of very deep networks by alleviating the vanishing gradient problem.

Machine Learning Practice Test

Machine Learning Practice Test

Machine Learning Deep Learning 2