Machine Learning Deep Learning 2 — Questions and Answers
Question 1: What is the vanishing gradient problem in deep neural networks?
- Gradients become too large during backpropagation
- Gradients shrink exponentially as they propagate backward through layers (Correct answer)
- The network forgets earlier training data over time
- Weights converge to zero during initialization
Correct answer: Gradients shrink exponentially as they propagate backward through layers
In deep networks, gradients can shrink exponentially during backpropagation, making it difficult to update weights in early layers.
Question 2: Which activation function is most commonly used in hidden layers of modern deep neural networks to mitigate the vanishing gradient problem?
- Sigmoid
- Tanh
- ReLU (Correct answer)
- Softmax
Correct answer: ReLU
ReLU (Rectified Linear Unit) outputs zero for negative inputs and the input itself for positives, avoiding gradient saturation in most cases.
Question 3: What does batch normalization do in a neural network?
- Reduces the number of batches needed during training
- Normalizes activations within each mini-batch to stabilize training (Correct answer)
- Randomly drops neurons to prevent overfitting
- Scales the learning rate based on batch size
Correct answer: Normalizes activations within each mini-batch to stabilize training
Batch normalization normalizes layer inputs within each mini-batch, which reduces internal covariate shift and speeds up training.
Question 4: In a convolutional neural network, what is the purpose of a pooling layer?
- To add more learnable parameters
- To apply non-linear activation functions
- To downsample feature maps and reduce spatial dimensions (Correct answer)
- To normalize batch statistics
Correct answer: To downsample feature maps and reduce spatial dimensions
Pooling layers reduce the spatial size of feature maps, decreasing computation and helping the network become more spatially invariant.
Question 5: What is a skip connection (residual connection) in deep learning?
- A connection that skips the activation function
- A shortcut path that adds a layer's input directly to its output (Correct answer)
- A dropout mask applied across multiple layers
- A connection that skips regularization
Correct answer: A shortcut path that adds a layer's input directly to its output
Skip connections allow gradients to flow directly through the network, enabling training of very deep networks by alleviating the vanishing gradient problem.
Question 6: Which loss function is typically used for binary classification tasks in deep learning?
- Mean squared error
- Categorical cross-entropy
- Binary cross-entropy (Correct answer)
- Hinge loss
Correct answer: Binary cross-entropy
Binary cross-entropy measures the difference between the predicted probability and the true binary label for two-class classification.
Question 7: What is the role of the softmax function in a multi-class classification neural network?
- It introduces non-linearity in hidden layers
- It converts raw logits into a probability distribution over classes (Correct answer)
- It normalizes the input data before training
- It selects the top-k predictions from the output
Correct answer: It converts raw logits into a probability distribution over classes
Softmax exponentiates and normalizes logits so that output values sum to 1, representing class probabilities.
Machine Learning Deep Learning 2
What is the vanishing gradient problem in deep neural networks?