Machine Learning Neural Networks 2 — Questions and Answers
Question 1: What problem does batch normalization primarily solve in deep neural networks?
- Overfitting by adding noise
- Internal covariate shift during training (Correct answer)
- Vanishing gradients by clipping values
- Slow convergence by increasing learning rate
Correct answer: Internal covariate shift during training
Batch normalization addresses internal covariate shift by normalizing layer inputs, stabilizing and accelerating training.
Question 2: In a convolutional neural network, what does the stride parameter control?
- The number of filters applied
- The size of each filter kernel
- How many pixels the filter moves per step (Correct answer)
- The depth of the feature maps
Correct answer: How many pixels the filter moves per step
Stride determines how many pixels the convolutional filter shifts at each step, controlling the spatial dimensions of the output.
Question 3: Which technique randomly sets neuron outputs to zero during training to prevent overfitting?
- Weight decay
- Dropout (Correct answer)
- Early stopping
- L1 regularization
Correct answer: Dropout
Dropout randomly deactivates a fraction of neurons during each training step, forcing the network to learn redundant representations.
Question 4: What is the vanishing gradient problem?
- Gradients become NaN during backpropagation
- Gradients shrink exponentially as they propagate through many layers (Correct answer)
- The learning rate becomes too large for gradient descent
- Weight updates cause gradients to oscillate
Correct answer: Gradients shrink exponentially as they propagate through many layers
In deep networks, gradients can become exponentially small during backpropagation, making it difficult to train early layers.
Question 5: What does the ReLU activation function return for a negative input value?
- The input value unchanged
- A small negative value proportional to the input
- Zero (Correct answer)
- One
Correct answer: Zero
ReLU (Rectified Linear Unit) outputs zero for any negative input and passes positive values unchanged.
Question 6: In LSTM networks, what is the purpose of the forget gate?
- To introduce new information into the cell state
- To decide what information to discard from the cell state (Correct answer)
- To control the output sent to the next time step
- To normalize the hidden state
Correct answer: To decide what information to discard from the cell state
The forget gate uses a sigmoid function to decide which parts of the previous cell state to erase or retain.
Question 7: Which loss function is most appropriate for a multi-class classification neural network?
- Mean squared error
- Binary cross-entropy
- Categorical cross-entropy (Correct answer)
- Hinge loss
Correct answer: Categorical cross-entropy
Categorical cross-entropy measures the dissimilarity between predicted probability distributions and one-hot encoded true labels in multi-class problems.
Machine Learning Neural Networks 2
What problem does batch normalization primarily solve in deep neural networks?