MS-DS Master of Data science Deep Learning and Neural Networks 1 — Questions and Answers
Question 1: Which activation function is most commonly used in hidden layers of deep neural networks to mitigate the vanishing gradient problem?
- Sigmoid
- Tanh
- ReLU (Correct answer)
- Softmax
Correct answer: ReLU
ReLU (Rectified Linear Unit) avoids vanishing gradients because its gradient is 1 for positive inputs, enabling deeper networks to train effectively.
Question 2: In a convolutional neural network (CNN), what is the primary purpose of a pooling layer?
- Add non-linearity
- Reduce spatial dimensions and computation (Correct answer)
- Normalize activations
- Increase filter depth
Correct answer: Reduce spatial dimensions and computation
Pooling layers downsample feature maps, reducing spatial size and computational cost while providing a degree of translation invariance.
Question 3: What technique is used in recurrent neural networks (RNNs) to address exploding gradients during backpropagation through time?
- Dropout regularization
- Batch normalization
- Gradient clipping (Correct answer)
- Weight decay
Correct answer: Gradient clipping
Gradient clipping caps gradient norms at a threshold, preventing them from growing unbounded during BPTT in RNNs.
Question 4: Which component of an LSTM cell is responsible for deciding what information to discard from the cell state?
- Input gate
- Output gate
- Forget gate (Correct answer)
- Memory gate
Correct answer: Forget gate
The forget gate uses a sigmoid function to output values between 0 and 1, determining how much of the previous cell state to retain.
Question 5: In deep learning, what does 'transfer learning' refer to?
- Moving model weights between GPUs
- Using a pre-trained model's weights as the starting point for a new task (Correct answer)
- Transferring data across training epochs
- Copying hyperparameters from one model to another
Correct answer: Using a pre-trained model's weights as the starting point for a new task
Transfer learning leverages a model pre-trained on a large dataset (e.g., ImageNet) and fine-tunes it on a smaller, task-specific dataset.
Question 6: Which loss function is standard for multi-class classification in neural networks?
- Mean squared error
- Binary cross-entropy
- Categorical cross-entropy (Correct answer)
- Huber loss
Correct answer: Categorical cross-entropy
Categorical cross-entropy measures the divergence between the predicted probability distribution and the one-hot encoded true labels across multiple classes.
MS-DS Master of Data science Deep Learning and Neural Networks 1
Which activation function is most commonly used in hidden layers of deep neural networks to mitigate the vanishing gradient problem?