Deep Learning and Neural Networks Flashcards
6 cards from real MS-DS Master of Data science practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 6 Deep Learning and Neural Networks flashcards as text
Which activation function is most commonly used in hidden layers of deep neural networks to mitigate the vanishing gradient problem?
Answer: ReLU
ReLU (Rectified Linear Unit) avoids vanishing gradients because its gradient is 1 for positive inputs, enabling deeper networks to train effectively.
In a convolutional neural network (CNN), what is the primary purpose of a pooling layer?
Answer: Reduce spatial dimensions and computation
Pooling layers downsample feature maps, reducing spatial size and computational cost while providing a degree of translation invariance.
What technique is used in recurrent neural networks (RNNs) to address exploding gradients during backpropagation through time?
Answer: Gradient clipping
Gradient clipping caps gradient norms at a threshold, preventing them from growing unbounded during BPTT in RNNs.
Which component of an LSTM cell is responsible for deciding what information to discard from the cell state?
Answer: Forget gate
The forget gate uses a sigmoid function to output values between 0 and 1, determining how much of the previous cell state to retain.
In deep learning, what does 'transfer learning' refer to?
Answer: Using a pre-trained model's weights as the starting point for a new task
Transfer learning leverages a model pre-trained on a large dataset (e.g., ImageNet) and fine-tunes it on a smaller, task-specific dataset.
Which loss function is standard for multi-class classification in neural networks?
Answer: Categorical cross-entropy
Categorical cross-entropy measures the divergence between the predicted probability distribution and the one-hot encoded true labels across multiple classes.