MS-DS Master of Data science Deep Learning and Neural Networks 2 — Questions and Answers
Question 1: What is the role of the attention mechanism in transformer-based deep learning models?
- Regularize weights
- Allow the model to weigh the relevance of each input token when producing an output (Correct answer)
- Reduce the number of parameters
- Replace activation functions
Correct answer: Allow the model to weigh the relevance of each input token when producing an output
Attention computes a weighted sum of value vectors, where weights reflect how relevant each key-query pair is, enabling long-range dependency modeling.
Question 2: Which optimizer adaptively adjusts learning rates for each parameter using estimates of first and second moments of the gradients?
- SGD with momentum
- RMSProp
- Adam (Correct answer)
- Adagrad
Correct answer: Adam
Adam combines momentum (first moment) and RMSProp (second moment) to compute adaptive per-parameter learning rates, making it robust across many tasks.
Question 3: In a generative adversarial network (GAN), what does the discriminator try to do?
- Generate realistic samples
- Minimize reconstruction error
- Distinguish real data from generated data (Correct answer)
- Encode data into a latent space
Correct answer: Distinguish real data from generated data
The discriminator is a binary classifier trained to output high probability for real samples and low probability for samples generated by the generator.
Question 4: What is batch normalization primarily used for in deep neural networks?
- Preventing overfitting by dropping neurons
- Normalizing layer inputs to stabilize and accelerate training (Correct answer)
- Reducing the number of parameters
- Clipping gradient magnitudes
Correct answer: Normalizing layer inputs to stabilize and accelerate training
Batch normalization normalizes each mini-batch's activations to zero mean and unit variance, reducing internal covariate shift and allowing higher learning rates.
Question 5: Which regularization technique randomly deactivates a fraction of neurons during each training step to reduce overfitting?
- L2 regularization
- Dropout (Correct answer)
- Early stopping
- Data augmentation
Correct answer: Dropout
Dropout randomly zeros neuron activations with probability p during training, forcing the network to learn redundant representations and reducing co-adaptation.
Question 6: In deep learning, what does the term 'epoch' refer to?
- A single gradient update
- One complete pass through the entire training dataset (Correct answer)
- The number of layers in the network
- The learning rate schedule
Correct answer: One complete pass through the entire training dataset
An epoch is one full cycle through all training samples; multiple epochs allow the model to refine its weights by seeing the data repeatedly.
MS-DS Master of Data science Deep Learning and Neural Networks 2
What is the role of the attention mechanism in transformer-based deep learning models?