MS-DS Master of Data science Deep Learning and Neural Networks 2

Question 1

What is the role of the attention mechanism in transformer-based deep learning models?

Accepted Answer

Allow the model to weigh the relevance of each input token when producing an output

Answer

Attention computes a weighted sum of value vectors, where weights reflect how relevant each key-query pair is, enabling long-range dependency modeling.

Question 2

Which optimizer adaptively adjusts learning rates for each parameter using estimates of first and second moments of the gradients?

Accepted Answer

Adam

Answer

Adam combines momentum (first moment) and RMSProp (second moment) to compute adaptive per-parameter learning rates, making it robust across many tasks.

Question 3

In a generative adversarial network (GAN), what does the discriminator try to do?

Accepted Answer

Distinguish real data from generated data

Answer

The discriminator is a binary classifier trained to output high probability for real samples and low probability for samples generated by the generator.

Question 4

What is batch normalization primarily used for in deep neural networks?

Accepted Answer

Normalizing layer inputs to stabilize and accelerate training

Answer

Batch normalization normalizes each mini-batch's activations to zero mean and unit variance, reducing internal covariate shift and allowing higher learning rates.

Question 5

Which regularization technique randomly deactivates a fraction of neurons during each training step to reduce overfitting?

Accepted Answer

Dropout

Answer

Dropout randomly zeros neuron activations with probability p during training, forcing the network to learn redundant representations and reducing co-adaptation.

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Deep Learning and Neural Networks 2