MS-DS Master of Data science Deep Learning and Neural Networks 2

Question 1

What is the role of the attention mechanism in transformer-based deep learning models?

Accepted Answer

Allow the model to weigh the relevance of each input token when producing an output

Answer

Regularize weights

Answer

Reduce the number of parameters

Answer

Replace activation functions

Question 2

Which optimizer adaptively adjusts learning rates for each parameter using estimates of first and second moments of the gradients?

Accepted Answer

Adam

Answer

SGD with momentum

Answer

RMSProp

Answer

Adagrad

Question 3

In a generative adversarial network (GAN), what does the discriminator try to do?

Accepted Answer

Distinguish real data from generated data

Answer

Generate realistic samples

Answer

Minimize reconstruction error

Answer

Encode data into a latent space

Question 4

What is batch normalization primarily used for in deep neural networks?

Accepted Answer

Normalizing layer inputs to stabilize and accelerate training

Answer

Preventing overfitting by dropping neurons

Answer

Reducing the number of parameters

Answer

Clipping gradient magnitudes

Question 5

Which regularization technique randomly deactivates a fraction of neurons during each training step to reduce overfitting?

Accepted Answer

Dropout

Answer

L2 regularization

Answer

Early stopping

Answer

Data augmentation

Question 6

In deep learning, what does the term 'epoch' refer to?

Accepted Answer

One complete pass through the entire training dataset

Answer

A single gradient update

Answer

The number of layers in the network

Answer

The learning rate schedule