AML Deep Learning & Neural Networks 2

Question 1

Which technique addresses the vanishing gradient problem by allowing gradients to flow directly through skip connections?

Accepted Answer

Residual connections (ResNets)

Answer

Residual connections let gradients bypass layers via identity shortcuts, preventing them from vanishing in very deep networks.

Question 2

In a transformer's multi-head attention, why are queries, keys, and values projected into multiple subspaces?

Accepted Answer

To allow the model to attend to information from different representation subspaces simultaneously

Answer

Multiple heads let the model jointly attend to information from different subspaces, capturing diverse relationship types in parallel.

Question 3

What is the primary purpose of the temperature parameter in a softmax output during inference?

Accepted Answer

To control the sharpness or diversity of the probability distribution

Answer

A lower temperature sharpens the distribution toward the argmax, while a higher temperature flattens it, increasing output diversity.

Question 4

Which regularization method randomly zeroes entire feature maps (channels) during training in CNNs?

Accepted Answer

Spatial dropout

Answer

Spatial dropout drops entire 2D feature maps, enforcing stronger feature independence than element-wise dropout in convolutional layers.

Question 5

In a GAN, what condition describes the theoretical equilibrium where the generator perfectly replicates the data distribution?

Accepted Answer

The discriminator outputs 0.5 for all inputs

Answer

At Nash equilibrium the discriminator cannot distinguish real from generated samples, outputting 0.5 (random chance) for every input.

(AML) Advanced Machine Learning Certification Practice Test