AML Deep Learning & Neural Networks 2 — Questions and Answers
Question 1: Which technique addresses the vanishing gradient problem by allowing gradients to flow directly through skip connections?
- Dropout regularization
- Residual connections (ResNets) (Correct answer)
- Batch normalization alone
- Weight decay
Correct answer: Residual connections (ResNets)
Residual connections let gradients bypass layers via identity shortcuts, preventing them from vanishing in very deep networks.
Question 2: In a transformer's multi-head attention, why are queries, keys, and values projected into multiple subspaces?
- To reduce memory usage during inference
- To allow the model to attend to information from different representation subspaces simultaneously (Correct answer)
- To enforce positional encoding constraints
- To apply dropout independently per head
Correct answer: To allow the model to attend to information from different representation subspaces simultaneously
Multiple heads let the model jointly attend to information from different subspaces, capturing diverse relationship types in parallel.
Question 3: What is the primary purpose of the temperature parameter in a softmax output during inference?
- To speed up gradient computation
- To control the sharpness or diversity of the probability distribution (Correct answer)
- To normalize batch statistics
- To clip exploding gradients
Correct answer: To control the sharpness or diversity of the probability distribution
A lower temperature sharpens the distribution toward the argmax, while a higher temperature flattens it, increasing output diversity.
Question 4: Which regularization method randomly zeroes entire feature maps (channels) during training in CNNs?
- Spatial dropout (Correct answer)
- L2 weight decay
- Label smoothing
- Gradient clipping
Correct answer: Spatial dropout
Spatial dropout drops entire 2D feature maps, enforcing stronger feature independence than element-wise dropout in convolutional layers.
Question 5: In a GAN, what condition describes the theoretical equilibrium where the generator perfectly replicates the data distribution?
- The discriminator outputs 0.5 for all inputs (Correct answer)
- The generator loss reaches zero
- The discriminator correctly classifies 100% of real images
- The gradient penalty term converges to one
Correct answer: The discriminator outputs 0.5 for all inputs
At Nash equilibrium the discriminator cannot distinguish real from generated samples, outputting 0.5 (random chance) for every input.
Question 6: What does 'depthwise separable convolution' decompose a standard convolution into?
- A depthwise convolution followed by a pointwise (1×1) convolution (Correct answer)
- Two sequential 3×3 convolutions
- A strided convolution followed by max pooling
- An atrous convolution followed by batch normalization
Correct answer: A depthwise convolution followed by a pointwise (1×1) convolution
Depthwise separable convolutions apply a single filter per channel then combine channels with 1×1 convolutions, dramatically reducing parameters.
Question 7: Which loss function is most appropriate for training a neural network on a multi-label classification task where each sample can belong to multiple classes?
- Categorical cross-entropy with softmax
- Binary cross-entropy applied independently per label (Correct answer)
- Mean squared error
- Hinge loss with multi-class margin
Correct answer: Binary cross-entropy applied independently per label
Binary cross-entropy treats each label independently with a sigmoid activation, allowing multiple classes to be simultaneously positive.
AML Deep Learning & Neural Networks 2
Which technique addresses the vanishing gradient problem by allowing gradients to flow directly through skip connections?