AML Deep Learning & Neural Networks 3

Question 1

What is 'gradient checkpointing' used for in deep learning training?

Accepted Answer

Trading compute for memory by recomputing activations during the backward pass

Answer

Gradient checkpointing discards intermediate activations and recomputes them during backpropagation, reducing memory at the cost of extra computation.

Question 2

In the LSTM architecture, which gate controls how much of the previous cell state is retained?

Accepted Answer

Forget gate

Answer

The forget gate applies a sigmoid-activated mask to the previous cell state, determining what proportion of historical information to carry forward.

Question 3

What distinguishes 'instance normalization' from 'batch normalization'?

Accepted Answer

Instance normalization normalizes each sample's spatial dimensions independently of other samples in the batch

Answer

Instance normalization computes mean and variance per sample per channel, making it batch-size-independent and preferred in style transfer tasks.

Question 4

Which concept in neural architecture search (NAS) allows gradient-based optimization of the architecture by making the search space continuous?

Accepted Answer

DARTS (Differentiable Architecture Search)

Answer

DARTS relaxes the discrete architecture choice into a continuous softmax over candidate operations, enabling end-to-end gradient optimization.

Question 5

In contrastive learning (e.g., SimCLR), what is the role of the projection head?

Accepted Answer

To map representations to a space where the contrastive loss is applied, then discarded at downstream fine-tuning

Answer

The projection head is a small MLP used only during pretraining to compute the contrastive loss; the backbone encoder is used for downstream tasks.

(AML) Advanced Machine Learning Certification Practice Test

(AML) Advanced Machine Learning Certification Practice Test

AML Deep Learning & Neural Networks 3