Deep Learning (Data Scientist) 3

Question 1

A data scientist training an RNN on long text sequences finds gradients shrinking toward zero in early timesteps. Which architecture change best addresses this?

Accepted Answer

Replace vanilla RNN cells with LSTM or GRU cells

Answer

LSTM and GRU gating mechanisms preserve gradient flow across long sequences, mitigating the vanishing gradient problem.

Question 2

In the Transformer architecture, what is the primary purpose of the self-attention mechanism?

Accepted Answer

To weigh the relevance of every token to every other token in a sequence

Answer

Self-attention computes pairwise relevance scores so each token's representation incorporates context from the entire sequence.

Question 3

Why do Transformers require positional encodings?

Accepted Answer

Self-attention is permutation-invariant and has no inherent sense of token order

Answer

Without positional encodings, attention treats the input as an unordered set, losing sequence order information.

Question 4

A convolutional layer has a 3x3 kernel, stride 1, and 'same' padding applied to a 64x64 input. What is the spatial size of the output?

Accepted Answer

64x64

Answer

'Same' padding with stride 1 preserves the input's spatial dimensions.

Question 5

Which optimizer combines momentum with per-parameter adaptive learning rates derived from first and second moment estimates?

Accepted Answer

Adam

Answer

Adam maintains exponential moving averages of both gradients and squared gradients to adapt each parameter's step size.

Deep Learning Practice Test

Deep Learning Practice Test

Deep Learning (Data Scientist) 3