Machine Learning Deep Learning 3

Question 1

What distinguishes a recurrent neural network (RNN) from a standard feedforward network?

Accepted Answer

RNNs have connections that loop back, giving them memory of previous inputs

Answer

RNNs have recurrent connections that pass hidden state from one time step to the next, enabling them to model sequential data.

Question 2

What problem does an LSTM network solve compared to a vanilla RNN?

Accepted Answer

It handles longer-term dependencies by using gating mechanisms

Answer

LSTM's cell state and gates (input, forget, output) allow it to retain or discard information over long sequences, mitigating the vanishing gradient problem in RNNs.

Question 3

In the transformer architecture, what does 'self-attention' compute?

Accepted Answer

Weighted relationships between all positions in a sequence simultaneously

Answer

Self-attention computes a weighted sum of all token representations, where weights reflect pairwise relevance between tokens.

Question 4

Why is positional encoding added to token embeddings in a transformer?

Accepted Answer

To inject information about the order of tokens since transformers have no inherent sequential structure

Answer

Transformers process all tokens in parallel and have no notion of order, so positional encodings provide sequence position information.

Question 5

What is weight initialization, and why does it matter in deep learning?

Accepted Answer

Choosing initial weight values to prevent vanishing/exploding gradients and speed convergence

Answer

Poor initialization (e.g., all zeros) causes symmetry problems; methods like Xavier or He initialization keep activations and gradients in a healthy range.

Machine Learning Practice Test

Machine Learning Practice Test

Machine Learning Deep Learning 3