Deep Learning Recurrent Neural Networks

Question 1

What distinguishes a recurrent neural network (RNN) from a feedforward network?

Accepted Answer

RNNs have feedback connections that carry hidden state across time steps

Answer

RNNs have no activation functions

Answer

RNNs only process fixed-size inputs

Answer

RNNs use convolutional operations

Question 2

What is the vanishing gradient problem particularly severe for in RNNs?

Accepted Answer

Long-range dependencies spanning many time steps

Answer

Short sequences of 3–5 tokens

Answer

Single-step predictions

Answer

Batch normalization in recurrent layers

Question 3

What are the key components of an LSTM cell that vanilla RNNs lack?

Accepted Answer

Forget gate, input gate, and output gate controlling information flow

Answer

Convolutional filters and pooling

Answer

Attention heads and positional encodings

Answer

Batch normalization and dropout layers

Question 4

What problem does the forget gate in an LSTM address?

Accepted Answer

Deciding how much of the previous cell state to retain or discard

Answer

Exploding activation values

Answer

Mapping hidden states to output predictions

Answer

Normalizing the input to the cell

Question 5

What is backpropagation through time (BPTT)?

Accepted Answer

Unrolling an RNN across time steps and applying backpropagation to compute gradients

Answer

Evaluating the model on future data

Answer

Using time-series data to initialize RNN weights

Answer

Applying dropout at each time step

Question 6

What is a GRU (Gated Recurrent Unit) and how does it differ from an LSTM?

Accepted Answer

A GRU simplifies LSTM by merging cell and hidden state and using only two gates

Answer

A GRU has more gates than an LSTM

Answer

A GRU uses attention instead of recurrence

Answer

A GRU replaces gradient-based learning with evolutionary methods