Deep Learning Recurrent Neural Networks

Question 1

What distinguishes a recurrent neural network (RNN) from a feedforward network?

Accepted Answer

RNNs have feedback connections that carry hidden state across time steps

Answer

RNNs pass a hidden state from one time step to the next, allowing the network to maintain memory of previous inputs in a sequence.

Question 2

What is the vanishing gradient problem particularly severe for in RNNs?

Accepted Answer

Long-range dependencies spanning many time steps

Answer

In RNNs, gradients must be backpropagated through many time steps, causing them to shrink exponentially and preventing the network from learning long-range dependencies.

Question 3

What are the key components of an LSTM cell that vanilla RNNs lack?

Accepted Answer

Forget gate, input gate, and output gate controlling information flow

Answer

LSTMs introduce a cell state and three gating mechanisms — forget, input, and output — that regulate what information is retained, added, or read from memory.

Question 4

What problem does the forget gate in an LSTM address?

Accepted Answer

Deciding how much of the previous cell state to retain or discard

Answer

The forget gate outputs values between 0 and 1 for each cell state element, controlling how much prior context is carried forward and what is discarded.

Question 5

What is backpropagation through time (BPTT)?

Accepted Answer

Unrolling an RNN across time steps and applying backpropagation to compute gradients

Answer

BPTT unfolds the RNN computation graph across all time steps and applies the standard backpropagation algorithm to compute gradients with respect to all shared weights.

Deep Learning Practice Test

Deep Learning Practice Test

Deep Learning Recurrent Neural Networks