Neural Network Recurrent Neural Networks

Question 1

What makes recurrent neural networks (RNNs) different from feedforward networks?

Accepted Answer

They maintain a hidden state that carries information across time steps

Answer

RNNs have recurrent connections that pass the hidden state from one time step to the next, allowing them to model sequential dependencies in variable-length inputs.

Question 2

What does LSTM stand for and what problem was it designed to solve?

Accepted Answer

Long Short-Term Memory; vanishing/exploding gradients in standard RNNs

Answer

LSTM (Long Short-Term Memory) was designed by Hochreiter and Schmidhuber to address vanishing and exploding gradients in standard RNNs by using gated cell states.

Question 3

What are the three gates in an LSTM cell?

Accepted Answer

Forget, input, and output gates

Answer

An LSTM cell has forget gate (what to discard from cell state), input gate (what new information to add), and output gate (what to output as hidden state).

Question 4

What is a bidirectional RNN?

Accepted Answer

An RNN that processes data both forwards and backwards in time simultaneously

Answer

A bidirectional RNN processes sequences in both forward and backward temporal directions, concatenating or summing the two hidden states to capture context from both past and future.

Question 5

What is Backpropagation Through Time (BPTT)?

Accepted Answer

The algorithm for computing gradients in RNNs by unrolling the network across time steps

Answer

BPTT unrolls the RNN across all time steps and applies standard backpropagation, computing gradients with respect to each time step's weights.

Neural Network Practice Test

Neural Network Practice Test

Neural Network Recurrent Neural Networks