Deep Learning Recurrent Neural Networks 3

Question 1

What does 'stateful' mode mean for an RNN in frameworks like Keras?

Accepted Answer

The hidden state from the end of one batch is used as the initial state of the next batch

Answer

The model saves weights after every batch

Answer

The model stores training data in memory

Answer

Gradients are accumulated across multiple forward passes

Question 2

What is the purpose of the output gate in an LSTM?

Accepted Answer

Controlling how much of the cell state is exposed as the hidden state output

Answer

Deciding how much of the new input to add to cell state

Answer

Erasing old cell state information

Answer

Computing the final class probabilities

Question 3

How does an encoder-decoder architecture with attention differ from one without attention?

Accepted Answer

Attention allows the decoder to look at all encoder hidden states rather than only a single context vector

Answer

Attention removes the encoder entirely

Answer

Attention adds convolutional layers to the decoder

Answer

Attention replaces the recurrent decoder with a feedforward network

Question 4

What is the main limitation of vanilla Seq2Seq models for long input sequences?

Accepted Answer

The fixed-size context vector loses information when compressing long sequences

Answer

They cannot handle variable-length inputs

Answer

They require labeled data for every time step

Answer

They cannot be trained with gradient-based methods

Question 5

What is the exploding gradient problem in RNNs?

Accepted Answer

Gradients growing exponentially through many time steps, causing unstable large weight updates

Answer

Loss increasing to infinity due to wrong architecture

Answer

Too many neurons being activated simultaneously

Answer

Activations becoming negative during training

Question 6

What is a many-to-one RNN architecture used for?

Accepted Answer

Mapping an input sequence to a single output such as sentiment classification

Answer

Generating a sequence from a single input

Answer

Processing a single input to produce a sequence

Answer

Mapping sequences of equal length