Neural Network Recurrent Neural Networks 2

Question 1

What is a sequence-to-sequence (seq2seq) model and what tasks is it used for?

Accepted Answer

An encoder-decoder RNN architecture that maps variable-length input sequences to variable-length output sequences

Answer

Seq2seq uses an encoder RNN to compress an input sequence into a context vector, and a decoder RNN to generate the output sequence, enabling tasks like machine translation.

Question 2

What problem does the attention mechanism solve in seq2seq models?

Accepted Answer

It allows the decoder to focus on relevant encoder states at each step instead of relying on a single context vector

Answer

Attention solves the bottleneck where all input information must pass through a single fixed-size context vector, by allowing the decoder to selectively attend to encoder hidden states.

Question 3

In a many-to-one RNN configuration, what type of task is being performed?

Accepted Answer

Classifying an entire sequence into a single output label

Answer

Many-to-one RNNs process an entire input sequence and produce a single output, used in tasks like sentiment classification where a whole sentence produces one sentiment label.

Question 4

What is teacher forcing in RNN training?

Accepted Answer

Feeding ground-truth previous tokens as decoder inputs during training instead of the model's own predictions

Answer

Teacher forcing feeds the actual ground-truth token at each decoder step during training rather than the model's predicted token, accelerating convergence but potentially causing exposure bias.

Question 5

What is beam search in the context of RNN text generation?

Accepted Answer

A decoding strategy that maintains the top-k most probable sequences at each step

Answer

Beam search keeps the top-k (beam width) most probable partial sequences at each decoding step, trading off between greedy decoding and exhaustive search.

Neural Network Practice Test

Neural Network Practice Test

Neural Network Recurrent Neural Networks 2