Natural Language Processing Machine Translation and Sequence-to-Sequence Models

Question 1

What is the encoder-decoder architecture in NLP?

Accepted Answer

A model where the encoder compresses an input sequence into a context representation and the decoder generates the output sequence from it

Answer

Encoder-decoder models (Seq2Seq) map variable-length input sequences to variable-length output sequences, forming the backbone of machine translation, summarization, and dialogue.

Question 2

What is the BLEU score used to evaluate in NLP?

Accepted Answer

Machine translation quality by measuring n-gram overlap between the generated output and human reference translations

Answer

BLEU (Bilingual Evaluation Understudy) computes a modified n-gram precision between the hypothesis and references, applying a brevity penalty for short outputs.

Question 3

What is the attention mechanism's key benefit in neural machine translation?

Accepted Answer

It allows the decoder to focus on relevant parts of the source sentence at each decoding step, overcoming the fixed-length bottleneck

Answer

Bahdanau-style attention lets the decoder dynamically reweight encoder hidden states, solving the information compression bottleneck of fixed-size context vectors in long sentences.

Question 4

What is beam search used for in sequence generation?

Accepted Answer

Exploring multiple candidate output sequences simultaneously and returning the most probable one

Answer

Beam search maintains a fixed-size set of the most probable partial sequences at each step, balancing exploration and computational cost better than greedy decoding.

Question 5

What is the main difference between statistical machine translation (SMT) and neural machine translation (NMT)?

Accepted Answer

SMT uses probabilistic phrase-based models with separately trained components, while NMT uses end-to-end neural networks trained jointly

Answer

SMT pipelines a language model, translation model, and reordering model trained separately, while NMT learns all transformations jointly in a single neural network.

Natural Language Processing Practice Test

Natural Language Processing Practice Test

Natural Language Processing Machine Translation and Sequence-to-Sequence Models