Natural Language Processing Machine Translation and Sequence-to-Sequence Models 1 — Questions and Answers
Question 1: What is the encoder-decoder architecture in NLP?
- A model where the encoder compresses an input sequence into a context representation and the decoder generates the output sequence from it (Correct answer)
- A model that only encodes text without generating output
- A two-layer feed-forward network for text classification
- A model that encodes text as images for multimodal tasks
Correct answer: A model where the encoder compresses an input sequence into a context representation and the decoder generates the output sequence from it
Encoder-decoder models (Seq2Seq) map variable-length input sequences to variable-length output sequences, forming the backbone of machine translation, summarization, and dialogue.
Question 2: What is the BLEU score used to evaluate in NLP?
- Machine translation quality by measuring n-gram overlap between the generated output and human reference translations (Correct answer)
- Sentiment analysis accuracy on benchmark datasets
- The perplexity of a language model on a held-out corpus
- Named entity recognition F1 on CoNLL datasets
Correct answer: Machine translation quality by measuring n-gram overlap between the generated output and human reference translations
BLEU (Bilingual Evaluation Understudy) computes a modified n-gram precision between the hypothesis and references, applying a brevity penalty for short outputs.
Question 3: What is the attention mechanism's key benefit in neural machine translation?
- It allows the decoder to focus on relevant parts of the source sentence at each decoding step, overcoming the fixed-length bottleneck (Correct answer)
- It reduces the vocabulary size of the target language model
- It speeds up training by reducing the number of parameters
- It eliminates the need for parallel bilingual training data
Correct answer: It allows the decoder to focus on relevant parts of the source sentence at each decoding step, overcoming the fixed-length bottleneck
Bahdanau-style attention lets the decoder dynamically reweight encoder hidden states, solving the information compression bottleneck of fixed-size context vectors in long sentences.
Question 4: What is beam search used for in sequence generation?
- Exploring multiple candidate output sequences simultaneously and returning the most probable one (Correct answer)
- Finding the shortest path in a dependency parse tree
- Selecting the best training batch based on difficulty
- Pruning the vocabulary to only the top k most frequent words
Correct answer: Exploring multiple candidate output sequences simultaneously and returning the most probable one
Beam search maintains a fixed-size set of the most probable partial sequences at each step, balancing exploration and computational cost better than greedy decoding.
Question 5: What is the main difference between statistical machine translation (SMT) and neural machine translation (NMT)?
- SMT uses probabilistic phrase-based models with separately trained components, while NMT uses end-to-end neural networks trained jointly (Correct answer)
- SMT requires more data than NMT
- NMT cannot handle morphologically rich languages
- SMT is faster at inference time than NMT
Correct answer: SMT uses probabilistic phrase-based models with separately trained components, while NMT uses end-to-end neural networks trained jointly
SMT pipelines a language model, translation model, and reordering model trained separately, while NMT learns all transformations jointly in a single neural network.
Question 6: What is back-translation in machine translation?
- Translating monolingual target-language text into the source language to create synthetic parallel training data (Correct answer)
- Translating a document and then translating it back to verify accuracy
- A technique to reverse the order of tokens in training data
- Augmenting training data by replacing words with their antonyms
Correct answer: Translating monolingual target-language text into the source language to create synthetic parallel training data
Back-translation uses an existing translation model to generate source-side sentences from target-language monolingual data, creating cheap additional training pairs.
Natural Language Processing Machine Translation and Sequence-to-Sequence Models 1
What is the encoder-decoder architecture in NLP?