NLP Machine Translation 4 — Questions and Answers
Question 1: Which evaluation metric for MT is specifically designed to correlate better with human judgments by incorporating synonyms and paraphrases?
- BLEU
- METEOR (Correct answer)
- WER
- NIST
Correct answer: METEOR
METEOR uses stemming, synonym matching, and paraphrase tables to better match human evaluations compared to BLEU's strict n-gram overlap.
Question 2: What is the 'exposure bias' problem in sequence-to-sequence MT training?
- The model is exposed to too much training data
- Training uses ground-truth tokens as input, but inference uses model predictions, causing a distribution mismatch (Correct answer)
- The encoder is over-trained relative to the decoder
- Attention is biased toward the beginning of the source sentence
Correct answer: Training uses ground-truth tokens as input, but inference uses model predictions, causing a distribution mismatch
Exposure bias arises because teacher-forcing during training always provides correct previous tokens, whereas at test time the model conditions on its own (possibly wrong) outputs.
Question 3: In phrase-based SMT, what does a 'distortion model' control?
- The probability of selecting a phrase pair from the table
- The reordering (permutation) of translated phrases relative to source order (Correct answer)
- The length of the output translation
- The smoothing of n-gram language model probabilities
Correct answer: The reordering (permutation) of translated phrases relative to source order
The distortion model assigns a cost to jumping between non-adjacent source phrases, controlling how much the target word order can differ from the source.
Question 4: Which post-editing metric measures the minimum number of edit operations needed to correct an MT output into an acceptable translation?
- BLEU
- METEOR
- TER (Translation Edit Rate) (Correct answer)
- chrF
Correct answer: TER (Translation Edit Rate)
TER counts the number of shifts, insertions, deletions, and substitutions needed to convert the MT hypothesis into a reference translation.
Question 5: What is 'pivot translation' and when is it used?
- Translating a sentence twice for quality verification
- Using an intermediate language to translate between two languages that lack parallel data (Correct answer)
- Pivoting the source sentence structure before encoding
- Reranking translations using a pivot language model
Correct answer: Using an intermediate language to translate between two languages that lack parallel data
Pivot (bridge) translation routes low-resource language pairs through a high-resource pivot language (often English) when direct parallel data is unavailable.
Question 6: Which component of a neural MT system is responsible for generating a fixed-length context vector in older encoder-decoder architectures (pre-attention)?
- Softmax layer
- Final encoder hidden state (Correct answer)
- Embedding matrix
- Positional encoding
Correct answer: Final encoder hidden state
In early seq2seq models, the last encoder hidden state compressed the entire source sentence into a single context vector passed to the decoder.
Question 7: What does 'document-level MT' aim to improve over sentence-level MT?
- Vocabulary coverage across domains
- Coherence, coreference resolution, and consistency across sentences (Correct answer)
- Training speed by batching entire documents
- BLEU scores on individual sentences
Correct answer: Coherence, coreference resolution, and consistency across sentences
Document-level MT models consider inter-sentence context to correctly resolve pronouns, maintain consistent terminology, and improve discourse coherence.
NLP Machine Translation 4
Which evaluation metric for MT is specifically designed to correlate better with human judgments by incorporating synonyms and paraphrases?