MS-DS Master of Data science Natural Language Processing 2 — Questions and Answers
Question 1: Which attention mechanism allows a model to attend to all positions in a sequence simultaneously, rather than sequentially?
- Recurrent attention
- Self-attention (scaled dot-product) (Correct answer)
- Additive attention (Bahdanau)
- Local attention
Correct answer: Self-attention (scaled dot-product)
Self-attention computes attention weights between all token pairs in parallel using query, key, and value matrices scaled by the square root of the dimension.
Question 2: In the context of language models, what does 'perplexity' measure?
- The number of unique tokens in a corpus
- How well a probability model predicts a sample (Correct answer)
- The depth of a parse tree
- The ratio of named entities to total tokens
Correct answer: How well a probability model predicts a sample
Perplexity is the exponentiated average negative log-likelihood of a test set; lower perplexity indicates a better-fitting language model.
Question 3: What is the primary purpose of the positional encoding added to token embeddings in a Transformer model?
- To reduce embedding dimensionality
- To inject information about token order since self-attention is permutation-invariant (Correct answer)
- To normalize token frequencies across the vocabulary
- To encode part-of-speech tags
Correct answer: To inject information about token order since self-attention is permutation-invariant
Because self-attention has no inherent notion of sequence order, sinusoidal or learned positional encodings are added to embeddings to convey position.
Question 4: Which NLP task involves assigning a semantic label such as ARG0 or ARG1 to constituents in a sentence relative to a predicate?
- Named entity recognition
- Semantic role labeling (Correct answer)
- Coreference resolution
- Dependency parsing
Correct answer: Semantic role labeling
Semantic role labeling (SRL) identifies who did what to whom by labeling predicate-argument structures using frameworks like PropBank.
Question 5: In byte-pair encoding (BPE) tokenization, what determines when to stop merging character pairs?
- When the vocabulary contains no more than 256 entries
- When a predefined vocabulary size limit is reached (Correct answer)
- When all tokens are single Unicode characters
- When token frequency drops below 1
Correct answer: When a predefined vocabulary size limit is reached
BPE iteratively merges the most frequent pair of adjacent symbols until the target vocabulary size is achieved.
Question 6: What distinguishes extractive summarization from abstractive summarization?
- Extractive uses neural networks; abstractive uses rule-based systems
- Extractive copies spans from the source; abstractive generates novel text (Correct answer)
- Extractive works only on structured data; abstractive on unstructured text
- Extractive produces longer summaries; abstractive produces shorter ones
Correct answer: Extractive copies spans from the source; abstractive generates novel text
Extractive summarization selects and concatenates existing sentences from the source document, while abstractive summarization generates new text that may paraphrase or compress content.
Question 7: Which evaluation metric for machine translation computes n-gram precision with a brevity penalty?
- ROUGE-L
- METEOR
- BLEU (Correct answer)
- CIDEr
Correct answer: BLEU
BLEU (Bilingual Evaluation Understudy) measures modified n-gram precision against reference translations and penalizes outputs shorter than the reference.
MS-DS Master of Data science Natural Language Processing 2
Which attention mechanism allows a model to attend to all positions in a sequence simultaneously, rather than sequentially?