DSE Natural Language Processing Fundamentals 2 — Questions and Answers
Question 1: Which technique is used in word2vec to predict surrounding context words given a target word?
- CBOW
- Skip-gram (Correct answer)
- GloVe
- FastText
Correct answer: Skip-gram
Skip-gram predicts surrounding context words given a center/target word, while CBOW does the opposite.
Question 2: What does the term 'perplexity' measure in the context of language models?
- Model accuracy on a test set
- How well a probability model predicts a sample (Correct answer)
- The number of unique tokens in a corpus
- The cosine similarity between word vectors
Correct answer: How well a probability model predicts a sample
Perplexity measures how well a language model predicts a held-out sample — lower perplexity indicates a better model.
Question 3: In transformer architectures, what is the purpose of positional encoding?
- To reduce the dimensionality of embeddings
- To inject sequence order information since attention is permutation-invariant (Correct answer)
- To normalize token frequencies across the vocabulary
- To mask padding tokens during training
Correct answer: To inject sequence order information since attention is permutation-invariant
Self-attention has no inherent notion of order, so positional encodings are added to embeddings to convey token position.
Question 4: Which evaluation metric is most appropriate for machine translation tasks?
- F1-score
- BLEU score (Correct answer)
- Perplexity
- Jaccard similarity
Correct answer: BLEU score
BLEU (Bilingual Evaluation Understudy) measures n-gram overlap between generated translations and reference translations.
Question 5: What is the main advantage of subword tokenization methods like Byte-Pair Encoding (BPE) over word-level tokenization?
- Faster training speed
- Handles out-of-vocabulary words more gracefully (Correct answer)
- Produces shorter sequences
- Requires no preprocessing
Correct answer: Handles out-of-vocabulary words more gracefully
BPE breaks rare and unknown words into subword units, reducing out-of-vocabulary issues while keeping common words intact.
Question 6: In Named Entity Recognition (NER), what does the BIO tagging scheme stand for?
- Binary, Indexed, Output
- Beginning, Inside, Outside (Correct answer)
- Base, Intermediate, Object
- Block, Identify, Omit
Correct answer: Beginning, Inside, Outside
BIO tagging marks the Beginning of an entity, tokens Inside an entity, and tokens Outside any entity.
Question 7: Which of the following best describes the attention mechanism's computational complexity relative to sequence length n?
- O(n)
- O(n log n)
- O(n²) (Correct answer)
- O(n³)
Correct answer: O(n²)
Standard self-attention computes pairwise interactions between all tokens, resulting in O(n²) time and memory complexity.
DSE Natural Language Processing Fundamentals 2
Which technique is used in word2vec to predict surrounding context words given a target word?