Machine Learning Natural Language Processing 2 — Questions and Answers
Question 1: Which attention mechanism allows a model to weigh different positions of the input sequence when encoding a single token?
- Convolutional attention
- Self-attention (Correct answer)
- Pooling attention
- Recurrent attention
Correct answer: Self-attention
Self-attention computes relationships between all token positions in a sequence simultaneously, allowing each token to attend to every other token.
Question 2: What does the term 'perplexity' measure in language model evaluation?
- The number of parameters in the model
- How well a probability model predicts a sample (Correct answer)
- The diversity of the training vocabulary
- The speed of token generation
Correct answer: How well a probability model predicts a sample
Perplexity measures how well a language model predicts a held-out text — lower perplexity indicates a better-fitting model.
Question 3: In Named Entity Recognition (NER), which tagging scheme uses B-, I-, and O- prefixes?
- BPE tagging
- BIO tagging (Correct answer)
- SentencePiece tagging
- Unigram tagging
Correct answer: BIO tagging
BIO tagging marks the Beginning of an entity, the Inside (continuation) of an entity, and Outside tokens that are not part of any entity.
Question 4: Which technique splits rare words into smaller subword units to handle out-of-vocabulary tokens?
- Word2Vec
- Byte-Pair Encoding (BPE) (Correct answer)
- TF-IDF
- Latent Semantic Analysis
Correct answer: Byte-Pair Encoding (BPE)
BPE iteratively merges the most frequent character pairs to form a fixed-size vocabulary of subword tokens, handling rare and unseen words.
Question 5: What is the primary purpose of the decoder's cross-attention layer in a Transformer seq2seq model?
- To mask future tokens during training
- To attend to encoder output while generating each target token (Correct answer)
- To normalize the position embeddings
- To apply dropout regularization
Correct answer: To attend to encoder output while generating each target token
Cross-attention allows each decoder token to query the encoder's hidden states, aligning source and target sequences during generation.
Question 6: Which metric is commonly used to evaluate machine translation quality by comparing n-gram overlap with reference translations?
- F1 score
- BLEU score (Correct answer)
- Perplexity
- ROUGE-L
Correct answer: BLEU score
BLEU (Bilingual Evaluation Understudy) computes precision of n-gram matches between the hypothesis and one or more reference translations.
Question 7: In the context of NLP, what does 'coreference resolution' solve?
- Identifying the grammatical subject of a sentence
- Linking pronouns and noun phrases that refer to the same entity (Correct answer)
- Parsing the syntactic dependency tree
- Detecting named entities in text
Correct answer: Linking pronouns and noun phrases that refer to the same entity
Coreference resolution determines when different expressions in text refer to the same real-world entity, such as linking 'she' back to 'Marie Curie'.
Machine Learning Natural Language Processing 2
Which attention mechanism allows a model to weigh different positions of the input sequence when encoding a single token?