DSE Natural Language Processing Fundamentals 4 — Questions and Answers
Question 1: Which problem does the 'vanishing gradient' issue primarily affect in recurrent neural networks for NLP?
- Tokenization accuracy
- Learning long-range dependencies in sequences (Correct answer)
- Vocabulary size management
- Attention weight normalization
Correct answer: Learning long-range dependencies in sequences
Gradients shrink exponentially as they propagate back through many time steps, making it hard for RNNs to capture long-range dependencies.
Question 2: In coreference resolution, what is the task the model must solve?
- Identifying the sentiment of pronouns
- Clustering mentions in text that refer to the same real-world entity (Correct answer)
- Detecting grammatical errors in pronoun usage
- Translating pronouns across languages
Correct answer: Clustering mentions in text that refer to the same real-world entity
Coreference resolution groups all mentions (nouns, pronouns, noun phrases) that refer to the same entity into clusters.
Question 3: What is the primary goal of sequence-to-sequence (seq2seq) models in NLP?
- Classify a fixed-length input into one of several categories
- Map an input sequence of tokens to an output sequence of potentially different length (Correct answer)
- Embed words into a fixed-dimensional vector space
- Predict the next token in an autoregressive language model
Correct answer: Map an input sequence of tokens to an output sequence of potentially different length
Seq2seq models use an encoder to compress the input and a decoder to generate an output sequence, used in tasks like translation and summarization.
Question 4: Which of the following is an example of an extractive summarization approach?
- Generating new sentences that paraphrase the source document
- Selecting and concatenating important sentences directly from the source (Correct answer)
- Using a language model to rewrite the document in a shorter form
- Training a classifier to rank document quality
Correct answer: Selecting and concatenating important sentences directly from the source
Extractive summarization selects actual sentences or phrases from the original document rather than generating new text.
Question 5: What does the term 'polysemy' mean in linguistics and NLP?
- A word that has multiple spellings
- A single word form that carries multiple related meanings (Correct answer)
- Two different words with identical meanings
- A word borrowed from another language
Correct answer: A single word form that carries multiple related meanings
Polysemy refers to a single word having multiple related senses, such as 'bank' meaning a financial institution or a riverbank.
Question 6: In the encoder-decoder attention of a transformer, what do the queries come from?
- The encoder's final hidden states
- The decoder's previous hidden states (Correct answer)
- A learned embedding table
- The positional encoding layer
Correct answer: The decoder's previous hidden states
In cross-attention, the decoder generates queries from its own states, while keys and values come from the encoder's output.
Question 7: Which metric is commonly used to evaluate named entity recognition (NER) performance?
- BLEU score
- Entity-level F1 score (Correct answer)
- Perplexity
- ROUGE-L
Correct answer: Entity-level F1 score
NER is evaluated with entity-level F1, which requires both the entity boundary and entity type to match for a prediction to count as correct.
DSE Natural Language Processing Fundamentals 4
Which problem does the 'vanishing gradient' issue primarily affect in recurrent neural networks for NLP?