DSE Natural Language Processing Fundamentals 5 — Questions and Answers
Question 1: What distinguishes zero-shot classification from few-shot classification in NLP?
- Zero-shot uses labeled data; few-shot uses unlabeled data
- Zero-shot provides no examples; few-shot provides a small number of examples in the prompt (Correct answer)
- Zero-shot fine-tunes the model; few-shot does not
- Zero-shot uses larger models than few-shot
Correct answer: Zero-shot provides no examples; few-shot provides a small number of examples in the prompt
Zero-shot classification relies only on task descriptions or label names, while few-shot provides a handful of labeled examples in the input.
Question 2: Which NLP task involves identifying the grammatical role of each word in a sentence, such as noun or verb?
- Dependency parsing
- Part-of-speech tagging (Correct answer)
- Named entity recognition
- Semantic role labeling
Correct answer: Part-of-speech tagging
Part-of-speech tagging assigns grammatical categories (noun, verb, adjective, etc.) to each token in a sentence.
Question 3: What is the key advantage of using pre-trained contextual embeddings (like those from BERT) over static word embeddings (like word2vec)?
- They require less computational memory
- They produce different representations for the same word in different contexts (Correct answer)
- They only need character-level input
- They eliminate the need for tokenization
Correct answer: They produce different representations for the same word in different contexts
Contextual embeddings dynamically encode a word's meaning based on its surrounding context, addressing polysemy that static embeddings cannot.
Question 4: In text classification with imbalanced classes, which strategy is most appropriate?
- Always use accuracy as the sole metric
- Use class-weighted loss or oversample the minority class (Correct answer)
- Remove the minority class from training data
- Increase the vocabulary size
Correct answer: Use class-weighted loss or oversample the minority class
Class-weighted loss or oversampling corrects the model's bias toward the majority class when training data is imbalanced.
Question 5: What is the purpose of the 'CLS' token in BERT-style models?
- To mark the end of a sentence
- To serve as an aggregate sequence representation for classification tasks (Correct answer)
- To separate two input sentences
- To indicate a masked token during pre-training
Correct answer: To serve as an aggregate sequence representation for classification tasks
The [CLS] token is prepended to inputs, and its final hidden state is used as a pooled sentence-level representation for downstream classification.
Question 6: Which of the following tasks is best described as a seq2seq problem?
- Sentiment classification
- Text summarization (Correct answer)
- Part-of-speech tagging
- Word sense disambiguation
Correct answer: Text summarization
Text summarization takes a long input sequence and generates a shorter output sequence, matching the encoder-decoder seq2seq paradigm.
Question 7: What is 'catastrophic forgetting' in the context of fine-tuning language models?
- The model forgets the fine-tuning task after deployment
- The model loses previously learned knowledge when updated on new task data (Correct answer)
- The model fails to converge during pre-training
- The optimizer forgets momentum values between epochs
Correct answer: The model loses previously learned knowledge when updated on new task data
Catastrophic forgetting occurs when fine-tuning on a new task overwrites the weights learned during pre-training, degrading performance on original capabilities.
DSE Natural Language Processing Fundamentals 5
What distinguishes zero-shot classification from few-shot classification in NLP?