Natural Language Processing (Artificial Intelligence) 5 ā Questions and Answers
Question 1: What does RLHF (Reinforcement Learning from Human Feedback) enable in language model training?
- Faster convergence by replacing gradient descent with policy gradients
- Aligning model outputs with human preferences using a reward model trained on human rankings (Correct answer)
- Enabling zero-shot learning without any pre-training
- Compressing model size through preference-based pruning
Correct answer: Aligning model outputs with human preferences using a reward model trained on human rankings
RLHF trains a reward model on human preference comparisons and then uses RL (typically PPO) to fine-tune the LLM to maximize that reward, improving helpfulness and safety.
Question 2: Which of the following is an example of a 'zero-shot' NLP task evaluation?
- Fine-tuning BERT on 1,000 labeled examples before testing
- Providing five labeled examples in the prompt before asking the model to classify
- Asking the model to classify sentiment with no task-specific training examples provided (Correct answer)
- Training a separate classification head on top of frozen embeddings
Correct answer: Asking the model to classify sentiment with no task-specific training examples provided
Zero-shot evaluation tests the model's ability to handle a task using only the task description in the prompt, without any demonstration examples or additional fine-tuning.
Question 3: What is 'byte-pair encoding' (BPE) used for in modern NLP pipelines?
- Compressing model weights for efficient deployment
- Building a subword vocabulary by iteratively merging the most frequent character pairs (Correct answer)
- Encoding positional information in token embeddings
- Tokenizing text using fixed-length byte windows
Correct answer: Building a subword vocabulary by iteratively merging the most frequent character pairs
BPE starts with characters and iteratively merges the most frequent adjacent pair, building a vocabulary of subword units that balances coverage and vocabulary size.
Question 4: In the context of NLP, what is 'word sense disambiguation' (WSD)?
- Splitting compound words into their constituent morphemes
- Determining which meaning of a polysemous word is intended in a given context (Correct answer)
- Linking named entities to entries in a knowledge base
- Identifying the grammatical role of a word in a sentence
Correct answer: Determining which meaning of a polysemous word is intended in a given context
WSD selects the correct sense of a word like 'bank' (financial institution vs. riverbank) based on surrounding context, which is critical for accurate semantic understanding.
Question 5: Which architecture introduced the encoder-decoder structure that became the foundation for modern neural machine translation?
- Convolutional Neural Network (CNN)
- Recurrent Seq2Seq model (Correct answer)
- Transformer
- Bidirectional LSTM
Correct answer: Recurrent Seq2Seq model
The sequence-to-sequence (seq2seq) model with RNN encoder-decoder, proposed by Sutskever et al. (2014), introduced the encode-then-decode paradigm for neural MT.
Question 6: What is the key advantage of 'sparse attention' over full self-attention in Transformer models processing very long sequences?
- It eliminates the need for positional encodings
- It reduces the quadratic memory and compute cost by attending to only a subset of tokens (Correct answer)
- It allows the model to attend to future tokens during training
- It replaces the softmax function with a linear operation
Correct answer: It reduces the quadratic memory and compute cost by attending to only a subset of tokens
Full self-attention scales O(nĀ²) with sequence length; sparse attention patterns (e.g., local windows, strided patterns) reduce this to O(n log n) or O(n), enabling much longer contexts.
Question 7: In NLP, what does 'named entity recognition' (NER) aim to identify?
- The grammatical dependencies between words in a sentence
- Spans of text that refer to real-world entities such as persons, organizations, and locations (Correct answer)
- The sentiment polarity of named subjects in a document
- The topic cluster to which a document belongs
Correct answer: Spans of text that refer to real-world entities such as persons, organizations, and locations
NER is a sequence labeling task that detects and classifies entity mentions (e.g., 'Apple' as ORG, 'Tim Cook' as PER, 'Cupertino' as LOC) within unstructured text.
Natural Language Processing (Artificial Intelligence) 5
What does RLHF (Reinforcement Learning from Human Feedback) enable in language model training?