Natural Language Processing (Artificial Intelligence) 2 — Questions and Answers
Question 1: Which attention mechanism allows each token in a sequence to attend to all other tokens simultaneously, enabling parallelization?
- Recurrent attention
- Self-attention (Correct answer)
- Causal attention
- Sparse attention
Correct answer: Self-attention
Self-attention computes relationships between all token pairs in parallel, which is the core innovation of the Transformer architecture.
Question 2: What problem does beam search solve in NLP sequence generation tasks?
- Overfitting during training
- Suboptimal output due to greedy decoding (Correct answer)
- Exploding gradients in RNNs
- Out-of-vocabulary token handling
Correct answer: Suboptimal output due to greedy decoding
Beam search maintains multiple candidate sequences at each step, producing better outputs than greedy decoding which commits to one token at a time.
Question 3: In the context of language models, what does 'perplexity' measure?
- Model size in parameters
- How well the model predicts a sample (Correct answer)
- Vocabulary coverage
- Training data diversity
Correct answer: How well the model predicts a sample
Perplexity measures how well a probability model predicts a test sample — lower perplexity means the model assigns higher probability to the correct tokens.
Question 4: Which technique allows a pre-trained language model to adapt to a new task using only a handful of examples shown in the prompt?
- Fine-tuning
- Distillation
- Few-shot prompting (Correct answer)
- Data augmentation
Correct answer: Few-shot prompting
Few-shot prompting includes a small number of input-output examples directly in the prompt, enabling the model to generalize without gradient updates.
Question 5: What is the role of the 'key' vector in the Transformer self-attention mechanism?
- Stores the output representation of a token
- Represents what a token has to offer for matching (Correct answer)
- Encodes the positional information of a token
- Masks padding tokens during attention computation
Correct answer: Represents what a token has to offer for matching
Keys are paired with queries via dot-product to compute attention scores that determine how much each value vector contributes to the output.
Question 6: In NLP, what does the term 'hallucination' refer to when discussing large language models?
- Model generating outputs that are factually incorrect but fluent (Correct answer)
- Overfitting to training noise
- Repetition of input tokens verbatim
- Failure to generate any output token
Correct answer: Model generating outputs that are factually incorrect but fluent
Hallucination describes when an LLM produces confident, fluent text that is factually wrong or fabricated, often without signaling uncertainty.
Question 7: Which NLP task involves identifying the most probable structured interpretation (e.g., slots and intents) from user utterances in conversational AI?
- Coreference resolution
- Dialogue state tracking (Correct answer)
- Semantic role labeling
- Constituency parsing
Correct answer: Dialogue state tracking
Dialogue state tracking extracts and updates slot-value pairs and user intent from each conversational turn to maintain context for task-oriented dialogue systems.
Natural Language Processing (Artificial Intelligence) 2
Which attention mechanism allows each token in a sequence to attend to all other tokens simultaneously, enabling parallelization?