Natural Language Processing Natural Language Processing MCQ 5 — Questions and Answers
Question 1: What is 'attention sink' phenomenon observed in large language models with long contexts?
- Attention weights collapsing to zero for all tokens beyond the context window
- Disproportionately high attention assigned to early tokens (especially the first) regardless of relevance (Correct answer)
- Attention heads specializing exclusively in syntactic rather than semantic patterns
- Gradient flow being blocked at attention layers during backpropagation
Correct answer: Disproportionately high attention assigned to early tokens (especially the first) regardless of relevance
In long contexts, LLMs tend to assign excessive attention to initial tokens (particularly the BOS token) as a form of 'attention dumping' for irrelevant positions.
Question 2: Which approach does 'retrieval-augmented generation' (RAG) use to improve LLM factual accuracy?
- Fine-tuning the model on verified factual datasets before deployment
- Retrieving relevant documents at inference time and conditioning generation on them (Correct answer)
- Using ensemble voting across multiple LLMs to filter hallucinations
- Constraining decoding to only output n-grams seen in a trusted knowledge base
Correct answer: Retrieving relevant documents at inference time and conditioning generation on them
RAG retrieves relevant passages from an external corpus and provides them as context to the LLM, grounding generation in retrieved evidence.
Question 3: What is 'constitutional AI' or 'RLHF' primarily used for in modern LLM development?
- Compressing large models into smaller distilled versions for deployment
- Aligning model outputs with human preferences and values through feedback-based training (Correct answer)
- Pre-training models on legally licensed corpora to avoid copyright issues
- Automatically detecting and removing biased training examples from web crawls
Correct answer: Aligning model outputs with human preferences and values through feedback-based training
RLHF (Reinforcement Learning from Human Feedback) trains a reward model on human preferences and uses it to fine-tune LLMs toward helpful, harmless behavior.
Question 4: In NLP, what does 'distributional hypothesis' state?
- Tokens follow a Zipfian power-law frequency distribution in natural corpora
- Words that appear in similar contexts tend to have similar meanings (Correct answer)
- Probability distributions over sequences converge given sufficient training data
- Syntactic distributions predict semantic roles across language families
Correct answer: Words that appear in similar contexts tend to have similar meanings
The distributional hypothesis, foundational to word embeddings, states that words occurring in similar contexts share similar semantic properties.
Question 5: What is 'chain-of-thought' (CoT) prompting?
- Providing multiple examples of final answers without intermediate reasoning
- Eliciting step-by-step reasoning from an LLM by including reasoning traces in the prompt (Correct answer)
- Chaining multiple fine-tuned models in a pipeline for complex NLP tasks
- Using prompt templates that chain topic sentences to improve coherence
Correct answer: Eliciting step-by-step reasoning from an LLM by including reasoning traces in the prompt
CoT prompting includes examples of intermediate reasoning steps, encouraging the model to produce its own reasoning chain before giving a final answer.
Question 6: Which metric is most appropriate for evaluating a named entity recognition (NER) system?
- BLEU score over recognized entity spans
- Entity-level F1 score (precision and recall over complete entity mentions) (Correct answer)
- Perplexity of the sequence labeling model on the test set
- Accuracy of individual token-level POS assignments
Correct answer: Entity-level F1 score (precision and recall over complete entity mentions)
NER is evaluated with entity-level F1, which counts a prediction correct only if the entity type and full span boundaries match the gold standard.
Question 7: What is 'prompt injection' in the context of LLM-based applications?
- A technique to insert task-specific examples into the model's context window
- An attack where malicious user input overrides or hijacks the system's intended instructions (Correct answer)
- A method to inject domain knowledge into pre-training via curated prompts
- The process of adding few-shot examples to improve zero-shot performance
Correct answer: An attack where malicious user input overrides or hijacks the system's intended instructions
Prompt injection is a security vulnerability where attacker-controlled input manipulates an LLM into ignoring its original instructions and following the attacker's commands instead.
Natural Language Processing Natural Language Processing MCQ 5
What is 'attention sink' phenomenon observed in large language models with long contexts?