Natural Language Processing Natural Language Processing MCQ 5

Question 1

What is 'attention sink' phenomenon observed in large language models with long contexts?

Accepted Answer

Disproportionately high attention assigned to early tokens (especially the first) regardless of relevance

Answer

In long contexts, LLMs tend to assign excessive attention to initial tokens (particularly the BOS token) as a form of 'attention dumping' for irrelevant positions.

Question 2

Which approach does 'retrieval-augmented generation' (RAG) use to improve LLM factual accuracy?

Accepted Answer

Retrieving relevant documents at inference time and conditioning generation on them

Answer

RAG retrieves relevant passages from an external corpus and provides them as context to the LLM, grounding generation in retrieved evidence.

Question 3

What is 'constitutional AI' or 'RLHF' primarily used for in modern LLM development?

Accepted Answer

Aligning model outputs with human preferences and values through feedback-based training

Answer

RLHF (Reinforcement Learning from Human Feedback) trains a reward model on human preferences and uses it to fine-tune LLMs toward helpful, harmless behavior.

Question 4

In NLP, what does 'distributional hypothesis' state?

Accepted Answer

Words that appear in similar contexts tend to have similar meanings

Answer

The distributional hypothesis, foundational to word embeddings, states that words occurring in similar contexts share similar semantic properties.

Question 5

What is 'chain-of-thought' (CoT) prompting?

Accepted Answer

Eliciting step-by-step reasoning from an LLM by including reasoning traces in the prompt

Answer

CoT prompting includes examples of intermediate reasoning steps, encouraging the model to produce its own reasoning chain before giving a final answer.

Natural Language Processing Practice Test

Natural Language Processing Practice Test

Natural Language Processing Natural Language Processing MCQ 5