Natural Language Processing (Artificial Intelligence) 2

Question 1

Which attention mechanism allows each token in a sequence to attend to all other tokens simultaneously, enabling parallelization?

Accepted Answer

Self-attention

Answer

Self-attention computes relationships between all token pairs in parallel, which is the core innovation of the Transformer architecture.

Question 2

What problem does beam search solve in NLP sequence generation tasks?

Accepted Answer

Suboptimal output due to greedy decoding

Answer

Beam search maintains multiple candidate sequences at each step, producing better outputs than greedy decoding which commits to one token at a time.

Question 3

In the context of language models, what does 'perplexity' measure?

Accepted Answer

How well the model predicts a sample

Answer

Perplexity measures how well a probability model predicts a test sample — lower perplexity means the model assigns higher probability to the correct tokens.

Question 4

Which technique allows a pre-trained language model to adapt to a new task using only a handful of examples shown in the prompt?

Accepted Answer

Few-shot prompting

Answer

Few-shot prompting includes a small number of input-output examples directly in the prompt, enabling the model to generalize without gradient updates.

Question 5

What is the role of the 'key' vector in the Transformer self-attention mechanism?

Accepted Answer

Represents what a token has to offer for matching

Answer

Keys are paired with queries via dot-product to compute attention scores that determine how much each value vector contributes to the output.

Natural Language Processing Practice Test

Natural Language Processing Practice Test

Natural Language Processing (Artificial Intelligence) 2