MS-DS Master of Data science Natural Language Processing 2

Question 1

Which attention mechanism allows a model to attend to all positions in a sequence simultaneously, rather than sequentially?

Accepted Answer

Self-attention (scaled dot-product)

Answer

Self-attention computes attention weights between all token pairs in parallel using query, key, and value matrices scaled by the square root of the dimension.

Question 2

In the context of language models, what does 'perplexity' measure?

Accepted Answer

How well a probability model predicts a sample

Answer

Perplexity is the exponentiated average negative log-likelihood of a test set; lower perplexity indicates a better-fitting language model.

Question 3

What is the primary purpose of the positional encoding added to token embeddings in a Transformer model?

Accepted Answer

To inject information about token order since self-attention is permutation-invariant

Answer

Because self-attention has no inherent notion of sequence order, sinusoidal or learned positional encodings are added to embeddings to convey position.

Question 4

Which NLP task involves assigning a semantic label such as ARG0 or ARG1 to constituents in a sentence relative to a predicate?

Accepted Answer

Semantic role labeling

Answer

Semantic role labeling (SRL) identifies who did what to whom by labeling predicate-argument structures using frameworks like PropBank.

Question 5

In byte-pair encoding (BPE) tokenization, what determines when to stop merging character pairs?

Accepted Answer

When a predefined vocabulary size limit is reached

Answer

BPE iteratively merges the most frequent pair of adjacent symbols until the target vocabulary size is achieved.

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Natural Language Processing 2