Machine Learning Natural Language Processing 2

Question 1

Which attention mechanism allows a model to weigh different positions of the input sequence when encoding a single token?

Accepted Answer

Self-attention

Answer

Self-attention computes relationships between all token positions in a sequence simultaneously, allowing each token to attend to every other token.

Question 2

What does the term 'perplexity' measure in language model evaluation?

Accepted Answer

How well a probability model predicts a sample

Answer

Perplexity measures how well a language model predicts a held-out text — lower perplexity indicates a better-fitting model.

Question 3

In Named Entity Recognition (NER), which tagging scheme uses B-, I-, and O- prefixes?

Accepted Answer

BIO tagging

Answer

BIO tagging marks the Beginning of an entity, the Inside (continuation) of an entity, and Outside tokens that are not part of any entity.

Question 4

Which technique splits rare words into smaller subword units to handle out-of-vocabulary tokens?

Accepted Answer

Byte-Pair Encoding (BPE)

Answer

BPE iteratively merges the most frequent character pairs to form a fixed-size vocabulary of subword tokens, handling rare and unseen words.

Question 5

What is the primary purpose of the decoder's cross-attention layer in a Transformer seq2seq model?

Accepted Answer

To attend to encoder output while generating each target token

Answer

Cross-attention allows each decoder token to query the encoder's hidden states, aligning source and target sequences during generation.

Machine Learning Practice Test