Machine Learning Natural Language Processing 3

Question 1

Which pre-training objective does BERT use to learn bidirectional representations?

Accepted Answer

Masked language modeling and next sentence prediction

Answer

BERT is pre-trained with Masked Language Modeling (predicting masked tokens) and Next Sentence Prediction (classifying sentence pairs) to learn deep bidirectional context.

Question 2

What differentiates an autoregressive language model from a masked language model?

Accepted Answer

Autoregressive models predict tokens left-to-right using only past context

Answer

Autoregressive models like GPT generate tokens sequentially, conditioning each prediction on all previous tokens, while masked models like BERT see full bidirectional context.

Question 3

In dependency parsing, what does the 'head' of a dependency arc represent?

Accepted Answer

The governing word that another word depends on

Answer

In a dependency parse, the head is the word that syntactically governs a dependent word, capturing grammatical relations like subject or object.

Question 4

Which technique is used to reduce the dimensionality of word co-occurrence matrices in traditional distributional semantics?

Accepted Answer

Singular Value Decomposition (SVD)

Answer

SVD factorizes the large sparse co-occurrence matrix into lower-dimensional dense word vectors, forming the basis of methods like LSA.

Question 5

What problem does beam search solve in sequence generation?

Accepted Answer

It approximates the most probable output sequence by maintaining top-k candidates

Answer

Beam search keeps the top-k (beam width) partial sequences at each step, balancing quality and computational cost versus greedy decoding.

Machine Learning Practice Test

Machine Learning Practice Test

Machine Learning Natural Language Processing 3