NLP Word Embeddings 4

Question 1

How do contextualized embeddings from ELMo differ fundamentally from word2vec embeddings?

Accepted Answer

ELMo produces a different vector for the same word depending on its sentence context using a biLSTM

Answer

ELMo generates dynamic, context-sensitive embeddings by passing the full sentence through a bidirectional LSTM, so 'bank' gets different vectors in different sentences.

Question 2

What is 'transfer learning' in the context of pre-trained word embeddings?

Accepted Answer

Using embeddings learned on a large general corpus as the starting point for a domain-specific task

Answer

Pre-trained embeddings encode general language knowledge that can be transferred to downstream tasks, especially when task-specific data is scarce.

Question 3

In a bag-of-words document representation, what information is lost compared to using averaged word embeddings?

Accepted Answer

Word order and compositionality are discarded in both, but BoW also ignores semantic similarity between different words

Answer

BoW treats vocabulary as a sparse orthogonal space where different words have zero similarity, whereas averaged embeddings encode semantic relatedness between words.

Question 4

What is 'domain adaptation' when applied to word embeddings?

Accepted Answer

Further training or fine-tuning general embeddings on in-domain text to capture domain-specific vocabulary and meaning

Answer

General-purpose embeddings may not capture specialized terminology well, so continuing training on domain text (e.g., biomedical papers) adapts them to the target domain.

Question 5

Which technique allows word embeddings from two different languages to be mapped into a shared cross-lingual vector space?

Accepted Answer

Learning a linear transformation (rotation) using bilingual anchor word pairs

Answer

Cross-lingual alignment methods like VecMap find a rotation matrix that maps monolingual embeddings into a shared space using bilingual seed lexicons.

NLP Practice Test

NLP Practice Test

NLP Word Embeddings 4