MS-DS Master of Data science Natural Language Processing 4

Question 1

What does the term 'word sense disambiguation' (WSD) refer to in NLP?

Accepted Answer

Determining which meaning of a polysemous word is intended in context

Answer

WSD is the task of automatically identifying the correct sense of an ambiguous word (e.g., 'bank' as financial institution vs. river bank) given its context.

Question 2

In information retrieval, what does TF-IDF measure and why is the IDF component important?

Accepted Answer

How often a term appears in a document weighted by how rare it is across all documents; IDF down-weights common terms

Answer

TF-IDF multiplies term frequency (importance within a document) by inverse document frequency (rarity across corpus), giving high scores to terms that are distinctive for a document.

Question 3

Which decoding strategy for sequence generation introduces randomness by sampling from the top-k most probable tokens at each step?

Accepted Answer

Top-k sampling

Answer

Top-k sampling restricts the sampling distribution to the k highest-probability tokens at each step, balancing diversity and coherence in generated text.

Question 4

What is the primary benefit of using subword tokenization over word-level tokenization for NLP models?

Accepted Answer

Subword tokenization handles morphological variants and rare words without requiring an open vocabulary

Answer

By splitting rare or unknown words into frequent subword units, methods like BPE and WordPiece eliminate out-of-vocabulary issues while keeping vocabulary size tractable.

Question 5

In the context of large language model fine-tuning, what does RLHF (Reinforcement Learning from Human Feedback) accomplish?

Accepted Answer

It aligns model outputs with human preferences by training a reward model and optimizing with PPO

Answer

RLHF first trains a reward model on human preference comparisons, then fine-tunes the LLM using PPO to maximize expected reward while minimizing KL divergence from the SFT policy.

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Natural Language Processing 4