Natural Language Processing (Artificial Intelligence) 4

Question 1

Which technique reduces the dimensionality of word embeddings while preserving semantic relationships, commonly used before the deep learning era?

Accepted Answer

Latent Semantic Analysis (LSA)

Answer

LSA applies SVD to a term-document matrix to discover latent semantic structure and reduce dimensionality, capturing synonymy and related concepts.

Question 2

What is 'instruction tuning' in large language models?

Accepted Answer

Fine-tuning a model on examples of (instruction, desired output) pairs to better follow natural language directives

Answer

Instruction tuning fine-tunes a pre-trained LLM on diverse (instruction, response) pairs, dramatically improving the model's ability to follow zero-shot user commands.

Question 3

In text classification, what does 'TF-IDF' weighting accomplish?

Accepted Answer

It up-weights terms that are frequent in a document but rare across the corpus

Answer

TF-IDF balances term frequency (local importance) against inverse document frequency (global rarity), surfacing distinctive words while down-weighting common stopwords.

Question 4

What distinguishes a 'generative' language model from a 'discriminative' model in NLP?

Accepted Answer

Generative models learn the joint probability P(x,y); discriminative models learn P(y|x)

Answer

Generative models model the full joint distribution and can sample new data, while discriminative models focus only on the conditional distribution needed for prediction.

Question 5

Which decoding strategy for language generation introduces randomness by sampling from the top-k most probable next tokens?

Accepted Answer

Top-k sampling

Answer

Top-k sampling restricts sampling to only the k highest-probability tokens at each step, balancing diversity and coherence in generated text.

Natural Language Processing Practice Test

Natural Language Processing Practice Test

Natural Language Processing (Artificial Intelligence) 4