NLP Language Models 5

Question 1

What is 'quantization' in the deployment of large language models?

Accepted Answer

Representing model weights with lower-precision numbers (e.g., INT8 instead of FP32) to reduce size and speed up inference

Answer

Quantization compresses model weights by using fewer bits per parameter, significantly reducing memory and improving inference speed with minimal accuracy loss.

Question 2

What is 'knowledge distillation' in the context of language models?

Accepted Answer

Training a smaller 'student' model to mimic the output distribution of a larger 'teacher' model

Answer

Knowledge distillation trains a compact student model using the soft probability outputs (logits) of a larger teacher model as training targets, transferring its learned representations.

Question 3

What is the 'hallucination' problem in large language models?

Accepted Answer

The model generates text that is fluent and confident but factually incorrect or fabricated

Answer

Hallucination refers to language models generating plausible-sounding but false information, a key reliability challenge since models optimize for fluency, not factual accuracy.

Question 4

In language modeling, what is the difference between 'top-k sampling' and 'greedy decoding'?

Accepted Answer

Greedy decoding always selects the most probable token; top-k samples from the k most probable tokens

Answer

Greedy decoding always picks the single highest-probability token (deterministic), while top-k restricts sampling to the k most probable tokens, introducing controlled randomness.

Question 5

What is a 'mixture of experts' (MoE) architecture in the context of large language models?

Accepted Answer

A model architecture where each input token is routed to only a subset of specialized sub-networks (experts) rather than all parameters

Answer

In MoE models, a learned gating network routes each token to a sparse subset of 'expert' feed-forward layers, enabling very large total parameter counts without proportionally increasing compute.

NLP Practice Test

NLP Practice Test

NLP Language Models 5