NLP Language Models 4

Question 1

What is 'transfer learning' in the context of NLP language models?

Accepted Answer

Pretraining a model on large general corpora then fine-tuning it on a downstream task

Answer

Transfer learning in NLP involves pretraining a model on a large dataset (e.g., Common Crawl) then adapting those learned representations to specific tasks with less data.

Question 2

What distinguishes a 'causal' language model from a 'non-causal' (bidirectional) one during training?

Accepted Answer

Causal models mask future tokens so each position only attends to past tokens; bidirectional models attend to all positions

Answer

Causal (autoregressive) models use a triangular attention mask to prevent each token from attending to future positions, making them suitable for generation.

Question 3

In language model training, what is 'teacher forcing'?

Accepted Answer

Feeding ground-truth tokens as inputs at each decoding step rather than the model's own previous predictions

Answer

Teacher forcing provides the ground-truth previous token as input at each training step, stabilizing gradients compared to feeding the model's own potentially wrong predictions.

Question 4

What is 'catastrophic forgetting' in the context of fine-tuning a pretrained language model?

Accepted Answer

The model loses general knowledge acquired during pretraining when fine-tuned aggressively on a narrow task

Answer

Catastrophic forgetting occurs when fine-tuning on task-specific data overwrites the broad representations learned during pretraining, degrading performance on other tasks.

Question 5

What does the acronym RLHF stand for, and how is it used in language model alignment?

Accepted Answer

Reinforcement Learning from Human Feedback — trains models to produce outputs preferred by human raters

Answer

RLHF uses human preference rankings to train a reward model, which then guides policy optimization (via PPO or similar) to make the language model more helpful and aligned.

NLP Practice Test

NLP Practice Test

NLP Language Models 4