Natural Language Processing Trivia 5

Question 1

What is 'transfer learning' in the context of NLP?

Accepted Answer

Applying knowledge gained from pre-training on large data to specific downstream tasks

Answer

Transfer learning in NLP involves pre-training a model on large unlabeled text, then fine-tuning it on smaller labeled datasets for specific tasks.

Question 2

What does 'hallucination' mean when describing large language model outputs?

Accepted Answer

Producing plausible-sounding but factually incorrect or fabricated information

Answer

Hallucination in LLMs refers to the model generating confident-sounding but factually incorrect, fabricated, or nonsensical information.

Question 3

In NLP, what is the 'vocabulary' of a language model?

Accepted Answer

The fixed set of tokens the model can process and generate

Answer

The vocabulary is the fixed set of tokens (words, subwords, or characters) that a language model recognizes and can produce as output.

Question 4

Which subword tokenization algorithm is used by GPT-2 and later GPT models?

Accepted Answer

Byte-Pair Encoding (BPE)

Answer

GPT models use Byte-Pair Encoding (BPE), which iteratively merges the most frequent byte pairs to build a subword vocabulary.

Question 5

What is the main advantage of subword tokenization over word-level tokenization?

Accepted Answer

It handles rare and out-of-vocabulary words by splitting them into known subword units

Answer

Subword tokenization breaks unknown or rare words into smaller subword pieces that are in the vocabulary, avoiding out-of-vocabulary problems entirely.

Natural Language Processing Practice Test

Natural Language Processing Practice Test

Natural Language Processing Trivia 5