Artificial Intelligence Natural Language Processing 1

Question 1

What is tokenization in natural language processing?

Accepted Answer

Splitting text into meaningful units such as words or subwords

Answer

Tokenization breaks raw text into tokens (words, subwords, or characters) that serve as the basic units for NLP model input.

Question 2

What does TF-IDF measure in information retrieval and NLP?

Accepted Answer

The importance of a word in a document relative to a corpus

Answer

TF-IDF multiplies term frequency (how often a word appears in a document) by inverse document frequency (penalizing words common across many documents).

Question 3

Which word embedding model learns vector representations by predicting surrounding words (skip-gram) or predicting a word from context (CBOW)?

Accepted Answer

Word2Vec

Answer

Word2Vec offers two architectures: skip-gram predicts context words from a target, and CBOW predicts a target word from its context window.

Question 4

What NLP task involves labeling each token in a sentence with its grammatical role (noun, verb, etc.)?

Accepted Answer

Part-of-speech tagging

Answer

Part-of-speech (POS) tagging assigns grammatical categories to each token, such as noun, verb, adjective, or adverb.

Question 5

What is the 'attention mechanism' in neural NLP models?

Accepted Answer

A method that allows the model to weight the importance of different input tokens when producing an output

Answer

Attention lets the model dynamically focus on relevant parts of the input sequence when generating each output token, enabling better handling of long-range dependencies.

Artificial Intelligence Practice Test

Artificial Intelligence Practice Test

Artificial Intelligence Natural Language Processing 1