AI AI Engineer: NLP and Large Language Models

Question 1

What is the 'attention mechanism' in transformer-based models?

Accepted Answer

A mechanism that computes weighted relationships between all positions in a sequence

Answer

Attention computes a weighted sum of values based on query-key similarity, allowing the model to focus on relevant parts of the input regardless of distance.

Question 2

What does 'tokenization' mean in the context of NLP preprocessing?

Accepted Answer

Splitting raw text into discrete units (tokens) for model input

Answer

Tokenization breaks raw text into tokens (words, subwords, or characters) that are then mapped to numeric IDs for model processing.

Question 3

What is 'fine-tuning' a large language model (LLM)?

Accepted Answer

Continuing training of a pretrained LLM on task-specific data to adapt its behavior

Answer

Fine-tuning updates the pretrained model's weights on a smaller, task-specific dataset, adapting general language understanding to a specific domain or task.

Question 4

What is 'prompt engineering' when working with LLMs?

Accepted Answer

Designing input text to guide LLM behavior without modifying model weights

Answer

Prompt engineering involves crafting input instructions, examples, and context to elicit desired outputs from an LLM without changing its parameters.

Question 5

What is Retrieval-Augmented Generation (RAG)?

Accepted Answer

Combining an LLM with a retrieval system to ground responses in external documents

Answer

RAG retrieves relevant documents from an external knowledge base and injects them into the LLM's context, enabling up-to-date, grounded responses.

AI - Engineer Practice Test

AI - Engineer Practice Test

AI AI Engineer: NLP and Large Language Models