Natural Language Processing Flashcards
6 cards from real Artificial Intelligence practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 6 Natural Language Processing flashcards as text
In a seq2seq model for machine translation, what is the role of the encoder?
Answer: To compress the source sentence into a context representation
The encoder processes the source sequence and produces a fixed-size context vector (or sequence of hidden states) that summarizes its meaning for the decoder.
What is coreference resolution in NLP?
Answer: Identifying all expressions in a text that refer to the same real-world entity
Coreference resolution links pronouns and noun phrases that refer to the same entity, enabling coherent understanding of who or what is being discussed.
What is 'perplexity' used to measure in language models?
Answer: How well the model predicts a sample of text; lower is better
Perplexity is the exponentiated average negative log-likelihood per token; it measures how surprised the model is by the test text, with lower values indicating better predictions.
Which technique allows a language model to answer questions about a document it wasn't trained on by providing that document as context?
Answer: Retrieval-augmented generation
Retrieval-augmented generation (RAG) retrieves relevant documents at inference time and includes them in the model's context, enabling factual answers beyond the model's training knowledge.
What does the Transformer's multi-head attention allow compared to single-head attention?
Answer: Attending to different parts of the sequence from multiple representational subspaces simultaneously
Multi-head attention runs several attention functions in parallel across different learned linear projections, allowing the model to jointly attend to information from different positions and representation subspaces.
What is zero-shot classification in the context of large language models?
Answer: Classifying inputs into categories the model has never explicitly been trained on, using only natural language descriptions
Zero-shot classification leverages a model's pre-trained knowledge to assign labels described in natural language without any task-specific training examples.