Machine Learning Natural Language Processing 5 — Questions and Answers
Question 1: Which task requires a model to determine whether a hypothesis sentence is entailed by, contradicts, or is neutral to a premise sentence?
- Coreference resolution
- Natural Language Inference (NLI) (Correct answer)
- Question answering
- Relation extraction
Correct answer: Natural Language Inference (NLI)
NLI (also called Textual Entailment) classifies the logical relationship between a premise and hypothesis into entailment, contradiction, or neutral.
Question 2: In transformer-based models, what is the computational complexity of full self-attention with respect to sequence length n?
- O(n)
- O(n log n)
- O(n²) (Correct answer)
- O(n³)
Correct answer: O(n²)
Full self-attention computes pairwise interactions between all n tokens, resulting in O(n²) time and memory complexity, which limits scalability to long sequences.
Question 3: What does 'instruction tuning' (or 'instruction fine-tuning') accomplish for large language models?
- It compresses model weights for faster inference
- It trains the model to follow natural language task instructions across diverse tasks (Correct answer)
- It applies reinforcement learning from human feedback
- It converts the model to a smaller distilled version
Correct answer: It trains the model to follow natural language task instructions across diverse tasks
Instruction tuning fine-tunes an LLM on (instruction, output) pairs across many tasks, dramatically improving its ability to generalize to new task descriptions.
Question 4: Which evaluation dataset is commonly used to benchmark reading comprehension and question answering in NLP?
- ImageNet
- SQuAD (Correct answer)
- MNIST
- LibriSpeech
Correct answer: SQuAD
SQuAD (Stanford Question Answering Dataset) consists of questions posed on Wikipedia passages where answers are extractable spans of text.
Question 5: In the context of NLP, what is a 'zero-shot' setting?
- The model is trained on zero examples and evaluated randomly
- The model solves a task at inference time without any task-specific training examples (Correct answer)
- The model uses zero attention heads
- The model is trained with a learning rate of zero
Correct answer: The model solves a task at inference time without any task-specific training examples
Zero-shot evaluation tests a model's ability to perform a new task using only a natural language description, without any labeled examples for that specific task.
Question 6: Which technique helps mitigate catastrophic forgetting when fine-tuning large language models on new tasks?
- Increasing the learning rate
- Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA (Correct answer)
- Removing dropout during training
- Using a larger batch size
Correct answer: Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA
PEFT methods like LoRA freeze most pre-trained weights and add small trainable adapter modules, preserving general knowledge while learning new tasks efficiently.
Question 7: What is the purpose of the 'temperature' parameter in language model sampling?
- It controls the number of layers in the model
- It scales the logits before softmax to control output randomness (Correct answer)
- It sets the maximum sequence length
- It determines the learning rate during fine-tuning
Correct answer: It scales the logits before softmax to control output randomness
Temperature divides the logits before softmax — lower values make the distribution sharper (more deterministic), while higher values make it flatter (more random).
Machine Learning Natural Language Processing 5
Which task requires a model to determine whether a hypothesis sentence is entailed by, contradicts, or is neutral to a premise sentence?