Artificial Intelligence Natural Language Processing 2 — Questions and Answers
Question 1: What architecture is BERT (Bidirectional Encoder Representations from Transformers) based on?
- Recurrent neural network
- Convolutional neural network
- Transformer encoder (Correct answer)
- Transformer decoder
Correct answer: Transformer encoder
BERT uses only the encoder portion of the Transformer and is pre-trained with bidirectional context, reading the entire sentence at once.
Question 2: What does 'fine-tuning' a pre-trained language model involve?
- Re-training the model entirely from scratch on new data
- Continuing training on a task-specific labeled dataset to adapt the model to a new task (Correct answer)
- Pruning the model to reduce its size
- Converting the model to a different programming language
Correct answer: Continuing training on a task-specific labeled dataset to adapt the model to a new task
Fine-tuning continues gradient-based training of a pre-trained model on a smaller task-specific dataset, adapting general representations to the target task.
Question 3: What is the primary difference between extractive and abstractive text summarization?
- Extractive summarization works only on emails; abstractive on articles
- Extractive selects existing sentences from the source; abstractive generates new sentences (Correct answer)
- Extractive uses neural networks; abstractive uses rule-based systems
- Extractive is always shorter than abstractive summaries
Correct answer: Extractive selects existing sentences from the source; abstractive generates new sentences
Extractive summarization picks and ranks existing sentences from the document, while abstractive summarization generates novel sentences that may not appear verbatim in the source.
Question 4: What is a 'language model' in NLP?
- A grammar checker for written documents
- A model that assigns probabilities to sequences of words or predicts the next word in a sequence (Correct answer)
- A model that translates between languages
- A classification model for identifying topics
Correct answer: A model that assigns probabilities to sequences of words or predicts the next word in a sequence
A language model learns the probability distribution over sequences of words, enabling tasks like text generation, completion, and scoring sentence fluency.
Question 5: Which NLP task determines whether the relationship between two sentences is entailment, contradiction, or neutral?
- Coreference resolution
- Natural language inference (Correct answer)
- Question answering
- Semantic role labeling
Correct answer: Natural language inference
Natural language inference (NLI) classifies the logical relationship between a premise and a hypothesis sentence as entailment, contradiction, or neutral.
Question 6: What does the subword tokenization algorithm BPE (Byte Pair Encoding) do?
- Splits text only on whitespace
- Iteratively merges the most frequent character pairs to build a vocabulary of subword units (Correct answer)
- Converts all tokens to a fixed embedding size
- Removes all punctuation from text
Correct answer: Iteratively merges the most frequent character pairs to build a vocabulary of subword units
BPE starts with individual characters and repeatedly merges the most frequent adjacent pair until a target vocabulary size is reached, balancing between word and character tokenization.
Artificial Intelligence Natural Language Processing 2
What architecture is BERT (Bidirectional Encoder Representations from Transformers) based on?