MS-DS Master of Data science Master of Data Science Natural Language Processing 1 — Questions and Answers
Question 1: Which algorithm is commonly used to segment text into subword units, helping models handle out-of-vocabulary words?
- TF-IDF weighting
- Byte Pair Encoding (BPE) (Correct answer)
- Latent Dirichlet Allocation
- Viterbi decoding
Correct answer: Byte Pair Encoding (BPE)
Byte Pair Encoding iteratively merges the most frequent character pairs to build a subword vocabulary, allowing models to represent rare or unseen words as combinations of known subword tokens.
Question 2: In the transformer architecture, what does the term 'positional encoding' accomplish?
- It normalizes token embeddings to unit length
- It injects information about each token's position in the sequence (Correct answer)
- It masks future tokens during training
- It reduces the dimensionality of the key-query product
Correct answer: It injects information about each token's position in the sequence
Because transformers process all tokens in parallel without recurrence, positional encodings (sinusoidal or learned vectors) are added to embeddings so the model can distinguish token order.
Question 3: Which evaluation metric computes the geometric mean of precision and recall, and is widely used for NLP classification tasks?
- BLEU score
- Perplexity
- F1 score (Correct answer)
- ROUGE-L
Correct answer: F1 score
The F1 score is the harmonic mean of precision and recall, giving a single balanced measure especially useful when class distributions are uneven.
Question 4: What distinguishes a bidirectional LSTM from a standard (unidirectional) LSTM?
- It uses gated recurrent units instead of memory cells
- It processes sequences from right to left only
- It reads the input sequence in both forward and backward directions simultaneously (Correct answer)
- It replaces recurrence with self-attention
Correct answer: It reads the input sequence in both forward and backward directions simultaneously
A bidirectional LSTM runs two separate LSTMs — one left-to-right and one right-to-left — then concatenates their hidden states, giving each token context from both past and future tokens.
Question 5: In the context of topic modeling, what does Latent Dirichlet Allocation (LDA) assume about documents?
- Each document belongs to exactly one topic
- Documents are mixtures of topics, and topics are mixtures of words (Correct answer)
- Topics are represented as dense neural embeddings
- Word order within a document is significant for topic inference
Correct answer: Documents are mixtures of topics, and topics are mixtures of words
LDA is a generative probabilistic model that treats each document as a mixture of latent topics and each topic as a probability distribution over vocabulary words, ignoring word order (bag-of-words assumption).
Question 6: Which pre-training objective does BERT use to learn deep bidirectional representations?
- Next sentence prediction only
- Causal language modeling (predict the next token)
- Masked language modeling combined with next sentence prediction (Correct answer)
- Contrastive learning with negative sampling
Correct answer: Masked language modeling combined with next sentence prediction
BERT is pre-trained with two objectives: Masked LM (randomly masking tokens and predicting them from surrounding context) and Next Sentence Prediction (predicting whether two sentences are consecutive), enabling rich bidirectional context.
MS-DS Master of Data science Master of Data Science Natural Language Processing 1
Which algorithm is commonly used to segment text into subword units, helping models handle out-of-vocabulary words?