Natural Language Processing Fundamentals Flashcards

Which problem does the 'vanishing gradient' issue primarily affect in recurrent neural networks for NLP?

Answer: Learning long-range dependencies in sequences

Gradients shrink exponentially as they propagate back through many time steps, making it hard for RNNs to capture long-range dependencies.

In coreference resolution, what is the task the model must solve?

Answer: Clustering mentions in text that refer to the same real-world entity

Coreference resolution groups all mentions (nouns, pronouns, noun phrases) that refer to the same entity into clusters.

What is the primary goal of sequence-to-sequence (seq2seq) models in NLP?

Answer: Map an input sequence of tokens to an output sequence of potentially different length

Seq2seq models use an encoder to compress the input and a decoder to generate an output sequence, used in tasks like translation and summarization.

Which of the following is an example of an extractive summarization approach?

Answer: Selecting and concatenating important sentences directly from the source

Extractive summarization selects actual sentences or phrases from the original document rather than generating new text.

What does the term 'polysemy' mean in linguistics and NLP?

Answer: A single word form that carries multiple related meanings

Polysemy refers to a single word having multiple related senses, such as 'bank' meaning a financial institution or a riverbank.

In the encoder-decoder attention of a transformer, what do the queries come from?

Answer: The decoder's previous hidden states

In cross-attention, the decoder generates queries from its own states, while keys and values come from the encoder's output.

Which metric is commonly used to evaluate named entity recognition (NER) performance?

Answer: Entity-level F1 score

NER is evaluated with entity-level F1, which requires both the entity boundary and entity type to match for a prediction to count as correct.