Named Entity Recognition Flashcards

Which character-level feature is commonly added to NER models to help identify proper nouns?

Answer: Capitalization features

Capitalization is a strong surface cue for proper nouns (entities) in English and many other languages, so it is a standard NER feature.

In transformer-based NER, why is subword tokenization (e.g., WordPiece) potentially problematic?

Answer: Entity boundaries may fall within a subword split, complicating label alignment

Subword tokenization can split a single word into multiple tokens, requiring strategies to align word-level NER labels with subword tokens.

What is 'gazetteer' in the context of NER?

Answer: A list of known entities used as a lookup feature

A gazetteer is a dictionary of known entity names (e.g., cities, companies) whose presence in text provides strong evidence for NER models.

Which NER approach does NOT require annotated training data?

Answer: Dictionary-based / rule-based NER

Dictionary-based and rule-based NER systems rely on predefined lists and patterns rather than labeled examples, requiring no annotated corpus.

What is 'entity linking' (also called entity disambiguation) in relation to NER?

Answer: Mapping a recognized entity mention to a specific entry in a knowledge base

Entity linking maps surface mentions (e.g., 'Apple') identified by NER to canonical knowledge base entries (e.g., Apple Inc. vs. the fruit).

Which of the following best describes the 'O' label in BIO tagging?

Answer: A token that belongs to no named entity

In BIO tagging, O (Outside) marks tokens that are not part of any named entity, forming the majority class in most NER datasets.

What technique is used to handle the class imbalance problem in NER, where O labels vastly outnumber entity labels?

Answer: Focal loss or weighted cross-entropy

Focal loss down-weights easy (O-label) examples and focal cross-entropy assigns higher loss weight to rare entity classes to counter imbalance.