Text Mining & Natural Language Processing Flashcards

Which NLP technique is used to reduce words to their base or root form by removing suffixes and prefixes?

Answer: Stemming

Stemming reduces words to their root form (e.g., 'running' → 'run') by stripping affixes, enabling normalization before text analysis.

In text mining, what does TF-IDF stand for?

Answer: Term Frequency–Inverse Document Frequency

TF-IDF (Term Frequency–Inverse Document Frequency) weights terms by how often they appear in a document relative to how common they are across the corpus.

Which process converts raw text into a list of individual words or tokens?

Answer: Tokenization

Tokenization splits text into its smallest meaningful units (tokens), which is typically the first step in any text-mining pipeline.

What is a 'bag-of-words' model in text mining?

Answer: A representation that counts word occurrences while ignoring order

The bag-of-words model represents text as an unordered collection of word counts, discarding syntax and position information.

Which category of words is typically removed during text preprocessing because they carry little semantic meaning?

Answer: Stop words

Stop words (e.g., 'the', 'is', 'and') are filtered out because they appear frequently but contribute minimal discriminative information.

Sentiment analysis is best described as which type of text-mining task?

Answer: Opinion classification

Sentiment analysis classifies text (positive, negative, neutral) based on the author's expressed opinion, making it a classification task.

Which dimensionality reduction technique is commonly used in text mining to uncover latent topics in a document collection?

Answer: Latent Dirichlet Allocation (LDA)

Latent Dirichlet Allocation (LDA) is a generative probabilistic model that discovers hidden topic distributions across a corpus.