DMC Text Mining & Natural Language Processing 2 — Questions and Answers
Question 1: What distinguishes lemmatization from stemming in NLP preprocessing?
- Lemmatization uses dictionary lookup to return valid base words; stemming applies heuristic rules (Correct answer)
- Stemming is slower and more accurate than lemmatization
- Lemmatization removes stop words; stemming does not
- Both are identical processes with different names
Correct answer: Lemmatization uses dictionary lookup to return valid base words; stemming applies heuristic rules
Lemmatization uses morphological analysis and a dictionary to return the canonical base form, producing valid words, while stemming uses rule-based trimming that may yield non-words.
Question 2: In the context of Named Entity Recognition (NER), which of the following is a typical entity category?
- Sentiment polarity
- Part-of-speech tag
- Organization name (Correct answer)
- Token frequency
Correct answer: Organization name
NER identifies and classifies real-world entities such as persons, organizations, and locations within text.
Question 3: Word embeddings like Word2Vec differ from bag-of-words representations primarily because they:
- Ignore word frequency entirely
- Capture semantic similarity through dense vector representations (Correct answer)
- Require no training data to generate
- Only work on numeric data
Correct answer: Capture semantic similarity through dense vector representations
Word2Vec trains neural networks to place semantically similar words close together in a dense vector space, capturing meaning that bag-of-words cannot.
Question 4: Which evaluation metric is most appropriate when measuring a text classifier's performance on an imbalanced dataset?
- Accuracy
- F1-Score (Correct answer)
- Mean Squared Error
- R-squared
Correct answer: F1-Score
F1-Score balances precision and recall, making it suitable when class imbalance means accuracy can be misleadingly high.
Question 5: What is the primary purpose of a confusion matrix in text classification evaluation?
- To visualize word frequency distributions across documents
- To show true/false positive and negative prediction counts per class (Correct answer)
- To reduce feature dimensions before model training
- To cluster similar documents into topic groups
Correct answer: To show true/false positive and negative prediction counts per class
A confusion matrix tabulates correct and incorrect predictions for each class, revealing where a classifier makes errors.
Question 6: In information retrieval and text mining, precision is defined as:
- The fraction of relevant documents retrieved out of all documents in the corpus
- The fraction of retrieved documents that are actually relevant (Correct answer)
- The total number of relevant documents in the dataset
- The harmonic mean of recall and F1-score
Correct answer: The fraction of retrieved documents that are actually relevant
Precision measures the proportion of retrieved items that are truly relevant, reflecting the quality of positive predictions.
Question 7: Which text-mining approach uses a trained model to automatically assign predefined categories to incoming documents?
- Document clustering
- Text classification (Correct answer)
- Association rule mining
- Sequence analysis
Correct answer: Text classification
Text classification trains a model on labeled examples so it can assign new documents to predefined categories such as spam/not-spam.
DMC Text Mining & Natural Language Processing 2
What distinguishes lemmatization from stemming in NLP preprocessing?