NLP Text Preprocessing 3 — Questions and Answers
Question 1: Which regular expression pattern correctly matches any whitespace character in Python's `re` module?
- \w
- \s (Correct answer)
- \d
- \b
Correct answer: \s
\s matches any whitespace character including spaces, tabs, newlines, and other Unicode whitespace.
Question 2: What is Unicode normalization form NFC used for in text preprocessing?
- Removing all non-ASCII characters from text
- Composing characters into their canonical composed form (Correct answer)
- Converting text to lowercase ASCII only
- Splitting compound words into morphemes
Correct answer: Composing characters into their canonical composed form
NFC (Canonical Decomposition followed by Canonical Composition) ensures characters with diacritics are stored in a single composed code point rather than multiple characters.
Question 3: In the context of text preprocessing for social media data, what does 'denoising' typically involve?
- Removing audio artifacts from voice transcriptions
- Cleaning hashtags, URLs, emojis, and slang from text (Correct answer)
- Applying Gaussian filters to word embeddings
- Normalizing sentence lengths to equal sizes
Correct answer: Cleaning hashtags, URLs, emojis, and slang from text
Denoising social media text involves removing or normalizing noisy elements like URLs, hashtags, @ mentions, emojis, and informal spelling.
Question 4: What does the 'max_features' parameter control in scikit-learn's CountVectorizer?
- The maximum number of tokens per document
- The size of the vocabulary built from top frequent terms (Correct answer)
- The maximum n-gram range allowed
- The maximum document length in characters
Correct answer: The size of the vocabulary built from top frequent terms
max_features limits the vocabulary to the top N most frequent terms across the corpus, reducing dimensionality.
Question 5: Which tokenization approach handles contractions like "don't" most correctly for downstream NLP tasks?
- Keeping 'don't' as a single token
- Splitting into 'don' and 't'
- Splitting into 'do' and "n't" as a negation marker (Correct answer)
- Replacing with 'do not' before tokenizing
Correct answer: Splitting into 'do' and "n't" as a negation marker
Splitting into 'do' and "n't" preserves the negation information as a distinct token, which is linguistically motivated and useful for sentiment analysis.
Question 6: What is the purpose of applying a minimum document frequency (min_df) threshold in text vectorization?
- To ensure all documents have the same length
- To remove rare terms that appear in fewer than N documents (Correct answer)
- To cap the maximum term frequency per document
- To filter out terms longer than a defined character limit
Correct answer: To remove rare terms that appear in fewer than N documents
min_df removes terms that appear in fewer than the specified number (or fraction) of documents, eliminating noise from very rare words.
Question 7: Which of the following is an example of a morphological inflection that stemming is designed to handle?
- 'bank' (financial) vs 'bank' (river)
- 'run', 'runs', 'running', 'ran' (Correct answer)
- 'New York' as a single named entity
- '2024' as a numeric token
Correct answer: 'run', 'runs', 'running', 'ran'
Stemming normalizes morphological variants like run/runs/running/ran to a common stem, reducing vocabulary size.
NLP Text Preprocessing 3
Which regular expression pattern correctly matches any whitespace character in Python's `re` module?