Data Science with Python Certification Data Science with Python Natural Language Processing 2 — Questions and Answers
Question 1: Which scikit-learn class converts a collection of text documents to a matrix of TF-IDF features?
- CountVectorizer
- TfidfVectorizer (Correct answer)
- HashingVectorizer
- LabelEncoder
Correct answer: TfidfVectorizer
`TfidfVectorizer` combines tokenization, vocabulary building, and TF-IDF scoring into a single transformer compatible with scikit-learn pipelines.
Question 2: What is a bigram in NLP?
- A pair of characters used in character-level modeling
- A sequence of two consecutive words used as a single feature (Correct answer)
- A document encoded with two separate embedding methods
- A two-layer neural network for text classification
Correct answer: A sequence of two consecutive words used as a single feature
A bigram is an n-gram where n=2, capturing pairs of adjacent words (e.g., 'machine learning') to preserve some local word order context.
Question 3: In spaCy, what attribute of a `Token` object returns its part-of-speech tag?
- token.lemma_
- token.pos_ (Correct answer)
- token.dep_
- token.ent_type_
Correct answer: token.pos_
`token.pos_` returns the coarse-grained part-of-speech tag (e.g., 'NOUN', 'VERB') assigned by spaCy's statistical model.
Question 4: What is the purpose of padding sequences when preparing text data for deep learning models?
- To remove rare words below a frequency threshold
- To make all input sequences the same length for batch processing (Correct answer)
- To encode words as one-hot vectors
- To apply dropout regularization to embeddings
Correct answer: To make all input sequences the same length for batch processing
Deep learning models require fixed-size inputs; padding (usually with zeros) extends shorter sequences to a uniform length so batches can be processed efficiently.
Question 5: Which method on a scikit-learn vectorizer both learns the vocabulary and transforms the training data in one step?
- transform()
- fit()
- fit_transform() (Correct answer)
- partial_fit()
Correct answer: fit_transform()
`fit_transform()` combines `fit()` (learn vocabulary/IDF) and `transform()` (convert documents to feature matrix) in a single efficient pass over training data.
Question 6: What does Named Entity Recognition (NER) identify in text?
- Grammatical dependencies between words in a sentence
- Real-world entities such as persons, organizations, and locations (Correct answer)
- The sentiment polarity of each sentence
- Duplicate or near-duplicate sentences in a corpus
Correct answer: Real-world entities such as persons, organizations, and locations
NER classifies text spans as named entities (PERSON, ORG, GPE, DATE, etc.), enabling extraction of structured information from unstructured text.
Question 7: When using `CountVectorizer` in scikit-learn, what does setting `ngram_range=(1, 2)` do?
- Limits the vocabulary to words appearing between 1 and 2 times
- Includes both unigrams and bigrams as features in the document-term matrix (Correct answer)
- Restricts documents to a length of 1 to 2 sentences
- Applies L1 and L2 normalization to the feature vectors
Correct answer: Includes both unigrams and bigrams as features in the document-term matrix
`ngram_range=(1, 2)` instructs the vectorizer to extract all unigrams (single words) and bigrams (word pairs), increasing feature richness at the cost of a larger vocabulary.
Data Science with Python Certification Data Science with Python Natural Language Processing 2
Which scikit-learn class converts a collection of text documents to a matrix of TF-IDF features?