Data Science with Python Natural Language Processing Flashcards
7 cards from real Data Science with Python Certification practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 Data Science with Python Natural Language Processing flashcards as text
Which Python library provides the `word_tokenize()` function for splitting text into individual tokens?
Answer: NLTK
NLTK (Natural Language Toolkit) provides `word_tokenize()`, which splits a string into a list of word and punctuation tokens.
What is the primary purpose of removing stop words in NLP preprocessing?
Answer: To reduce vocabulary size and eliminate low-information words
Stop words like 'the', 'is', and 'and' carry little semantic meaning; removing them reduces noise and shrinks the feature space.
What is the difference between stemming and lemmatization in NLP?
Answer: Stemming crudely chops suffixes while lemmatization returns a valid dictionary base form
Stemming applies heuristic suffix stripping (e.g., 'running' → 'run'), while lemmatization uses morphological analysis to return the proper dictionary form.
In Python's NLTK library, which class implements Porter stemming?
Answer: nltk.stem.PorterStemmer
`nltk.stem.PorterStemmer` implements the Porter algorithm, one of the oldest and most commonly used stemming algorithms.
What does a 'corpus' refer to in the context of NLP?
Answer: A large structured collection of text used for training or analysis
A corpus is a large, structured collection of text documents used to train models, build vocabulary, or perform linguistic analysis.
Which of the following best describes a 'bag of words' (BoW) representation?
Answer: A document representation as an unordered set of word frequency counts
BoW represents a document as a vector of word occurrence counts, discarding grammar and word order entirely.
What does TF-IDF measure in text analysis?
Answer: The importance of a word to a document relative to a collection of documents
TF-IDF (Term Frequency–Inverse Document Frequency) scores a term by how often it appears in a document (TF) discounted by how common it is across all documents (IDF), highlighting distinctive words.
How this page was made: written with AI tools and published by Nathan Beja, founder of Practice Test Geeks, who is responsible for its content. We have not yet matched this exam's facts to an official source, so confirm fees, dates and rules with the exam body. Not reviewed by a subject-matter expert. Editorial policy