Data Science with Python Certification Data Science with Python Natural Language Processing

Question 1

Which Python library provides the `word_tokenize()` function for splitting text into individual tokens?

Accepted Answer

NLTK

Answer

NLTK (Natural Language Toolkit) provides `word_tokenize()`, which splits a string into a list of word and punctuation tokens.

Question 2

What is the primary purpose of removing stop words in NLP preprocessing?

Accepted Answer

To reduce vocabulary size and eliminate low-information words

Answer

Stop words like 'the', 'is', and 'and' carry little semantic meaning; removing them reduces noise and shrinks the feature space.

Question 3

What is the difference between stemming and lemmatization in NLP?

Accepted Answer

Stemming crudely chops suffixes while lemmatization returns a valid dictionary base form

Answer

Stemming applies heuristic suffix stripping (e.g., 'running' → 'run'), while lemmatization uses morphological analysis to return the proper dictionary form.

Question 4

In Python's NLTK library, which class implements Porter stemming?

Accepted Answer

nltk.stem.PorterStemmer

Answer

`nltk.stem.PorterStemmer` implements the Porter algorithm, one of the oldest and most commonly used stemming algorithms.

Question 5

What does a 'corpus' refer to in the context of NLP?

Accepted Answer

A large structured collection of text used for training or analysis

Answer

A corpus is a large, structured collection of text documents used to train models, build vocabulary, or perform linguistic analysis.

Data Science with Python Certification Practice Test

Data Science with Python Certification Practice Test

Data Science with Python Certification Data Science with Python Natural Language Processing