Data Science with Python Certification Data Science with Python Feature Engineering Techniques 3

Question 1

In the context of text feature engineering, what does TF-IDF penalize compared to a raw term frequency vector?

Accepted Answer

Terms that appear frequently across many documents

Answer

TF-IDF down-weights terms that appear in many documents (high IDF denominator), reducing the influence of common words like 'the' or 'is'.

Question 2

Which Python library method can be used to apply a custom feature transformation to each group within a grouped DataFrame?

Accepted Answer

GroupBy.transform()

Answer

GroupBy.transform() applies a function to each group and returns a result with the same index as the original DataFrame, enabling group-level feature creation.

Question 3

When creating lag features for a time series, what must you ensure to prevent data leakage?

Accepted Answer

Future values are not used to construct features for earlier time steps

Answer

Lag features must only use past observations relative to the prediction point; using future values would constitute leakage and inflate model performance.

Question 4

What problem does the 'drop=first' option in OneHotEncoder solve when encoding binary or multi-category variables?

Accepted Answer

Eliminates perfect multicollinearity among the encoded columns

Answer

Dropping one dummy column eliminates the dummy variable trap, where one column is a perfect linear combination of the others, causing multicollinearity.

Question 5

Which sklearn class is designed to apply different transformers to different subsets of columns in a single pipeline step?

Accepted Answer

ColumnTransformer

Answer

ColumnTransformer allows specifying different preprocessing pipelines for different column subsets (e.g., numeric vs. categorical) within one object.

Data Science with Python Certification Practice Test

Data Science with Python Certification Practice Test

Data Science with Python Certification Data Science with Python Feature Engineering Techniques 3