Data Science with Python Certification Data Science with Python Feature Engineering Techniques 3 — Questions and Answers
Question 1: In the context of text feature engineering, what does TF-IDF penalize compared to a raw term frequency vector?
- Terms that appear in very few documents
- Terms that appear frequently across many documents (Correct answer)
- Terms with very short character length
- Terms that appear only once in a document
Correct answer: Terms that appear frequently across many documents
TF-IDF down-weights terms that appear in many documents (high IDF denominator), reducing the influence of common words like 'the' or 'is'.
Question 2: Which Python library method can be used to apply a custom feature transformation to each group within a grouped DataFrame?
- DataFrame.apply()
- GroupBy.transform() (Correct answer)
- DataFrame.map()
- GroupBy.aggregate()
Correct answer: GroupBy.transform()
GroupBy.transform() applies a function to each group and returns a result with the same index as the original DataFrame, enabling group-level feature creation.
Question 3: When creating lag features for a time series, what must you ensure to prevent data leakage?
- The lag values are standardized before use
- Future values are not used to construct features for earlier time steps (Correct answer)
- All lag features are log-transformed
- Lag features are one-hot encoded
Correct answer: Future values are not used to construct features for earlier time steps
Lag features must only use past observations relative to the prediction point; using future values would constitute leakage and inflate model performance.
Question 4: What problem does the 'drop=first' option in OneHotEncoder solve when encoding binary or multi-category variables?
- Prevents duplicate rows in the dataset
- Eliminates perfect multicollinearity among the encoded columns (Correct answer)
- Reduces the number of NaN values after encoding
- Ensures the encoded columns are normalized
Correct answer: Eliminates perfect multicollinearity among the encoded columns
Dropping one dummy column eliminates the dummy variable trap, where one column is a perfect linear combination of the others, causing multicollinearity.
Question 5: Which sklearn class is designed to apply different transformers to different subsets of columns in a single pipeline step?
- Pipeline
- FeatureUnion
- ColumnTransformer (Correct answer)
- FunctionTransformer
Correct answer: ColumnTransformer
ColumnTransformer allows specifying different preprocessing pipelines for different column subsets (e.g., numeric vs. categorical) within one object.
Question 6: What is 'mean encoding' also commonly called in feature engineering literature?
- Frequency encoding
- Target encoding (Correct answer)
- Binary encoding
- Hash encoding
Correct answer: Target encoding
Mean encoding and target encoding refer to the same technique: replacing a categorical value with the mean of the target variable for that category.
Question 7: Which method in pandas computes rolling statistics over a sliding window along a Series, useful for time-series feature engineering?
- Series.expanding()
- Series.rolling() (Correct answer)
- Series.resample()
- Series.shift()
Correct answer: Series.rolling()
Series.rolling(window=n) creates a rolling window object allowing computation of statistics like mean, std, or min over the last n observations.
Data Science with Python Certification Data Science with Python Feature Engineering Techniques 3
In the context of text feature engineering, what does TF-IDF penalize compared to a raw term frequency vector?