Machine Learning Feature Engineering 2 — Questions and Answers
Question 1: Which technique replaces a categorical feature with the mean of the target variable for each category?
- One-hot encoding
- Target encoding (Correct answer)
- Label encoding
- Frequency encoding
Correct answer: Target encoding
Target encoding substitutes each category with the mean target value, which can capture ordinal relationships in high-cardinality features.
Question 2: What is the main risk of applying target encoding without regularization?
- Underfitting due to information loss
- Target leakage causing data leakage (Correct answer)
- Increased dimensionality
- Loss of feature interpretability
Correct answer: Target leakage causing data leakage
Without regularization or cross-fitting, target encoding can leak target information into the training features, causing overfitting.
Question 3: What does the Box-Cox transformation require that the Yeo-Johnson transformation does not?
- A predefined lambda parameter
- Positive input values only (Correct answer)
- Normally distributed inputs
- Standardized features
Correct answer: Positive input values only
Box-Cox only works on strictly positive values, while Yeo-Johnson handles zero and negative values as well.
Question 4: When creating polynomial features from two features x1 and x2 of degree 2, which interaction term is included?
- x1 + x2
- x1 * x2 (Correct answer)
- x1 / x2
- x1 - x2
Correct answer: x1 * x2
Degree-2 polynomial features include squared terms (x1², x2²) and the cross-product interaction term x1*x2.
Question 5: Which imputation strategy is most appropriate for a feature with a heavy right-skewed distribution?
- Mean imputation
- Median imputation (Correct answer)
- Mode imputation
- Zero imputation
Correct answer: Median imputation
The median is robust to outliers and skew, making it more representative than the mean for skewed distributions.
Question 6: What is 'feature hashing' (the hashing trick) primarily used for?
- Encrypting sensitive feature values
- Reducing high-cardinality categorical features to a fixed-size vector (Correct answer)
- Normalizing numerical features
- Detecting duplicate rows in a dataset
Correct answer: Reducing high-cardinality categorical features to a fixed-size vector
Feature hashing maps categories to indices in a fixed-size array using a hash function, controlling dimensionality without a full vocabulary.
Question 7: In time-series feature engineering, what does a 'lag feature' represent?
- The moving average of the series
- The value of a variable at a previous time step (Correct answer)
- The rate of change between consecutive observations
- The Fourier transform of the signal
Correct answer: The value of a variable at a previous time step
A lag feature captures the value of the target or a predictor at an earlier time point, allowing models to learn temporal dependencies.
Machine Learning Feature Engineering 2
Which technique replaces a categorical feature with the mean of the target variable for each category?