DSE Feature Engineering and Selection 3 — Questions and Answers
Question 1: What is the 'curse of dimensionality' and how does it affect feature engineering decisions?
- Too many missing values making imputation impossible
- Data becoming increasingly sparse as the number of features grows, degrading model performance (Correct answer)
- Categorical features having too many unique levels
- The cost of computing feature interactions growing linearly
Correct answer: Data becoming increasingly sparse as the number of features grows, degrading model performance
As dimensionality increases, data points become sparse and distance metrics lose meaning, motivating dimensionality reduction and careful feature selection.
Question 2: Which of the following is an advantage of mean encoding (target encoding) over one-hot encoding for high-cardinality categoricals?
- It avoids introducing any new information from the target
- It reduces the dimensionality increase caused by many unique categories (Correct answer)
- It works without any training data
- It never causes overfitting
Correct answer: It reduces the dimensionality increase caused by many unique categories
Mean encoding replaces each category with the mean target value, keeping the feature as a single column instead of creating hundreds of binary columns.
Question 3: In time-series feature engineering, what is a 'lag feature'?
- A feature derived from future values to predict the past
- A feature that uses a past observation's value as a predictor for the current time step (Correct answer)
- A feature representing the difference between two non-adjacent time points divided by time
- A feature that encodes the weekday of a timestamp
Correct answer: A feature that uses a past observation's value as a predictor for the current time step
Lag features use values from previous time steps (e.g., sales yesterday) as predictors for the current time step.
Question 4: What does the Variance Inflation Factor (VIF) measure in feature selection?
- The variance of a single feature relative to the target
- The degree to which a feature's variance is explained by other features, indicating multicollinearity (Correct answer)
- The ratio of between-class variance to within-class variance
- The proportion of variance captured by each principal component
Correct answer: The degree to which a feature's variance is explained by other features, indicating multicollinearity
VIF quantifies how much a feature's variance is inflated due to correlation with other features; VIF > 10 typically signals severe multicollinearity.
Question 5: What is 'feature crosses' as used in systems like TensorFlow Feature Columns?
- Removing features that cross a correlation threshold
- Synthetic features created by crossing (combining) two or more categorical features (Correct answer)
- Splitting a continuous feature into categorical bins
- Averaging features across rows
Correct answer: Synthetic features created by crossing (combining) two or more categorical features
Feature crosses combine categorical features to create new synthetic features that capture interactions, such as crossing 'city' and 'day_of_week'.
Question 6: Which of the following best describes 'polynomial feature expansion'?
- Encoding each category as a polynomial function
- Generating new features as powers and cross-products of original features up to a specified degree (Correct answer)
- Reducing features using polynomial regression
- Applying a polynomial activation function to feature values
Correct answer: Generating new features as powers and cross-products of original features up to a specified degree
Polynomial expansion creates new features like x², x³, and x₁·x₂ to allow linear models to fit non-linear relationships.
Question 7: Why might you use Recursive Feature Elimination (RFE) instead of simply selecting the top-k features by importance score?
- RFE is faster than importance-score ranking
- RFE accounts for feature interactions by re-evaluating importances after each removal (Correct answer)
- RFE does not require a trained model
- RFE always selects fewer features than importance ranking
Correct answer: RFE accounts for feature interactions by re-evaluating importances after each removal
RFE iteratively removes the least important features and retrains, so the importance of remaining features is reassessed in the context of those still present.
DSE Feature Engineering and Selection 3
What is the 'curse of dimensionality' and how does it affect feature engineering decisions?