MS-DS Master of Data science Feature Engineering 2 — Questions and Answers
Question 1: Which of the following is a valid technique to handle high-cardinality categorical features?
- One-hot encoding all unique values
- Target encoding with cross-validation (Correct answer)
- Treating every category as an independent binary feature
- Applying PCA to the raw strings
Correct answer: Target encoding with cross-validation
Target encoding with cross-validation handles high cardinality by summarizing each category with the mean target value while preventing data leakage through cross-validation folds.
Question 2: In time series feature engineering, what is a 'lag feature'?
- A feature that measures processing delay in the pipeline
- A feature whose value is the observation from a prior time step (Correct answer)
- A feature derived from the frequency spectrum of the time series
- A feature that encodes the time zone of each observation
Correct answer: A feature whose value is the observation from a prior time step
A lag feature captures the value of a variable at a previous time step, allowing models to learn temporal dependencies such as how past values influence current outcomes.
Question 3: What is the 'curse of dimensionality' and how does it motivate feature selection?
- Computational overhead increases linearly with features; motivates GPU usage
- Data becomes increasingly sparse as dimensions grow, degrading model performance; motivates removing irrelevant features (Correct answer)
- Memory limitations prevent storing high-dimensional arrays; motivates data compression
- Gradient descent diverges in high-dimensional spaces; motivates learning rate tuning
Correct answer: Data becomes increasingly sparse as dimensions grow, degrading model performance; motivates removing irrelevant features
As the number of features grows, the data becomes exponentially sparser in the feature space, making distance metrics unreliable and models prone to overfitting, which motivates removing irrelevant or redundant features.
Question 4: Which method uses a model's built-in feature importance scores to select features?
- Filter method
- Wrapper method
- Embedded method (Correct answer)
- Projection method
Correct answer: Embedded method
Embedded methods perform feature selection as part of the model training process itself (e.g., LASSO regression or tree-based feature importances), integrating selection and learning simultaneously.
Question 5: What transformation is typically applied to convert a cyclical feature such as 'hour of day' into usable numerical features?
- Label encoding the hour values 0–23
- Min-max scaling the hour to [0, 1]
- Applying sine and cosine transformations based on the cycle length (Correct answer)
- Converting hour to a categorical ordinal variable
Correct answer: Applying sine and cosine transformations based on the cycle length
Sine and cosine transformations map cyclical features onto a circle, ensuring that the model perceives hour 23 and hour 0 as close to each other rather than far apart.
Question 6: When using Recursive Feature Elimination (RFE), how does the algorithm select features?
- It trains a model on all features and removes the least important feature iteratively until the desired number remains (Correct answer)
- It ranks features by correlation with the target and selects the top-k
- It projects features onto orthogonal components and selects those with highest variance
- It tests every possible subset of features and chooses the subset with best cross-validation score
Correct answer: It trains a model on all features and removes the least important feature iteratively until the desired number remains
RFE trains a model, removes the feature with the lowest importance score, retrains, and repeats this process iteratively until the specified number of features remains.
Question 7: What is the main advantage of using Polynomial Features in scikit-learn for feature engineering?
- It reduces the feature space by combining redundant variables
- It generates interaction terms and higher-degree features to capture non-linear relationships (Correct answer)
- It applies a kernel function to project data into infinite-dimensional space
- It automatically selects the optimal polynomial degree via cross-validation
Correct answer: It generates interaction terms and higher-degree features to capture non-linear relationships
Polynomial Features creates all combinations of feature interactions and powers up to a specified degree, enabling linear models to capture non-linear patterns in the data.
MS-DS Master of Data science Feature Engineering 2
Which of the following is a valid technique to handle high-cardinality categorical features?