Data Science Feature Engineering and Selection 3 — Questions and Answers
Question 1: Which feature selection method evaluates subsets of features using the actual model's performance?
- Wrapper methods (Correct answer)
- Filter methods
- One-hot encoding
- Normalization
Correct answer: Wrapper methods
Wrapper methods (e.g., recursive feature elimination) train the model to score feature subsets.
Question 2: Filter methods select features based on:
- Statistical measures independent of any model (Correct answer)
- The trained model's coefficients
- Cross-validated model accuracy
- Gradient updates during training
Correct answer: Statistical measures independent of any model
Filter methods rank features by statistics like correlation or chi-square, independent of a model.
Question 3: L1 (Lasso) regularization performs feature selection by:
- Shrinking some coefficients exactly to zero (Correct answer)
- Squaring all coefficients
- Adding interaction terms
- Scaling features to unit variance
Correct answer: Shrinking some coefficients exactly to zero
L1 penalty can drive coefficients to exactly zero, effectively removing those features.
Question 4: Two features have a correlation of 0.97. What is a common action in feature selection?
- Drop one of the redundant features (Correct answer)
- Multiply them together
- Keep both and double their weight
- Convert both to categorical
Correct answer: Drop one of the redundant features
Highly correlated features are redundant, so dropping one reduces multicollinearity.
Question 5: Which metric measures multicollinearity among features?
- Variance Inflation Factor (VIF) (Correct answer)
- F1 score
- ROC AUC
- Silhouette score
Correct answer: Variance Inflation Factor (VIF)
VIF quantifies how much a feature is linearly explained by the others.
Question 6: Recursive Feature Elimination (RFE) works by:
- Iteratively removing the least important features and refitting (Correct answer)
- Adding random features each round
- Scaling features recursively
- Imputing missing values repeatedly
Correct answer: Iteratively removing the least important features and refitting
RFE repeatedly trains the model and prunes the weakest features until the target count remains.
Question 7: A filter method using mutual information captures:
- Both linear and non-linear dependency with the target (Correct answer)
- Only linear correlation
- Only the feature's variance
- The model's training time
Correct answer: Both linear and non-linear dependency with the target
Mutual information measures any statistical dependency, including non-linear relationships.
Data Science Feature Engineering and Selection 3
Which feature selection method evaluates subsets of features using the actual model's performance?