Machine Learning Feature Engineering 4 — Questions and Answers
Question 1: Which of the following is an example of a 'date-derived' feature useful for forecasting retail sales?
- Customer age
- Day of the week (Correct answer)
- Product price
- Store location latitude
Correct answer: Day of the week
Extracting the day of the week from a timestamp allows models to capture weekly seasonality patterns in sales data.
Question 2: What is 'recursive feature elimination' (RFE)?
- Removing features with zero variance iteratively
- Training a model and repeatedly removing the least important features until a target count is reached (Correct answer)
- Selecting features based on correlation with the target
- Projecting features onto principal components recursively
Correct answer: Training a model and repeatedly removing the least important features until a target count is reached
RFE fits a model, ranks features by importance, removes the weakest feature(s), and repeats until the desired number of features remains.
Question 3: What does the Variance Inflation Factor (VIF) measure in the context of feature engineering?
- The importance of a feature for the target variable
- The degree of multicollinearity between a feature and other features (Correct answer)
- The proportion of variance explained by a feature
- The ratio of between-group variance to within-group variance
Correct answer: The degree of multicollinearity between a feature and other features
A high VIF for a feature indicates it is highly correlated with other features, which can destabilize regression coefficient estimates.
Question 4: Which technique helps prevent data leakage when encoding categorical variables using cross-validation?
- Applying encoding after the train-test split on the full dataset
- Using out-of-fold target encoding so each fold is encoded using only the other folds (Correct answer)
- Fitting the encoder on the test set
- Using one-hot encoding instead
Correct answer: Using out-of-fold target encoding so each fold is encoded using only the other folds
Out-of-fold (cross-validated) encoding ensures that the target statistics used to encode each sample are computed from different samples, preventing leakage.
Question 5: In NLP feature engineering, what does TF-IDF downweight relative to raw term frequency?
- Rare words that appear in few documents
- Common words that appear across many documents (Correct answer)
- Words with high sentiment scores
- Short words with fewer than four characters
Correct answer: Common words that appear across many documents
TF-IDF penalizes terms appearing in many documents (high document frequency) because they are less discriminative between documents.
Question 6: What is the purpose of 'feature crosses' in models like logistic regression?
- To remove redundant features from the dataset
- To allow linear models to learn non-linear decision boundaries by combining features (Correct answer)
- To scale features to the same range
- To impute missing values using neighboring features
Correct answer: To allow linear models to learn non-linear decision boundaries by combining features
Feature crosses create new features by combining two or more categorical or discretized features, enabling linear models to learn conjunctive rules.
Question 7: Which of the following correctly describes 'robust scaling' (RobustScaler)?
- Scales features using the mean and standard deviation
- Scales features using the median and interquartile range, making it resistant to outliers (Correct answer)
- Clips feature values to a fixed range
- Applies a log transform before min-max scaling
Correct answer: Scales features using the median and interquartile range, making it resistant to outliers
RobustScaler subtracts the median and divides by the IQR, so outliers have minimal influence on the scaling parameters.
Machine Learning Feature Engineering 4
Which of the following is an example of a 'date-derived' feature useful for forecasting retail sales?