Machine Learning Feature Engineering 3 — Questions and Answers
Question 1: What is the purpose of creating 'interaction features' between two variables?
- To reduce the number of features in the dataset
- To capture combined effects that neither feature expresses alone (Correct answer)
- To normalize feature scales
- To handle missing values
Correct answer: To capture combined effects that neither feature expresses alone
Interaction features represent the joint effect of two variables, enabling linear models to learn relationships that depend on both simultaneously.
Question 2: Which of the following best describes 'binning' (discretization) of a continuous feature?
- Normalizing the feature to a [0,1] range
- Converting continuous values into discrete intervals or categories (Correct answer)
- Removing outliers beyond a threshold
- Replacing missing values with the median
Correct answer: Converting continuous values into discrete intervals or categories
Binning groups continuous values into buckets, which can help linear models capture non-linear patterns and reduce the effect of outliers.
Question 3: What does 'mean normalization' do to a feature?
- Scales values to have unit variance only
- Subtracts the mean and divides by the range (max - min) (Correct answer)
- Clips values to three standard deviations
- Converts values to their z-scores
Correct answer: Subtracts the mean and divides by the range (max - min)
Mean normalization centers the feature at zero and scales it by the range, resulting in values roughly between -1 and 1.
Question 4: What is the primary advantage of using 'embeddings' over one-hot encoding for high-cardinality categorical variables?
- Embeddings are always faster to compute
- Embeddings produce dense, low-dimensional representations that capture semantic similarity (Correct answer)
- Embeddings avoid any risk of overfitting
- Embeddings require no training data
Correct answer: Embeddings produce dense, low-dimensional representations that capture semantic similarity
Embeddings learn a compact dense vector per category, capturing relationships between categories in a low-dimensional space unlike sparse one-hot vectors.
Question 5: Which feature selection method uses model coefficients or feature importances to rank features?
- Filter method
- Wrapper method
- Embedded method (Correct answer)
- Projection method
Correct answer: Embedded method
Embedded methods (like Lasso regularization or tree feature importances) perform feature selection as part of the model training process.
Question 6: When should you apply log transformation to a target variable rather than to input features?
- When input features are highly correlated
- When the target variable is right-skewed and spans several orders of magnitude (Correct answer)
- When the model is a support vector machine
- When you want to reduce the number of features
Correct answer: When the target variable is right-skewed and spans several orders of magnitude
Log-transforming a heavily skewed target compresses large values and can make residuals more homoscedastic, improving regression model performance.
Question 7: What problem does 'count encoding' address for categorical features?
- It resolves class imbalance by oversampling rare categories
- It encodes categories by how frequently they appear in the dataset (Correct answer)
- It removes categories with very low counts
- It one-hot encodes only the top-k most frequent categories
Correct answer: It encodes categories by how frequently they appear in the dataset
Count encoding replaces each category with its occurrence frequency, providing ordinal information about how common a category is.
Machine Learning Feature Engineering 3
What is the purpose of creating 'interaction features' between two variables?