DSE Feature Engineering and Selection 4 — Questions and Answers
Question 1: What is the main advantage of using quantile transformation over standard normalization for highly skewed data?
- It preserves the original feature values exactly
- It maps values to a uniform or normal distribution, making it robust to outliers (Correct answer)
- It works only with categorical features
- It increases the number of features
Correct answer: It maps values to a uniform or normal distribution, making it robust to outliers
Quantile transformation maps data to a target distribution based on rank, making it robust to extreme outliers unlike mean/std-based scaling.
Question 2: In feature engineering for NLP, what does TF-IDF stand for and what does it measure?
- Text Frequency–Inverse Document Filter; filters rare words
- Term Frequency–Inverse Document Frequency; measures how important a word is to a document relative to a corpus (Correct answer)
- Token Frequency–Inverse Data Function; counts tokens per sentence
- Topic Frequency–Inverse Distribution Factor; clusters topics
Correct answer: Term Frequency–Inverse Document Frequency; measures how important a word is to a document relative to a corpus
TF-IDF weights a term by how often it appears in a document (TF) discounted by how common it is across all documents (IDF), highlighting distinctive words.
Question 3: When would you prefer forward feature selection over backward feature elimination?
- When you have very few features and need to remove some
- When the number of features is large relative to samples, making fitting a full model impractical (Correct answer)
- When all features are continuous
- When the model does not support partial feature sets
Correct answer: When the number of features is large relative to samples, making fitting a full model impractical
Forward selection starts with no features and adds one at a time, avoiding the need to fit a model on the full high-dimensional feature set.
Question 4: What is 'Weight of Evidence' (WoE) encoding primarily used for?
- Encoding ordinal features in regression tasks
- Encoding categorical features in binary classification, particularly in credit scoring models (Correct answer)
- Normalizing continuous features using Bayesian priors
- Reducing dimensionality via matrix factorization
Correct answer: Encoding categorical features in binary classification, particularly in credit scoring models
WoE encodes each category as the log ratio of the proportion of events to non-events, making it well-suited for logistic regression in credit risk models.
Question 5: What problem does 'feature hashing' (the hashing trick) solve in machine learning pipelines?
- Removes duplicate rows from the dataset
- Handles high-cardinality or unknown categorical values with a fixed-size feature vector (Correct answer)
- Converts continuous features to binary
- Eliminates multicollinearity between features
Correct answer: Handles high-cardinality or unknown categorical values with a fixed-size feature vector
Feature hashing maps categories to a fixed-size vector using a hash function, allowing the model to handle unseen categories and large vocabularies without storing a full mapping.
Question 6: What is the 'information gain' criterion in filter-based feature selection?
- The reduction in model loss when a feature is added
- The reduction in entropy of the target variable given the knowledge of a feature's values (Correct answer)
- The correlation coefficient between the feature and the target
- The percentage increase in accuracy when a feature is included
Correct answer: The reduction in entropy of the target variable given the knowledge of a feature's values
Information gain measures how much knowing a feature reduces uncertainty (entropy) about the target class, commonly used in decision tree feature ranking.
Question 7: Why is it problematic to impute missing values using the global mean before splitting data into train and test sets?
- The global mean is always a poor imputation strategy
- It introduces data leakage because the imputation uses information from the test set (Correct answer)
- The mean cannot be computed when data contains categorical features
- It increases the variance of the feature artificially
Correct answer: It introduces data leakage because the imputation uses information from the test set
Computing the mean on the full dataset before splitting means the training imputation is informed by test set values, violating the independence of the test set.
DSE Feature Engineering and Selection 4
What is the main advantage of using quantile transformation over standard normalization for highly skewed data?