Feature Engineering and Selection Flashcards

A data scientist is preparing a dataset with a categorical feature "City" ('New York', 'London', 'Tokyo'). The feature has high cardinality and no inherent order. Which encoding technique is most appropriate to convert this feature for a linear model without imposing a false ordinal relationship?

Answer: One-Hot Encoding

One-Hot Encoding is the correct method for nominal categorical variables (where no order exists) that will be used in linear models. It creates new binary columns for each category, representing its presence or absence without implying any sort of ranking, which would be an incorrect assumption for a feature like 'City'.

Which category of feature selection methods evaluates feature subsets by training and testing a specific machine learning model on each subset?

Answer: Wrapper Methods

Wrapper methods use the performance of a chosen machine learning model to assess the quality of a subset of features. Methods like Recursive Feature Elimination (RFE) and Sequential Feature Selection (SFS) are classic examples where a model is repeatedly trained to find the optimal feature set.

You are building a model using an algorithm that relies on distance calculations, such as K-Nearest Neighbors. Your dataset contains features with vastly different scales: 'Age' (18-90) and 'Annual_Income' (25,000-500,000). Which feature engineering step is most critical to ensure model performance is not biased by feature scale?

Answer: Applying standardization or normalization.

For distance-based algorithms, features with larger scales (like 'Annual_Income') can dominate the distance metric, making the contribution of features with smaller scales (like 'Age') insignificant. Standardization (e.g., StandardScaler) or normalization (e.g., MinMaxScaler) rescales features to a comparable range, ensuring that each feature contributes more equally to the result.

A data scientist is working with a retail dataset that includes 'price_per_item' and 'quantity_sold' for each transaction. To improve a sales prediction model, they create a new feature called 'total_transaction_value' by multiplying these two features. What is this process an example of?

Answer: Creating Interaction Features

Creating a new feature by combining two or more existing features (through multiplication, division, addition, etc.) is known as creating an interaction feature. This technique can help a model capture a combined effect or relationship between variables that it might not find on its own.

When dealing with missing numerical data in a feature that is known to have a skewed distribution with significant outliers, which imputation method is generally the most robust?

Answer: Median Imputation

The mean is highly sensitive to outliers and skewness, as extreme values can distort it significantly. The median, representing the middle value of the sorted data, is not affected by the magnitude of outliers, making it a more robust and representative measure of central tendency for skewed distributions.

Which of the following is a key characteristic of embedded feature selection methods?

Answer: The feature selection process is an intrinsic part of the model training algorithm itself.

Embedded methods, such as L1 (Lasso) regularization and the feature importance calculated by tree-based models, perform feature selection naturally during the model fitting process. This is different from filter methods (pre-processing) and wrapper methods (which train a model multiple times).