Feature Engineering and Selection Flashcards

A data scientist is working with a dataset that includes a 'temperature' feature with continuous values. To improve model performance, they decide to convert this feature into categories like 'Cold', 'Warm', and 'Hot'. Which feature engineering technique is being applied?

Answer: Binning

Binning, also known as discretization, is the process of transforming continuous numerical variables into discrete categorical intervals or 'bins'. This can help models capture non-linear relationships. [8, 22, 24]

In a dataset for predicting customer churn, a categorical feature 'Subscription_Tier' has three levels: 'Basic', 'Standard', and 'Premium'. A data scientist uses one-hot encoding on this feature. What is the primary reason for choosing one-hot encoding over label encoding in this scenario?

Answer: To prevent the model from assuming a false ordinal relationship between the categories.

One-hot encoding is used for nominal categorical variables where no intrinsic order exists. It creates new binary columns for each category to avoid implying a ranked relationship (e.g., Premium > Standard > Basic), which label encoding might suggest to the model. [20, 21, 27, 34]

A machine learning engineer is using L1 regularization (Lasso) for a linear regression model with a large number of features. What is a key benefit of using L1 regularization in this context?

Answer: It can perform automatic feature selection by shrinking some feature coefficients to exactly zero.

A primary advantage of L1 regularization (Lasso) is its ability to produce sparse models. It adds a penalty equal to the absolute value of the magnitude of coefficients, which can force the coefficients of less important features to become exactly zero, effectively removing them from the model. [2, 11, 28, 29]

Which of the following feature selection methods is characterized by its use of a predictive model to evaluate the usefulness of a feature subset, but is computationally expensive due to its iterative nature?

Answer: Wrapper methods

Wrapper methods use a specific machine learning algorithm to evaluate the quality of a subset of features. They train and test the model with different feature subsets (e.g., through forward selection or backward elimination) to find the optimal combination, which makes them computationally intensive but often leads to better performance for the chosen model. [16, 17, 18]

A data scientist is tasked with reducing the dimensionality of a dataset with 100 correlated features while retaining as much of the original variance as possible. The goal is to create a new, smaller set of uncorrelated features. Which technique is most appropriate for this task?

Answer: Principal Component Analysis (PCA)

Principal Component Analysis (PCA) is a feature extraction technique used for dimensionality reduction. It transforms a set of correlated variables into a smaller set of new, uncorrelated variables called principal components, which are ordered by the amount of original variance they capture. [5, 15, 32, 33]

When dealing with missing data in a numerical feature, a data scientist decides to replace the missing entries with the feature's median value. This approach is most appropriate under which of the following circumstances?

Answer: The goal is to remove all rows containing any missing values.

Median imputation is often preferred over mean imputation when the numerical feature's distribution is skewed. The median is a robust measure of central tendency that is less affected by outliers than the mean, providing a more representative value for replacement in such cases. [4, 9, 12]