DSE Feature Engineering and Selection 5

Question 1

What does 'feature importance' from a gradient boosting model (e.g., XGBoost) typically measure?

Accepted Answer

The average gain in model performance when a feature is used in a split across all trees

Answer

XGBoost's gain-based feature importance averages the improvement in the loss function brought by each feature across all splits where it is used.

Question 2

Which of the following correctly describes 'entity embeddings' as a feature engineering technique?

Accepted Answer

Learning dense low-dimensional vector representations for categorical features via a neural network

Answer

Entity embeddings train a neural network to map each category to a dense vector, capturing semantic relationships that one-hot encoding cannot.

Question 3

In geospatial feature engineering, what is the purpose of computing Haversine distance as a derived feature?

Accepted Answer

To compute the great-circle distance between two latitude/longitude points on a sphere

Answer

The Haversine formula calculates the shortest distance over the Earth's surface between two coordinate pairs, creating a meaningful numeric proximity feature.

Question 4

What is 'Boruta' and how does it differ from standard feature importance thresholding?

Accepted Answer

A feature selection algorithm that compares each feature's importance against shadow (permuted) features to determine statistical significance

Answer

Boruta creates randomized copies of features (shadows), trains a Random Forest, and accepts only features that consistently outperform the best shadow feature.

Question 5

When engineering date/time features, which decomposition is most useful for capturing weekly seasonality in a demand forecasting model?

Accepted Answer

Day of week extraction

Answer

Day of week captures the recurring weekly pattern in demand (e.g., weekday vs. weekend effects) that is typically the strongest seasonal cycle in retail data.

DSE - Data Science Practice Test

DSE - Data Science Practice Test

DSE Feature Engineering and Selection 5