Data Science Feature Engineering and Selection 5

Question 1

Tree-based models provide feature importance primarily based on:

Accepted Answer

How much each feature reduces impurity across splits

Answer

Impurity-based importance measures each feature's total reduction in impurity over the tree splits.

Question 2

Permutation importance measures a feature's value by:

Accepted Answer

Shuffling its values and observing the drop in model performance

Answer

Permutation importance randomly shuffles a feature and records how much performance degrades.

Question 3

Which dimensionality reduction technique is unsupervised and linear?

Accepted Answer

Principal Component Analysis (PCA)

Answer

PCA is an unsupervised, linear method that projects data onto directions of maximum variance.

Question 4

A key difference between PCA and feature selection is that PCA:

Accepted Answer

Creates new combined features rather than keeping original ones

Answer

PCA produces transformed components, whereas selection retains a subset of original features.

Question 5

For text data, TF-IDF weighting down-weights words that:

Accepted Answer

Appear frequently across many documents

Answer

TF-IDF reduces the weight of common words appearing in many documents, emphasizing distinctive terms.

Data Science Practice Test