DSE Unsupervised Learning: Clustering 5

Question 1

What is the primary purpose of the 'gap statistic' method in clustering?

Accepted Answer

To compare within-cluster dispersion to a null reference distribution to find optimal k

Answer

The gap statistic compares WCSS of the actual clustering to that expected under a null uniform distribution, with the optimal k being where the gap is maximized.

Question 2

In the context of clustering high-dimensional data, what is the 'curse of dimensionality' problem?

Accepted Answer

Distance metrics become less meaningful as all pairwise distances converge

Answer

As dimensionality increases, the ratio of maximum to minimum pairwise distances approaches 1, making distance-based clustering unreliable since all points appear equally far apart.

Question 3

Which subspace clustering technique projects data into lower dimensions before clustering to mitigate high-dimensionality issues?

Accepted Answer

Applying PCA before K-Means

Answer

Applying PCA before K-Means reduces noise dimensions and retains variance-rich components, making distance computations more meaningful in the reduced space.

Question 4

When using Fuzzy C-Means clustering, what does the fuzziness parameter 'm' control?

Accepted Answer

The degree of overlap between cluster memberships

Answer

Higher values of m (m > 1) increase fuzziness so memberships are spread more evenly across clusters; as m → 1, Fuzzy C-Means approaches hard K-Means partitioning.

Question 5

A retail analyst clusters purchase transactions and finds that scaling features dramatically changes cluster assignments. Why does this happen with K-Means?

Accepted Answer

K-Means is sensitive to feature scale because it uses Euclidean distance

Answer

Euclidean distance gives more weight to features with larger numeric ranges, so an unscaled feature like 'transaction amount in dollars' dominates 'number of items bought'.

DSE - Data Science Practice Test

DSE - Data Science Practice Test

DSE Unsupervised Learning: Clustering 5