Machine Learning Clustering 3 — Questions and Answers
Question 1: When applying K-Means to high-dimensional data, which phenomenon often causes the algorithm to perform poorly?
- Overfitting to training data
- The curse of dimensionality making distances less meaningful (Correct answer)
- Underfitting due to too few clusters
- Gradient vanishing in the distance computation
Correct answer: The curse of dimensionality making distances less meaningful
In high dimensions, distances between points converge, making it difficult for K-Means to distinguish between close and far neighbors.
Question 2: Which initialization strategy for K-Means selects initial centroids that are spread far apart to improve convergence?
- Random initialization
- K-Means++ (Correct answer)
- PCA-based initialization
- Forgy method
Correct answer: K-Means++
K-Means++ probabilistically selects each subsequent centroid farther from already-chosen ones, reducing the chance of poor local optima.
Question 3: In DBSCAN, what term describes a point that does not belong to any cluster?
- Border point
- Core point
- Noise point (Correct answer)
- Orphan point
Correct answer: Noise point
Noise points (also called outliers) in DBSCAN are points that are neither core points nor reachable from any core point.
Question 4: What distinguishes soft clustering from hard clustering?
- Soft clustering requires labeled data
- Soft clustering assigns each point a probability of belonging to each cluster (Correct answer)
- Soft clustering uses distance metrics instead of density
- Soft clustering always produces more clusters
Correct answer: Soft clustering assigns each point a probability of belonging to each cluster
In soft clustering (e.g., GMM), each point has a fractional membership across all clusters rather than exclusive assignment to one.
Question 5: Which of the following is a valid reason to prefer hierarchical clustering over K-Means?
- Hierarchical clustering is always faster
- Hierarchical clustering does not require specifying K beforehand (Correct answer)
- Hierarchical clustering handles noise better than DBSCAN
- Hierarchical clustering works better on very large datasets
Correct answer: Hierarchical clustering does not require specifying K beforehand
Hierarchical clustering builds a full dendrogram, allowing the number of clusters to be chosen by cutting the tree at any level after computation.
Question 6: What is the time complexity of the basic K-Means algorithm per iteration?
- O(n log n)
- O(n * K * d) (Correct answer)
- O(n²)
- O(K²)
Correct answer: O(n * K * d)
Each iteration requires computing distances from all n points to all K centroids in d dimensions, giving O(n*K*d) per iteration.
Question 7: Which scenario best illustrates a case where K-Means would fail but DBSCAN would succeed?
- Clusters that are well-separated spheres of equal size
- Two interleaved crescent-shaped clusters (Correct answer)
- Data with exactly 3 distinct Gaussian blobs
- High-dimensional data with 100 features
Correct answer: Two interleaved crescent-shaped clusters
K-Means assumes convex clusters and would split crescent shapes incorrectly, while DBSCAN detects arbitrary shapes through density.
Machine Learning Clustering 3
When applying K-Means to high-dimensional data, which phenomenon often causes the algorithm to perform poorly?