MS-DS Master of Data science Unsupervised Learning Techniques 5 — Questions and Answers
Question 1: What is the 'manifold hypothesis' and why is it important for unsupervised learning?
- It states that real-world data is always linearly separable in sufficiently high dimensions
- It posits that high-dimensional data lies on a lower-dimensional manifold, motivating nonlinear dimensionality reduction (Correct answer)
- It claims that all clusters in real data have a Gaussian distribution
- It asserts that the number of meaningful features equals the square root of the sample size
Correct answer: It posits that high-dimensional data lies on a lower-dimensional manifold, motivating nonlinear dimensionality reduction
The manifold hypothesis states that real-world high-dimensional data concentrates near a low-dimensional curved surface, justifying methods like t-SNE, UMAP, and Isomap for dimensionality reduction.
Question 2: Which hyperparameter in DBSCAN determines the minimum number of points required to form a dense region (core point)?
- epsilon (ε)
- minPts (Correct answer)
- linkage_threshold
- bandwidth
Correct answer: minPts
A point is a core point if at least minPts points (including itself) fall within its epsilon-neighborhood; minPts controls the density requirement for cluster formation.
Question 3: When comparing GMMs to k-means, which advantage do GMMs offer?
- GMMs always converge to the global optimum; k-means does not
- GMMs can model clusters of varying shapes, sizes, and densities through covariance matrices (Correct answer)
- GMMs have lower computational complexity per iteration than k-means
- GMMs do not require specifying the number of clusters in advance
Correct answer: GMMs can model clusters of varying shapes, sizes, and densities through covariance matrices
GMMs learn full covariance matrices per component, allowing elliptical clusters of different orientations and sizes, while k-means assumes spherical, equal-variance clusters.
Question 4: A data scientist applies PCA to a dataset and finds the first two principal components explain 62% of variance. What is the most appropriate interpretation?
- The remaining 38% of variance is noise and should be discarded
- The 2D projection retains 62% of the information; the rest is lost in the low-dimensional representation (Correct answer)
- PCA failed because 95% explained variance is required for valid results
- The dataset has exactly two meaningful features
Correct answer: The 2D projection retains 62% of the information; the rest is lost in the low-dimensional representation
Explained variance ratio tells you how much information is retained in the projection; 62% means substantial structure is captured, but 38% is sacrificed in this 2D reduction.
Question 5: What is the Calinski-Harabasz index, and what does a higher value indicate?
- It measures cluster overlap; higher values indicate more overlapping clusters
- It is the ratio of between-cluster dispersion to within-cluster dispersion; higher values indicate better-defined clusters (Correct answer)
- It measures the average silhouette width; higher values indicate more noise points
- It computes the log-likelihood of a GMM; higher values indicate more components are needed
Correct answer: It is the ratio of between-cluster dispersion to within-cluster dispersion; higher values indicate better-defined clusters
The Calinski-Harabasz index (also called Variance Ratio Criterion) rewards tight, well-separated clusters: higher values mean clusters are dense internally and far from each other.
Question 6: In the context of topic modeling, what does Latent Dirichlet Allocation (LDA) model as latent variables?
- Word frequencies and document lengths
- Per-document topic distributions and per-topic word distributions (Correct answer)
- The co-occurrence matrix of words across documents
- The TF-IDF weights of terms in each document
Correct answer: Per-document topic distributions and per-topic word distributions
LDA treats each document as a mixture of topics (θ) and each topic as a distribution over words (φ), inferring both as latent variables from observed word counts.
Question 7: Which phenomenon explains why distance-based clustering methods can struggle with very high-dimensional data?
- High-dimensional data always requires more clusters than low-dimensional data
- The curse of dimensionality causes distances between points to become increasingly uniform, reducing discriminative power (Correct answer)
- High-dimensional feature spaces prevent the computation of covariance matrices
- Clustering algorithms cannot scale beyond 100 dimensions without GPU acceleration
Correct answer: The curse of dimensionality causes distances between points to become increasingly uniform, reducing discriminative power
In high dimensions, the ratio of maximum to minimum pairwise distances converges to 1, making all points seem equidistant and undermining distance-based similarity measures used in clustering.
MS-DS Master of Data science Unsupervised Learning Techniques 5
What is the 'manifold hypothesis' and why is it important for unsupervised learning?