AML Supervised & Unsupervised Learning Algorithms 2 — Questions and Answers
Question 1: In a gradient boosting model, what does the 'learning rate' (shrinkage) parameter control?
- The number of trees added per iteration
- The fraction by which each tree's contribution is scaled down (Correct answer)
- The maximum depth of individual decision trees
- The proportion of training samples used per tree
Correct answer: The fraction by which each tree's contribution is scaled down
The learning rate (shrinkage) multiplies each new tree's contribution, preventing overfitting by making the ensemble learn more slowly and conservatively.
Question 2: Which unsupervised algorithm is best suited for detecting arbitrarily shaped clusters in noisy data?
- K-Means
- Gaussian Mixture Models
- DBSCAN (Correct answer)
- Hierarchical Agglomerative Clustering
Correct answer: DBSCAN
DBSCAN (Density-Based Spatial Clustering of Applications with Noise) discovers clusters of arbitrary shape and explicitly labels outliers as noise points.
Question 3: What is the primary purpose of using kernel functions in Support Vector Machines?
- To reduce training time by approximating the margin
- To implicitly map data into a higher-dimensional feature space where it becomes linearly separable (Correct answer)
- To normalize feature scales before optimization
- To select the most informative support vectors
Correct answer: To implicitly map data into a higher-dimensional feature space where it becomes linearly separable
Kernel functions compute dot products in a higher-dimensional space without explicitly transforming the data, enabling SVMs to find non-linear decision boundaries.
Question 4: In Expectation-Maximization (EM) for Gaussian Mixture Models, what happens during the E-step?
- Model parameters (means, covariances, weights) are updated
- The number of Gaussian components is selected via BIC
- Soft cluster membership probabilities are computed for each data point (Correct answer)
- Outlier points are removed from the training set
Correct answer: Soft cluster membership probabilities are computed for each data point
The E-step computes the posterior probability (responsibility) that each Gaussian component generated each data point, given the current parameter estimates.
Question 5: Which statement correctly describes the bias-variance tradeoff in the context of k-Nearest Neighbors (kNN)?
- Larger k increases variance and decreases bias
- Smaller k decreases variance and increases bias
- Larger k increases bias and decreases variance (Correct answer)
- The choice of k does not affect bias or variance
Correct answer: Larger k increases bias and decreases variance
A larger k smooths the decision boundary by averaging more neighbors, increasing bias (underfitting) but reducing variance (sensitivity to noise).
Question 6: What does the 'elbow method' evaluate when selecting the optimal number of clusters for K-Means?
- Silhouette coefficient as a function of cluster count
- Within-cluster sum of squares (inertia) as a function of k (Correct answer)
- Between-cluster variance divided by total variance
- The gap statistic compared to a reference null distribution
Correct answer: Within-cluster sum of squares (inertia) as a function of k
The elbow method plots within-cluster sum of squares (inertia) against k and selects the point where the rate of decrease sharply diminishes, forming an 'elbow.'
Question 7: In a Random Forest, how does the 'max_features' hyperparameter reduce correlation between trees?
- It limits tree depth, forcing each tree to use fewer splits
- It randomly subsamples training rows for each tree
- It restricts each split to consider only a random subset of features (Correct answer)
- It penalizes features already used in prior trees
Correct answer: It restricts each split to consider only a random subset of features
By considering only a random subset of features at each split, trees are forced to use different predictors, reducing correlation and improving ensemble diversity.
AML Supervised & Unsupervised Learning Algorithms 2
In a gradient boosting model, what does the 'learning rate' (shrinkage) parameter control?