Machine Learning & Algorithms Flashcards

A decision tree grows until every leaf is pure and achieves 100% training accuracy but performs poorly on new data. What is the most likely problem?

Answer: Overfitting due to high variance

A fully grown tree memorizes training noise, which gives low bias but high variance and poor generalization.

Which splitting criterion does the CART algorithm use for classification trees?

Answer: Gini impurity

CART uses Gini impurity for classification and variance reduction (MSE) for regression.

In k-means clustering, what happens to the within-cluster sum of squares (WCSS) as k increases?

Answer: It never increases and decreases toward zero when k equals the number of points

More centroids can only reduce or maintain WCSS, reaching zero when each point is its own cluster, which is why the elbow method looks for diminishing returns.

Which statement best describes bagging as used in random forests?

Answer: Models are trained independently on bootstrap samples and their predictions are aggregated

Bagging trains models in parallel on bootstrap samples and averages or votes, mainly reducing variance.

Which metric is most appropriate when only 1% of transactions are fraudulent and you want to evaluate the fraud class specifically?

Answer: Precision and recall (or the PR curve)

With heavy imbalance, accuracy is misleading, so precision, recall and PR-AUC reflect minority-class performance.

What is the main purpose of the kernel trick in support vector machines?

Answer: To compute inner products in a high-dimensional space without explicitly transforming the data

Kernels let SVMs find non-linear boundaries by implicitly mapping data to a higher-dimensional feature space.

Which algorithm requires specifying a neighborhood radius (epsilon) and a minimum number of points to define clusters?

Answer: DBSCAN

DBSCAN uses eps and minPts to find density-based clusters and labels sparse points as noise.