Supervised Learning Flashcards

Which technique helps prevent overfitting by adding a penalty term proportional to the square of the model's coefficients?

Answer: L2 regularization

L2 regularization (Ridge) adds a penalty equal to the sum of squared coefficients, shrinking them toward zero without eliminating them.

In a decision tree, what does 'pruning' accomplish?

Answer: Removes branches that provide little predictive power to reduce overfitting

Pruning removes overly specific branches from a decision tree to improve generalization on unseen data.

What is the primary advantage of using an ensemble method like Random Forest over a single decision tree?

Answer: Lower variance through averaging multiple trees

Random Forest reduces variance by averaging predictions from many decorrelated trees built on random subsets of features and data.

In the context of SVMs, what does the 'kernel trick' allow?

Answer: Mapping data into a higher-dimensional space without explicit computation

The kernel trick implicitly maps data to a higher-dimensional feature space using a kernel function, enabling linear separation of nonlinearly separable data.

What does the learning rate hyperparameter control in gradient descent?

Answer: The size of steps taken toward the minimum of the loss function

The learning rate determines how large each parameter update step is; too large causes divergence, too small causes slow convergence.

Which evaluation metric is most appropriate when the cost of false negatives greatly exceeds the cost of false positives?

Answer: Recall (Sensitivity)

Recall measures the proportion of actual positives correctly identified, making it critical when missing a positive (false negative) is costly, such as in disease detection.

What is a key characteristic of Naive Bayes classifiers that makes them 'naive'?

Answer: They assume all features are conditionally independent given the class

Naive Bayes assumes conditional independence between features given the class label, which simplifies computation but rarely holds in practice.