Supervised Learning Models Flashcards

What is the bias-variance tradeoff?

Answer: A balance between underfitting and overfitting errors

High bias underfits while high variance overfits; the tradeoff seeks minimal total error.

In k-nearest neighbors, what happens as k increases?

Answer: The decision boundary becomes smoother

Larger k averages over more neighbors, smoothing the boundary and reducing variance.

Which ensemble method builds trees sequentially, each correcting the previous one's errors?

Answer: Gradient boosting

Gradient boosting fits each new tree to the residual errors of the prior ensemble.

What does a high recall but low precision indicate?

Answer: Many false positives but few false negatives

High recall catches most positives, but low precision means many predicted positives are wrong.

Why is feature scaling important for SVM and KNN?

Answer: They rely on distance calculations sensitive to feature magnitude

Distance-based models let large-magnitude features dominate unless features are scaled.

What is the role of the learning rate in gradient descent?

Answer: It controls the step size of weight updates

The learning rate scales how far weights move along the gradient each iteration.

Which model assumes feature independence given the class label?

Answer: Naive Bayes

Naive Bayes assumes conditional independence of features, simplifying the joint probability.