Supervised Learning Flashcards

In k-Nearest Neighbors (k-NN), what is the effect of choosing a very large value of k?

Answer: Smooths the decision boundary, potentially underfitting

A large k averages over many neighbors, creating a smoother but potentially underfit decision boundary that may miss local patterns.

Which of the following best describes 'stratified k-fold cross-validation'?

Answer: Each fold preserves the same class distribution as the full dataset

Stratified k-fold ensures each fold has roughly the same proportion of class labels as the original dataset, important for imbalanced data.

What is the purpose of a validation set in a supervised learning workflow?

Answer: To tune hyperparameters without contaminating the test set

A validation set is used during training to tune hyperparameters and make model selection decisions, keeping the test set pristine for final evaluation.

In logistic regression, what function maps the linear output to a probability between 0 and 1?

Answer: Sigmoid

The sigmoid function squashes any real-valued number to the range (0, 1), making it suitable for binary probability estimation.

Which phenomenon occurs when a model learns noise and specific patterns of the training data that do not generalize to new data?

Answer: Overfitting

Overfitting happens when a model is too complex and memorizes training data details, resulting in high training accuracy but poor test accuracy.

What does the 'bias' term represent in the bias-variance tradeoff?

Answer: Error due to overly simplistic model assumptions

Bias reflects error introduced by assuming a model form that is too simple to capture the true underlying relationship in the data.

In gradient boosting, how are successive trees trained?

Answer: On the residual errors of the previous ensemble

Each new tree in gradient boosting is trained to predict the residual errors (pseudo-residuals) left by the current ensemble, iteratively improving predictions.