Machine Learning Supervised Learning 3 — Questions and Answers
Question 1: In k-Nearest Neighbors (k-NN), what is the effect of choosing a very large value of k?
- Increases model variance
- Creates a more complex decision boundary
- Smooths the decision boundary, potentially underfitting (Correct answer)
- Speeds up prediction time
Correct answer: Smooths the decision boundary, potentially underfitting
A large k averages over many neighbors, creating a smoother but potentially underfit decision boundary that may miss local patterns.
Question 2: Which of the following best describes 'stratified k-fold cross-validation'?
- Each fold contains only one class
- Each fold preserves the same class distribution as the full dataset (Correct answer)
- The model is trained on all folds simultaneously
- Features are randomly shuffled within each fold
Correct answer: Each fold preserves the same class distribution as the full dataset
Stratified k-fold ensures each fold has roughly the same proportion of class labels as the original dataset, important for imbalanced data.
Question 3: What is the purpose of a validation set in a supervised learning workflow?
- To compute the final test accuracy
- To tune hyperparameters without contaminating the test set (Correct answer)
- To augment the training data
- To initialize model weights
Correct answer: To tune hyperparameters without contaminating the test set
A validation set is used during training to tune hyperparameters and make model selection decisions, keeping the test set pristine for final evaluation.
Question 4: In logistic regression, what function maps the linear output to a probability between 0 and 1?
- ReLU
- Softmax
- Sigmoid (Correct answer)
- Tanh
Correct answer: Sigmoid
The sigmoid function squashes any real-valued number to the range (0, 1), making it suitable for binary probability estimation.
Question 5: Which phenomenon occurs when a model learns noise and specific patterns of the training data that do not generalize to new data?
- Underfitting
- Regularization
- Overfitting (Correct answer)
- Normalization
Correct answer: Overfitting
Overfitting happens when a model is too complex and memorizes training data details, resulting in high training accuracy but poor test accuracy.
Question 6: What does the 'bias' term represent in the bias-variance tradeoff?
- Sensitivity to small fluctuations in training data
- Error due to overly simplistic model assumptions (Correct answer)
- The mean of the training labels
- Noise in the test dataset
Correct answer: Error due to overly simplistic model assumptions
Bias reflects error introduced by assuming a model form that is too simple to capture the true underlying relationship in the data.
Question 7: In gradient boosting, how are successive trees trained?
- Independently on random subsets of features
- On the residual errors of the previous ensemble (Correct answer)
- Using the same training data without modification
- On randomly selected subsets of training samples
Correct answer: On the residual errors of the previous ensemble
Each new tree in gradient boosting is trained to predict the residual errors (pseudo-residuals) left by the current ensemble, iteratively improving predictions.
Machine Learning Supervised Learning 3
In k-Nearest Neighbors (k-NN), what is the effect of choosing a very large value of k?