Machine Learning Classification 4 — Questions and Answers
Question 1: What is the purpose of cross-validation in classifier evaluation?
- To increase training data size through augmentation
- To get a more reliable estimate of model performance by training and testing on multiple data splits (Correct answer)
- To automatically tune hyperparameters using gradient descent
- To reduce the number of features needed for classification
Correct answer: To get a more reliable estimate of model performance by training and testing on multiple data splits
Cross-validation repeatedly splits data into train/test folds, averaging performance across folds to produce a less biased and more stable generalization estimate.
Question 2: Which boosting algorithm uses gradient descent in function space to minimize classification loss?
- AdaBoost
- Gradient Boosting (GBM) (Correct answer)
- Bagging
- Stacking
Correct answer: Gradient Boosting (GBM)
Gradient Boosting fits each new tree to the negative gradient of the loss function, effectively performing gradient descent to sequentially reduce errors.
Question 3: In a confusion matrix for binary classification, what does a 'False Positive' represent?
- A negative sample correctly classified as negative
- A positive sample incorrectly classified as negative
- A negative sample incorrectly classified as positive (Correct answer)
- A positive sample correctly classified as positive
Correct answer: A negative sample incorrectly classified as positive
A false positive occurs when the model predicts the positive class but the true label is negative, also known as a Type I error.
Question 4: Which regularization technique in logistic regression adds the absolute value of coefficients as a penalty, promoting sparsity?
- L2 (Ridge)
- L1 (Lasso) (Correct answer)
- Elastic Net
- Dropout
Correct answer: L1 (Lasso)
L1 regularization penalizes the sum of absolute values of weights, driving irrelevant feature coefficients exactly to zero and performing implicit feature selection.
Question 5: What is the F1 score?
- The arithmetic mean of precision and recall
- The harmonic mean of precision and recall (Correct answer)
- The geometric mean of accuracy and AUC
- The ratio of true positives to total predictions
Correct answer: The harmonic mean of precision and recall
The F1 score is the harmonic mean of precision and recall, giving a single metric that balances both and penalizes extreme imbalances between them.
Question 6: Which statement best describes 'discriminative' classification models?
- They model the joint distribution P(X, Y) and use Bayes' rule
- They directly model the conditional probability P(Y|X) or the decision boundary (Correct answer)
- They always require labeled and unlabeled data simultaneously
- They generate new samples from the learned class distributions
Correct answer: They directly model the conditional probability P(Y|X) or the decision boundary
Discriminative models like logistic regression and SVMs learn P(Y|X) or the boundary directly, without modeling how input features are generated.
Question 7: When should you prefer macro-averaged F1 over micro-averaged F1 in multi-class evaluation?
- When class sizes are roughly equal and all classes are equally important
- When minority classes matter as much as majority classes and class imbalance exists (Correct answer)
- When you want the metric to be dominated by the largest class
- When training time needs to be minimized
Correct answer: When minority classes matter as much as majority classes and class imbalance exists
Macro-average computes F1 per class and averages them equally, giving equal weight to rare classes, unlike micro-average which is dominated by frequent classes.
Machine Learning Classification 4
What is the purpose of cross-validation in classifier evaluation?