DSE Supervised Learning: Classification 4 — Questions and Answers
Question 1: Which of the following is the log-loss (cross-entropy loss) formula's primary purpose in classification?
- Penalize large weights to prevent overfitting
- Measure the difference between predicted probabilities and true class labels (Correct answer)
- Compute the Euclidean distance between class centroids
- Maximize the margin between support vectors
Correct answer: Measure the difference between predicted probabilities and true class labels
Log-loss quantifies how well a classifier's predicted probabilities match the actual binary or multi-class labels.
Question 2: In gradient boosting for classification, what does each successive tree learn?
- A separate random subset of features
- The residual errors (pseudo-residuals) of the previous ensemble (Correct answer)
- An independent model on a bootstrapped dataset
- A linear combination of all previous trees
Correct answer: The residual errors (pseudo-residuals) of the previous ensemble
Each new tree in gradient boosting is fit to the negative gradient (pseudo-residuals) of the loss function from the current ensemble.
Question 3: What is the primary reason to use stratified k-fold cross-validation for classification?
- It reduces training time by using fewer folds
- It ensures each fold maintains the same class proportion as the full dataset (Correct answer)
- It prevents data leakage between training and test sets
- It automatically handles missing values in the dataset
Correct answer: It ensures each fold maintains the same class proportion as the full dataset
Stratified k-fold ensures each fold reflects the overall class distribution, giving a more reliable estimate of model performance, especially with imbalanced classes.
Question 4: Which regularization approach in logistic regression corresponds to an L1 penalty?
- Ridge regression, which shrinks all coefficients toward zero
- Lasso, which can shrink some coefficients to exactly zero (Correct answer)
- Elastic net, which combines L1 and L2 penalties
- Dropout, which randomly zeroes out coefficients during training
Correct answer: Lasso, which can shrink some coefficients to exactly zero
L1 regularization (Lasso) in logistic regression adds the absolute value of coefficients to the loss, promoting sparsity by zeroing out some weights.
Question 5: What does the F-beta score allow you to adjust compared to the standard F1 score?
- The number of classes in the classification problem
- The relative weight given to precision versus recall (Correct answer)
- The learning rate during model training
- The threshold for converting probabilities to class labels
Correct answer: The relative weight given to precision versus recall
The F-beta score uses a beta parameter to weight recall beta times more than precision, allowing domain-specific tuning of the precision-recall trade-off.
Question 6: Which of the following describes a 'discriminative' classification model?
- It models the joint probability P(X, Y) of features and labels
- It models the conditional probability P(Y|X) directly (Correct answer)
- It generates new data samples from learned class distributions
- It requires a prior probability for each class
Correct answer: It models the conditional probability P(Y|X) directly
Discriminative models like logistic regression and SVMs learn the decision boundary by modeling P(Y|X) directly, without modeling how features are generated.
Question 7: When would you prefer a Decision Tree over Logistic Regression for classification?
- When you need probability calibration and linear decision boundaries
- When the data has complex nonlinear feature interactions and interpretability is needed (Correct answer)
- When the dataset has very few features and a large number of samples
- When regularization of feature weights is the primary concern
Correct answer: When the data has complex nonlinear feature interactions and interpretability is needed
Decision trees naturally capture nonlinear interactions without feature engineering, and their structure is human-interpretable via tree visualization.
DSE Supervised Learning: Classification 4
Which of the following is the log-loss (cross-entropy loss) formula's primary purpose in classification?