DSE Supervised Learning: Classification 3 — Questions and Answers
Question 1: In a decision tree, what criterion does the Gini impurity measure?
- The depth of the tree required for a split
- The probability of misclassifying a randomly chosen element (Correct answer)
- The information gained by splitting on a feature
- The variance of the target variable
Correct answer: The probability of misclassifying a randomly chosen element
Gini impurity measures the probability that a randomly selected sample would be incorrectly classified if labeled according to the class distribution in the node.
Question 2: What is the effect of setting a very high value for the regularization parameter C in an SVM?
- Wider margin with more misclassifications allowed
- Narrower margin with fewer misclassifications allowed (Correct answer)
- The kernel function is ignored
- The model becomes equivalent to Naive Bayes
Correct answer: Narrower margin with fewer misclassifications allowed
A high C value penalizes misclassification heavily, leading to a narrower margin and a model that tries hard to correctly classify all training points (risk of overfitting).
Question 3: Which of the following is an assumption of Gaussian Naive Bayes?
- Features are correlated within each class
- Features follow a Gaussian distribution within each class (Correct answer)
- The decision boundary is always linear
- All classes have equal prior probability
Correct answer: Features follow a Gaussian distribution within each class
Gaussian Naive Bayes assumes continuous features are normally distributed within each class.
Question 4: When using k-Nearest Neighbors for classification, what is the effect of choosing a very large k?
- The model becomes more sensitive to noise
- The decision boundary becomes smoother and simpler (Correct answer)
- Computation time decreases
- The model can only handle binary classification
Correct answer: The decision boundary becomes smoother and simpler
Large k values smooth out the decision boundary by averaging over more neighbors, reducing variance but potentially increasing bias.
Question 5: The ROC curve plots which two quantities against each other?
- Precision vs. Recall
- True Positive Rate vs. False Positive Rate (Correct answer)
- Accuracy vs. F1 Score
- Sensitivity vs. Specificity on a log scale
Correct answer: True Positive Rate vs. False Positive Rate
The ROC curve plots the True Positive Rate (sensitivity) on the y-axis against the False Positive Rate (1 - specificity) on the x-axis.
Question 6: Which of the following best describes a multiclass classification strategy known as 'One-vs-Rest' (OvR)?
- One classifier is trained for all classes simultaneously using a softmax output
- One binary classifier is trained per class, treating that class as positive and all others as negative (Correct answer)
- Classes are organized in a hierarchy and binary classifiers are trained at each level
- The model randomly selects two classes per iteration and trains a binary classifier
Correct answer: One binary classifier is trained per class, treating that class as positive and all others as negative
In OvR, one binary classifier is trained per class to distinguish it from all other classes combined.
Question 7: A classifier achieves 99% accuracy on a dataset where 99% of samples belong to class 0. What does this most likely indicate?
- The model is well-calibrated
- The model may be simply predicting class 0 for every sample (Correct answer)
- Accuracy is a reliable metric for this dataset
- The model has low bias and low variance
Correct answer: The model may be simply predicting class 0 for every sample
On a highly imbalanced dataset, a trivial classifier that always predicts the majority class can achieve high accuracy, making accuracy misleading.
DSE Supervised Learning: Classification 3
In a decision tree, what criterion does the Gini impurity measure?