Machine Learning Classification 2 — Questions and Answers
Question 1: Which metric is most appropriate when false negatives are far more costly than false positives, such as in cancer screening?
- Precision
- Recall (Correct answer)
- Specificity
- Accuracy
Correct answer: Recall
Recall (sensitivity) measures how many actual positives are correctly identified, making it critical when missing a positive (false negative) has severe consequences.
Question 2: In a Support Vector Machine, what is the 'margin'?
- The misclassification rate on training data
- The distance between the decision boundary and the nearest data points from each class (Correct answer)
- The number of support vectors divided by total samples
- The regularization penalty applied to the weight vector
Correct answer: The distance between the decision boundary and the nearest data points from each class
The margin is the distance between the hyperplane and the closest data points (support vectors) from each class; SVM maximizes this margin.
Question 3: What does the 'kernel trick' allow an SVM to do?
- Reduce training time by pruning irrelevant features
- Operate in a higher-dimensional feature space without explicitly computing the transformation (Correct answer)
- Automatically select the optimal regularization parameter C
- Ensemble multiple SVMs for better accuracy
Correct answer: Operate in a higher-dimensional feature space without explicitly computing the transformation
The kernel trick computes dot products in a transformed space implicitly, enabling SVMs to learn non-linear decision boundaries without expensive feature mapping.
Question 4: Which of the following is an example of a generative classification model?
- Logistic Regression
- Naive Bayes (Correct answer)
- Support Vector Machine
- k-Nearest Neighbors
Correct answer: Naive Bayes
Naive Bayes is generative because it models the joint probability P(X, Y) and uses Bayes' theorem to compute posterior class probabilities.
Question 5: In k-Nearest Neighbors classification, increasing k generally causes the decision boundary to become:
- More jagged and complex
- Smoother and less sensitive to individual points (Correct answer)
- Identical to a linear boundary
- Dependent only on support vectors
Correct answer: Smoother and less sensitive to individual points
Larger k averages over more neighbors, smoothing the decision boundary and reducing variance at the cost of potentially increased bias.
Question 6: What is the primary purpose of the softmax function in a multi-class neural network classifier?
- To introduce non-linearity into hidden layers
- To convert raw output scores into a probability distribution over classes (Correct answer)
- To normalize input features to zero mean
- To apply L2 regularization to the output layer weights
Correct answer: To convert raw output scores into a probability distribution over classes
Softmax exponentiates each logit and normalizes by the sum, producing a valid probability distribution where all class probabilities sum to one.
Question 7: A classifier achieves 98% accuracy on a dataset where 98% of samples belong to class A. What problem does this illustrate?
- Overfitting to the minority class
- The accuracy paradox with imbalanced data (Correct answer)
- Underfitting due to insufficient model complexity
- Data leakage from test set to training set
Correct answer: The accuracy paradox with imbalanced data
A trivial classifier predicting only class A achieves 98% accuracy, demonstrating that accuracy is misleading when class distributions are severely imbalanced.
Machine Learning Classification 2
Which metric is most appropriate when false negatives are far more costly than false positives, such as in cancer screening?