Machine Learning Classification 5 — Questions and Answers
Question 1: What is 'label smoothing' in classification training?
- Replacing hard 0/1 targets with soft values (e.g., 0.1/0.9) to prevent overconfident predictions (Correct answer)
- Removing noisy labels from the training set before fitting
- Averaging predictions from multiple classifiers to smooth decision boundaries
- Applying Gaussian noise to input features during training
Correct answer: Replacing hard 0/1 targets with soft values (e.g., 0.1/0.9) to prevent overconfident predictions
Label smoothing replaces hard class labels with softer distributions, regularizing the model and preventing it from becoming overconfident on training examples.
Question 2: Which phenomenon occurs when a classifier performs well on training data but fails to generalize to unseen examples?
- Underfitting
- Overfitting (Correct answer)
- Covariate Shift
- Concept Drift
Correct answer: Overfitting
Overfitting means the model has memorized training patterns including noise, resulting in high training accuracy but poor generalization to new data.
Question 3: In Naive Bayes classification, what 'naive' assumption is made?
- All classes have equal prior probability
- Features are conditionally independent given the class label (Correct answer)
- The decision boundary is always linear
- The data follows a Gaussian distribution
Correct answer: Features are conditionally independent given the class label
Naive Bayes assumes that all features are conditionally independent given the class, allowing the joint likelihood to be factored into a product of individual likelihoods.
Question 4: What is the primary advantage of using an ensemble method like Gradient Boosting over a single decision tree?
- Faster training time and lower memory usage
- Higher predictive accuracy by combining many weak learners sequentially (Correct answer)
- Better interpretability through explicit rule extraction
- Elimination of the need for hyperparameter tuning
Correct answer: Higher predictive accuracy by combining many weak learners sequentially
Gradient Boosting combines many shallow trees sequentially, each correcting previous errors, typically achieving much higher accuracy than any single tree.
Question 5: Which of the following best describes 'calibration' of a classifier?
- Adjusting hyperparameters to maximize accuracy on a validation set
- Ensuring predicted probabilities align with actual observed frequencies (Correct answer)
- Scaling input features to have zero mean and unit variance
- Reducing model complexity to prevent overfitting
Correct answer: Ensuring predicted probabilities align with actual observed frequencies
A well-calibrated classifier produces probability estimates that match empirical outcomes; for example, 70% predicted probability should correspond to 70% actual positive rate.
Question 6: What does 'stratified sampling' ensure when creating train/test splits for a classification task?
- Each split has the same total number of samples
- Each split preserves the original class proportion from the full dataset (Correct answer)
- Features are normalized identically in both splits
- No data point appears in both training and test sets
Correct answer: Each split preserves the original class proportion from the full dataset
Stratified sampling maintains the same class distribution across splits, preventing a split from accidentally containing mostly one class and yielding misleading evaluation metrics.
Question 7: Which technique reduces multicollinearity's effect on logistic regression by adding the sum of squared coefficients as a penalty?
- L1 Regularization (Lasso)
- L2 Regularization (Ridge) (Correct answer)
- Dropout
- Early Stopping
Correct answer: L2 Regularization (Ridge)
L2 regularization adds a penalty proportional to squared weights, shrinking all coefficients and stabilizing estimates when input features are correlated.
Machine Learning Classification 5
What is 'label smoothing' in classification training?