Machine Learning Supervised Learning 4 — Questions and Answers
Question 1: Which loss function is typically used for training a binary classification model with logistic regression?
- Mean Squared Error
- Binary Cross-Entropy (Correct answer)
- Hinge Loss
- Huber Loss
Correct answer: Binary Cross-Entropy
Binary cross-entropy (log loss) measures the divergence between predicted probabilities and true binary labels, making it ideal for logistic regression.
Question 2: What is the main difference between bagging and boosting ensemble methods?
- Bagging uses different algorithms; boosting uses the same
- Bagging trains models sequentially; boosting trains them in parallel
- Bagging trains models in parallel on random subsets; boosting trains sequentially focusing on errors (Correct answer)
- Bagging is only for regression; boosting is only for classification
Correct answer: Bagging trains models in parallel on random subsets; boosting trains sequentially focusing on errors
Bagging (e.g., Random Forest) trains models in parallel on bootstrap samples to reduce variance, while boosting trains models sequentially to reduce bias by focusing on previous errors.
Question 3: In a confusion matrix for binary classification, what does a 'False Positive' represent?
- A negative instance correctly classified as negative
- A positive instance correctly classified as positive
- A negative instance incorrectly classified as positive (Correct answer)
- A positive instance incorrectly classified as negative
Correct answer: A negative instance incorrectly classified as positive
A false positive (Type I error) occurs when the model predicts positive but the true label is negative.
Question 4: Which regularization method can perform automatic feature selection by driving some coefficients exactly to zero?
- L2 (Ridge)
- Elastic Net
- L1 (Lasso) (Correct answer)
- Dropout
Correct answer: L1 (Lasso)
L1 (Lasso) regularization uses the absolute value of coefficients in its penalty, which tends to produce sparse solutions with some weights exactly zero.
Question 5: What is 'feature scaling' and why is it important for algorithms like SVMs and k-NN?
- Selecting the most important features; reduces computation
- Normalizing feature ranges so no single feature dominates distance calculations (Correct answer)
- Adding polynomial features to increase model complexity
- Removing outliers from the training set
Correct answer: Normalizing feature ranges so no single feature dominates distance calculations
Feature scaling (e.g., standardization or min-max normalization) ensures all features contribute equally to distance-based calculations, which is critical for SVMs and k-NN.
Question 6: What does the ROC curve plot in a binary classification problem?
- Precision vs. Recall at various thresholds
- True Positive Rate vs. False Positive Rate at various thresholds (Correct answer)
- Training loss vs. validation loss over epochs
- Model accuracy vs. number of features
Correct answer: True Positive Rate vs. False Positive Rate at various thresholds
The ROC curve plots the True Positive Rate (sensitivity) against the False Positive Rate (1-specificity) at various classification thresholds.
Question 7: Which of the following is an example of a regression problem in supervised learning?
- Predicting whether an email is spam or not spam
- Classifying handwritten digits 0-9
- Predicting the selling price of a house based on its features (Correct answer)
- Identifying whether a tumor is malignant or benign
Correct answer: Predicting the selling price of a house based on its features
Predicting house prices produces a continuous numeric output, making it a regression problem rather than a classification problem.
Machine Learning Supervised Learning 4
Which loss function is typically used for training a binary classification model with logistic regression?