Machine Learning Supervised Learning 2 — Questions and Answers
Question 1: Which technique helps prevent overfitting by adding a penalty term proportional to the square of the model's coefficients?
- L1 regularization
- L2 regularization (Correct answer)
- Dropout
- Early stopping
Correct answer: L2 regularization
L2 regularization (Ridge) adds a penalty equal to the sum of squared coefficients, shrinking them toward zero without eliminating them.
Question 2: In a decision tree, what does 'pruning' accomplish?
- Increases tree depth to improve accuracy
- Removes branches that provide little predictive power to reduce overfitting (Correct answer)
- Adds more features to the training set
- Converts a classification tree into a regression tree
Correct answer: Removes branches that provide little predictive power to reduce overfitting
Pruning removes overly specific branches from a decision tree to improve generalization on unseen data.
Question 3: What is the primary advantage of using an ensemble method like Random Forest over a single decision tree?
- Faster training time
- Reduced memory usage
- Lower variance through averaging multiple trees (Correct answer)
- Simpler model interpretation
Correct answer: Lower variance through averaging multiple trees
Random Forest reduces variance by averaging predictions from many decorrelated trees built on random subsets of features and data.
Question 4: In the context of SVMs, what does the 'kernel trick' allow?
- Training on smaller datasets
- Mapping data into a higher-dimensional space without explicit computation (Correct answer)
- Reducing the number of support vectors
- Speeding up gradient descent
Correct answer: Mapping data into a higher-dimensional space without explicit computation
The kernel trick implicitly maps data to a higher-dimensional feature space using a kernel function, enabling linear separation of nonlinearly separable data.
Question 5: What does the learning rate hyperparameter control in gradient descent?
- The number of training epochs
- The size of steps taken toward the minimum of the loss function (Correct answer)
- The ratio of training to validation data
- The number of features used per iteration
Correct answer: The size of steps taken toward the minimum of the loss function
The learning rate determines how large each parameter update step is; too large causes divergence, too small causes slow convergence.
Question 6: Which evaluation metric is most appropriate when the cost of false negatives greatly exceeds the cost of false positives?
- Accuracy
- Specificity
- Recall (Sensitivity) (Correct answer)
- Precision
Correct answer: Recall (Sensitivity)
Recall measures the proportion of actual positives correctly identified, making it critical when missing a positive (false negative) is costly, such as in disease detection.
Question 7: What is a key characteristic of Naive Bayes classifiers that makes them 'naive'?
- They ignore the training labels
- They assume all features are conditionally independent given the class (Correct answer)
- They require no hyperparameter tuning
- They only work with binary classification
Correct answer: They assume all features are conditionally independent given the class
Naive Bayes assumes conditional independence between features given the class label, which simplifies computation but rarely holds in practice.
Machine Learning Supervised Learning 2
Which technique helps prevent overfitting by adding a penalty term proportional to the square of the model's coefficients?