Data Science Supervised Learning Algorithms 3 — Questions and Answers
Question 1: In random forests, what technique reduces correlation between individual trees?
- Pruning every tree
- Random feature subsampling at each split (Correct answer)
- Using a single deep tree
- Increasing the learning rate
Correct answer: Random feature subsampling at each split
Selecting a random subset of features at each split decorrelates the trees in a random forest.
Question 2: What is the key difference between bagging and boosting?
- Bagging trains models sequentially; boosting trains in parallel
- Bagging trains models independently in parallel; boosting trains sequentially focusing on errors (Correct answer)
- Both train identically
- Boosting only uses one model
Correct answer: Bagging trains models independently in parallel; boosting trains sequentially focusing on errors
Bagging builds independent models in parallel while boosting builds models sequentially to correct prior errors.
Question 3: In gradient boosting, what do successive trees primarily fit?
- The raw target values
- The residual errors of previous trees (Correct answer)
- Random noise
- The feature means
Correct answer: The residual errors of previous trees
Each new tree in gradient boosting fits the residuals (errors) left by the prior ensemble.
Question 4: What does a large C value in an SVM control?
- A wider margin with more misclassifications allowed
- A narrower margin penalizing misclassifications more (Correct answer)
- The kernel type
- The number of support vectors only
Correct answer: A narrower margin penalizing misclassifications more
A large C penalizes misclassifications heavily, producing a narrower margin and less regularization.
Question 5: Which loss function is typically minimized in standard linear regression?
- Cross-entropy loss
- Hinge loss
- Mean squared error (Correct answer)
- Log loss
Correct answer: Mean squared error
Ordinary least squares linear regression minimizes the mean squared error between predictions and targets.
Question 6: What problem does L1 (Lasso) regularization help address that L2 does not?
- It always increases bias
- It can drive some coefficients exactly to zero, performing feature selection (Correct answer)
- It prevents any overfitting
- It speeds up training only
Correct answer: It can drive some coefficients exactly to zero, performing feature selection
L1 regularization can shrink coefficients to exactly zero, effectively selecting features.
Question 7: In a confusion matrix, what does recall measure?
- True positives over all predicted positives
- True positives over all actual positives (Correct answer)
- True negatives over all negatives
- Overall correct predictions
Correct answer: True positives over all actual positives
Recall is the proportion of actual positives correctly identified (TP / (TP + FN)).
Data Science Supervised Learning Algorithms 3
In random forests, what technique reduces correlation between individual trees?