MS-DS Master of Data science Supervised Learning Algorithms 3 — Questions and Answers
Question 1: What is the primary advantage of gradient boosting over a single decision tree?
- It requires no hyperparameter tuning
- It sequentially corrects errors of previous models, reducing bias (Correct answer)
- It trains faster on large datasets than a single tree
- It eliminates the need for feature scaling
Correct answer: It sequentially corrects errors of previous models, reducing bias
Gradient boosting builds an ensemble by sequentially fitting new trees to the residual errors of the combined model so far, progressively reducing bias.
Question 2: In linear regression, what does the coefficient of determination (R²) measure?
- The average absolute error between predicted and actual values
- The proportion of variance in the target explained by the model (Correct answer)
- Whether the residuals follow a normal distribution
- The correlation between two independent features
Correct answer: The proportion of variance in the target explained by the model
R² measures the fraction of the total variance in the dependent variable that is captured by the model's predictions, ranging from 0 to 1.
Question 3: Which criterion is most commonly used to measure node impurity in classification trees?
- Mean squared error
- Gini impurity (Correct answer)
- Log-likelihood
- F1 score
Correct answer: Gini impurity
Gini impurity measures the probability of misclassifying a randomly chosen element and is the default splitting criterion in most classification tree implementations.
Question 4: A Naive Bayes classifier assumes which key property about features?
- Features follow a Gaussian distribution
- All features are linearly correlated with the target
- Features are conditionally independent given the class label (Correct answer)
- Features must be standardized before training
Correct answer: Features are conditionally independent given the class label
Naive Bayes applies Bayes' theorem with the 'naive' assumption that each feature is conditionally independent of every other feature given the class label.
Question 5: What is the effect of increasing the regularization strength (C parameter) in an SVM?
- The margin widens and misclassifications are penalized less (Correct answer)
- The model focuses more on correctly classifying training points, narrowing the margin
- The kernel becomes more complex
- Training converges faster due to fewer support vectors
Correct answer: The margin widens and misclassifications are penalized less
In SVM, a smaller C allows more margin violations (softer margin), widening the margin and reducing sensitivity to individual training points.
Question 6: In the context of neural network training, what problem does the vanishing gradient address?
- Gradients become too large and destabilize weight updates in deep layers
- Gradients become negligibly small in early layers, preventing effective weight updates (Correct answer)
- The learning rate decays to zero after many epochs
- Batch normalization causes gradients to be clipped incorrectly
Correct answer: Gradients become negligibly small in early layers, preventing effective weight updates
Vanishing gradients occur when backpropagated gradients shrink exponentially through many layers, making early layers learn extremely slowly or not at all.
Question 7: Which ensemble method trains multiple models in parallel on random subsets of data and averages their predictions?
- Gradient Boosting
- AdaBoost
- Random Forest (Correct answer)
- Stacking
Correct answer: Random Forest
Random Forest builds multiple decision trees independently on bootstrapped data samples and random feature subsets, then aggregates their outputs via majority vote or averaging.
MS-DS Master of Data science Supervised Learning Algorithms 3
What is the primary advantage of gradient boosting over a single decision tree?