MS-DS Master of Data science Master of Data science Supervised Learning Algorithms 1 — Questions and Answers
Question 1: Which of the following algorithms builds an ensemble by training each new model to correct the errors made by the previous models sequentially?
- Random Forest
- Bagging
- AdaBoost (Correct answer)
- K-Nearest Neighbors
Correct answer: AdaBoost
AdaBoost (Adaptive Boosting) sequentially trains weak learners, assigning higher weights to misclassified samples so that each subsequent learner focuses on the errors of its predecessor.
Question 2: In a decision tree, which criterion measures the impurity of a node by computing the probability that two randomly chosen samples belong to different classes?
- Entropy
- Gini Impurity (Correct answer)
- Information Gain
- Variance Reduction
Correct answer: Gini Impurity
Gini Impurity measures node impurity as 1 minus the sum of squared class probabilities, representing the likelihood that a randomly selected sample would be incorrectly classified.
Question 3: What is the primary purpose of the kernel trick in Support Vector Machines?
- To reduce the number of support vectors
- To convert a regression problem into a classification problem
- To implicitly map input features into a higher-dimensional space without computing the transformation explicitly (Correct answer)
- To normalize feature scales before training
Correct answer: To implicitly map input features into a higher-dimensional space without computing the transformation explicitly
The kernel trick allows SVMs to find a linear separating hyperplane in a higher-dimensional feature space by computing inner products via a kernel function, avoiding the computational cost of explicit transformation.
Question 4: In logistic regression, which function maps the raw linear output to a probability between 0 and 1?
- ReLU function
- Hyperbolic tangent function
- Sigmoid function (Correct answer)
- Softmax function
Correct answer: Sigmoid function
The sigmoid (logistic) function σ(z) = 1/(1+e^−z) squashes any real-valued linear combination of features into the (0,1) interval, making it suitable for binary probability estimation.
Question 5: Which evaluation metric is most appropriate when the cost of false negatives is much higher than the cost of false positives in a binary classification problem?
- Accuracy
- Specificity
- Recall (Sensitivity) (Correct answer)
- F1 Score
Correct answer: Recall (Sensitivity)
Recall (TP / (TP + FN)) directly measures how many actual positives are correctly identified, making it the priority metric when missing a positive case (false negative) is especially costly, such as in medical diagnosis.
Question 6: What distinguishes Ridge Regression from Ordinary Least Squares regression?
- Ridge Regression minimizes the absolute value of coefficients rather than their squares
- Ridge Regression adds an L2 penalty term to the loss function, shrinking coefficients toward zero but not eliminating them (Correct answer)
- Ridge Regression removes correlated features automatically
- Ridge Regression fits a non-linear boundary between classes
Correct answer: Ridge Regression adds an L2 penalty term to the loss function, shrinking coefficients toward zero but not eliminating them
Ridge Regression augments the OLS cost function with a penalty proportional to the sum of squared coefficients (L2 norm), which shrinks estimates to reduce variance and overfitting, but unlike Lasso it does not produce exact zero coefficients.
MS-DS Master of Data science Master of Data science Supervised Learning Algorithms 1
Which of the following algorithms builds an ensemble by training each new model to correct the errors made by the previous models sequentially?