MS-DS Master of Data science Supervised Learning Algorithms 4 — Questions and Answers
Question 1: What distinguishes Ridge regression from ordinary least squares?
- Ridge adds an L1 penalty that drives some coefficients to exactly zero
- Ridge adds an L2 penalty that shrinks coefficients toward zero without eliminating them (Correct answer)
- Ridge uses a different loss function based on absolute errors
- Ridge applies feature selection before fitting the model
Correct answer: Ridge adds an L2 penalty that shrinks coefficients toward zero without eliminating them
Ridge regression (L2 regularization) adds a penalty equal to the sum of squared coefficients, shrinking them uniformly toward zero but rarely setting any exactly to zero.
Question 2: In a random forest, what is the purpose of feature randomness (selecting a subset of features at each split)?
- It speeds up training by reducing the number of computations per split
- It decorrelates the individual trees, reducing ensemble variance (Correct answer)
- It ensures all features contribute equally to every tree
- It prevents trees from exceeding their maximum depth
Correct answer: It decorrelates the individual trees, reducing ensemble variance
By considering only a random subset of features at each split, trees in a random forest are de-correlated, so their errors are less likely to coincide, reducing overall variance.
Question 3: Which scenario is most likely to benefit from using a polynomial regression model instead of linear regression?
- When the relationship between the feature and target is curved or non-linear (Correct answer)
- When there are more features than training samples
- When you need to enforce sparsity in the coefficient vector
- When the target variable is categorical rather than continuous
Correct answer: When the relationship between the feature and target is curved or non-linear
Polynomial regression extends linear regression by adding polynomial terms, allowing it to fit curved relationships between predictors and a continuous outcome.
Question 4: What is the purpose of the learning rate in gradient descent optimization?
- It determines how many training epochs are run
- It controls the step size taken in the direction of the negative gradient (Correct answer)
- It sets the initial values of the model weights
- It scales the regularization penalty applied to weights
Correct answer: It controls the step size taken in the direction of the negative gradient
The learning rate scales the gradient to determine how large a step is taken when updating parameters; too large causes divergence, too small causes slow convergence.
Question 5: In logistic regression, what function maps the linear combination of inputs to a probability?
- ReLU activation function
- Softmax function
- Sigmoid (logistic) function (Correct answer)
- Hyperbolic tangent (tanh) function
Correct answer: Sigmoid (logistic) function
The sigmoid function maps any real-valued linear combination to a value between 0 and 1, which can be interpreted as a class probability in binary logistic regression.
Question 6: Which metric is most appropriate for evaluating a classifier on a highly imbalanced dataset where the minority class is critical?
- Accuracy
- Mean squared error
- F1 score (Correct answer)
- R-squared
Correct answer: F1 score
F1 score is the harmonic mean of precision and recall, making it sensitive to performance on the minority class and appropriate when false negatives and false positives both have high cost.
Question 7: What is early stopping in neural network training?
- Terminating training when the learning rate drops below a threshold
- Halting training when validation loss stops improving to prevent overfitting (Correct answer)
- Stopping training after a fixed number of mini-batches
- Removing neurons that have not activated in the last epoch
Correct answer: Halting training when validation loss stops improving to prevent overfitting
Early stopping monitors validation performance and halts training when it stops improving, acting as a regularizer that prevents the model from overfitting the training set.
MS-DS Master of Data science Supervised Learning Algorithms 4
What distinguishes Ridge regression from ordinary least squares?