Machine Learning Regression 3 — Questions and Answers
Question 1: What distinguishes Lasso regression from Ridge regression in terms of feature selection?
- Lasso uses L2 penalty; Ridge uses L1
- Lasso can shrink coefficients exactly to zero; Ridge cannot (Correct answer)
- Lasso requires standardization; Ridge does not
- Lasso minimizes MSE; Ridge minimizes MAE
Correct answer: Lasso can shrink coefficients exactly to zero; Ridge cannot
Lasso's L1 penalty produces sparse solutions by setting some coefficients exactly to zero, effectively performing feature selection.
Question 2: In logistic regression, what function transforms the linear combination of features into a probability?
- ReLU function
- Softmax function
- Sigmoid function (Correct answer)
- Tanh function
Correct answer: Sigmoid function
The sigmoid (logistic) function maps any real-valued number to a probability between 0 and 1.
Question 3: A residual plot shows a fan-shaped pattern as fitted values increase. This indicates:
- Multicollinearity
- Heteroscedasticity (Correct answer)
- Autocorrelation
- Nonlinearity in predictors
Correct answer: Heteroscedasticity
A fan-shaped residual plot indicates heteroscedasticity — non-constant variance of residuals across fitted values.
Question 4: What is the purpose of the Variance Inflation Factor (VIF) in regression?
- Measure prediction error
- Detect multicollinearity among predictors (Correct answer)
- Test for heteroscedasticity
- Evaluate model fit
Correct answer: Detect multicollinearity among predictors
VIF quantifies how much the variance of a coefficient is inflated due to linear correlation with other predictors; VIF > 10 is concerning.
Question 5: In elastic net regression, which two penalties are combined?
- L0 and L1
- L1 and L2 (Correct answer)
- L2 and L3
- L1 and L∞
Correct answer: L1 and L2
Elastic net combines L1 (Lasso) and L2 (Ridge) penalties, balancing sparsity and coefficient shrinkage.
Question 6: Which of the following correctly describes a negative R² value?
- The model explains negative variance, which is impossible
- The model performs worse than a horizontal mean line (Correct answer)
- The model has negative prediction error
- R² can never be negative
Correct answer: The model performs worse than a horizontal mean line
Negative R² occurs when the model fits the data worse than simply predicting the mean of the target variable.
Question 7: What does 'stepwise regression' refer to?
- Regression performed one step at a time on time-series data
- Iterative procedure that adds or removes predictors based on statistical criteria (Correct answer)
- Gradient descent with fixed step sizes
- Piecewise linear regression with breakpoints
Correct answer: Iterative procedure that adds or removes predictors based on statistical criteria
Stepwise regression iteratively selects predictors by adding (forward) or removing (backward) variables based on criteria like AIC or p-value.
Machine Learning Regression 3
What distinguishes Lasso regression from Ridge regression in terms of feature selection?