Machine Learning Regression 2 — Questions and Answers
Question 1: Which metric measures the average of squared differences between predicted and actual values?
- Mean Absolute Error
- Mean Squared Error (Correct answer)
- R-squared
- Huber Loss
Correct answer: Mean Squared Error
Mean Squared Error (MSE) computes the average of squared residuals, penalizing large errors more heavily than MAE.
Question 2: In Ridge regression, what happens to the coefficients as the regularization parameter λ increases?
- Coefficients grow unboundedly
- Coefficients shrink toward zero (Correct answer)
- Coefficients become binary
- Coefficients oscillate
Correct answer: Coefficients shrink toward zero
Ridge regression adds an L2 penalty that shrinks all coefficients toward (but not exactly to) zero as λ increases.
Question 3: A regression model has R² = 0.85. What does this indicate?
- The model explains 85% of the variance in the target variable (Correct answer)
- The model has 85% prediction accuracy
- 85% of predictions fall within one standard deviation
- The RMSE equals 0.15
Correct answer: The model explains 85% of the variance in the target variable
R² (coefficient of determination) represents the proportion of variance in the dependent variable explained by the model.
Question 4: Which assumption of linear regression states that residuals should not be correlated with each other?
- Homoscedasticity
- Multicollinearity
- Independence of errors (Correct answer)
- Normality of residuals
Correct answer: Independence of errors
The independence of errors assumption requires that residuals are uncorrelated — violation often occurs in time-series data.
Question 5: What does the Durbin-Watson statistic test for in regression analysis?
- Multicollinearity
- Heteroscedasticity
- Autocorrelation in residuals (Correct answer)
- Normality of coefficients
Correct answer: Autocorrelation in residuals
The Durbin-Watson statistic detects autocorrelation in residuals; values near 2 indicate no autocorrelation.
Question 6: In polynomial regression, what is the primary risk of choosing a very high degree polynomial?
- Underfitting the training data
- Overfitting with poor generalization (Correct answer)
- Slower convergence of gradient descent
- Loss of interpretability only
Correct answer: Overfitting with poor generalization
High-degree polynomials can perfectly fit training data but overfit, capturing noise and generalizing poorly to new data.
Question 7: Which technique is used to select the optimal regularization parameter in Ridge or Lasso regression?
- Bootstrapping
- Cross-validation (Correct answer)
- Bayesian inference
- Maximum likelihood estimation
Correct answer: Cross-validation
Cross-validation evaluates model performance across different λ values to select the one that minimizes validation error.
Machine Learning Regression 2
Which metric measures the average of squared differences between predicted and actual values?