Machine Learning Regression 4 — Questions and Answers
Question 1: In simple linear regression, what does the slope coefficient represent?
- The predicted value when X equals zero
- The change in Y for a one-unit increase in X (Correct answer)
- The correlation coefficient between X and Y
- The standard error of the regression
Correct answer: The change in Y for a one-unit increase in X
The slope coefficient β₁ represents the expected change in the dependent variable Y for each one-unit increase in the predictor X.
Question 2: Which loss function is more robust to outliers in regression?
- Mean Squared Error (MSE)
- Mean Absolute Error (MAE) (Correct answer)
- R-squared
- Log-likelihood
Correct answer: Mean Absolute Error (MAE)
MAE is more robust to outliers because it uses absolute differences, whereas MSE squares errors and thus amplifies large residuals.
Question 3: What is 'regularization' primarily designed to prevent in regression models?
- Underfitting
- Overfitting (Correct answer)
- Data leakage
- Class imbalance
Correct answer: Overfitting
Regularization penalizes model complexity to prevent overfitting, encouraging simpler models that generalize better.
Question 4: In multiple linear regression with p predictors, what is the adjusted R² designed to correct for?
- Multicollinearity between predictors
- The increase in R² from adding irrelevant predictors (Correct answer)
- Heteroscedasticity in residuals
- Non-normality of error terms
Correct answer: The increase in R² from adding irrelevant predictors
Adjusted R² penalizes for the number of predictors, preventing the illusion that adding variables always improves the model.
Question 5: What is the key advantage of quantile regression over ordinary least squares?
- It runs faster on large datasets
- It models specific quantiles of the conditional distribution, not just the mean (Correct answer)
- It automatically handles missing values
- It requires no distributional assumptions about predictors
Correct answer: It models specific quantiles of the conditional distribution, not just the mean
Quantile regression estimates any conditional quantile (e.g., median, 90th percentile), providing a richer picture of the response distribution.
Question 6: A leverage point in regression refers to an observation that:
- Has a large residual
- Has an unusual predictor value that strongly influences the fitted line (Correct answer)
- Causes the R² to decrease
- Introduces multicollinearity
Correct answer: Has an unusual predictor value that strongly influences the fitted line
Leverage measures how far an observation's predictor values are from the mean; high-leverage points can disproportionately influence the regression fit.
Question 7: Which method estimates regression coefficients by maximizing the probability of observing the training data?
- Ordinary Least Squares
- Maximum Likelihood Estimation (Correct answer)
- Ridge Regression
- Principal Component Regression
Correct answer: Maximum Likelihood Estimation
Maximum Likelihood Estimation (MLE) finds parameter values that maximize the likelihood function; for normally distributed errors, MLE equals OLS.
Machine Learning Regression 4
In simple linear regression, what does the slope coefficient represent?