Machine Learning Regression 5 — Questions and Answers
Question 1: In Bayesian linear regression, what replaces the single point estimate of coefficients?
- A regularization penalty
- A posterior probability distribution over coefficients (Correct answer)
- A cross-validated error estimate
- A bootstrap confidence interval
Correct answer: A posterior probability distribution over coefficients
Bayesian regression represents uncertainty by computing a posterior distribution over coefficients, combining prior beliefs with observed data.
Question 2: What does Cook's Distance measure in regression diagnostics?
- The correlation between residuals and fitted values
- The influence of removing an observation on all fitted values (Correct answer)
- The leverage of high-dimensional data points
- The degree of multicollinearity
Correct answer: The influence of removing an observation on all fitted values
Cook's Distance quantifies how much all predicted values change if a particular observation is removed, identifying influential data points.
Question 3: Which transformation is commonly applied to the response variable to address right-skewed data in regression?
- Square root or log transformation (Correct answer)
- Z-score standardization
- Min-max normalization
- One-hot encoding
Correct answer: Square root or log transformation
Log or square root transformations can stabilize variance and normalize right-skewed response distributions for regression analysis.
Question 4: In principal component regression (PCR), what is regressed on the response variable?
- The original correlated predictors
- Principal components derived from predictors (Correct answer)
- Residuals from a first-stage regression
- Standardized predictor ranks
Correct answer: Principal components derived from predictors
PCR uses uncorrelated principal components of the predictors as inputs, addressing multicollinearity by projecting into a lower-dimensional space.
Question 5: What is the 'bias-variance tradeoff' in the context of regression models?
- Balancing training speed against prediction accuracy
- Balancing underfitting (high bias) against overfitting (high variance) (Correct answer)
- Choosing between L1 and L2 regularization
- Tradeoff between interpretability and accuracy
Correct answer: Balancing underfitting (high bias) against overfitting (high variance)
The bias-variance tradeoff describes how increasing model complexity reduces bias but increases variance, requiring a balance for optimal generalization.
Question 6: Which scenario best illustrates multicollinearity in multiple regression?
- Two predictors have zero correlation with the response
- Two predictors (height in cm and height in inches) are nearly perfectly correlated (Correct answer)
- The response variable has a non-normal distribution
- Residuals increase with fitted values
Correct answer: Two predictors (height in cm and height in inches) are nearly perfectly correlated
Multicollinearity occurs when predictors are highly correlated with each other, making coefficient estimates unstable and hard to interpret.
Question 7: In support vector regression (SVR), what does the ε (epsilon) parameter control?
- The regularization strength of the model
- The width of the tube within which errors are ignored (Correct answer)
- The kernel bandwidth for non-linear mappings
- The learning rate of the optimization
Correct answer: The width of the tube within which errors are ignored
The ε-insensitive tube in SVR means errors smaller than ε incur no penalty, controlling tolerance for deviation from the regression line.
Machine Learning Regression 5
In Bayesian linear regression, what replaces the single point estimate of coefficients?