MS-DS Master of Data science Statistical Inference & Regression Models 4 — Questions and Answers
Question 1: LASSO regression differs from ridge regression primarily because LASSO:
- Uses an L2 penalty and shrinks all coefficients equally
- Can reduce some coefficients exactly to zero, performing variable selection (Correct answer)
- Does not require standardization of predictors
- Always produces lower prediction error than ridge
Correct answer: Can reduce some coefficients exactly to zero, performing variable selection
LASSO's L1 penalty (λΣ|β|) can set coefficients to exactly zero, effectively performing automatic variable selection unlike ridge.
Question 2: A researcher applies the Box-Cox transformation to the response variable in a linear regression. The primary reason for this transformation is to:
- Reduce the number of predictor variables
- Address non-normality and non-constant variance in residuals (Correct answer)
- Increase the R² of the model
- Remove multicollinearity between predictors
Correct answer: Address non-normality and non-constant variance in residuals
The Box-Cox transformation finds an optimal power transformation of Y to better satisfy normality and homoscedasticity assumptions.
Question 3: In Bayesian inference, the posterior distribution is proportional to:
- The prior distribution only
- The likelihood function only
- The likelihood times the prior (Correct answer)
- The marginal likelihood divided by the prior
Correct answer: The likelihood times the prior
Bayes' theorem states posterior ∝ likelihood × prior, combining observed data evidence with prior beliefs.
Question 4: When using stepwise regression for variable selection, a major statistical concern is:
- Increased computational complexity
- Inflated Type I error rates and optimistically biased model fit statistics (Correct answer)
- Inability to handle continuous predictors
- Violation of the central limit theorem
Correct answer: Inflated Type I error rates and optimistically biased model fit statistics
Stepwise selection involves multiple testing, inflating Type I error rates and producing overly optimistic R² and p-values for the selected model.
Question 5: The Gauss-Markov theorem guarantees that OLS estimators are BLUE. What does BLUE stand for?
- Biased, Linear, Unbiased, Efficient
- Best Linear Unbiased Estimators (Correct answer)
- Bayesian Linear Unbiased Estimators
- Best Logistic Unbiased Estimators
Correct answer: Best Linear Unbiased Estimators
Gauss-Markov proves OLS produces Best (minimum variance) Linear Unbiased Estimators when its classical assumptions hold.
Question 6: A leverage point in regression analysis is an observation that:
- Has a large residual relative to the fitted value
- Has an extreme value in predictor space and high influence on the fitted line (Correct answer)
- Is an outlier in the response variable only
- Has high Cook's distance but small residual
Correct answer: Has an extreme value in predictor space and high influence on the fitted line
Leverage measures how far an observation's predictor values are from the mean of predictors; high leverage points can strongly influence regression estimates.
Question 7: The Wald test for a logistic regression coefficient β tests H₀: β = 0 by computing:
- The ratio of the coefficient to its standard error, squared (Correct answer)
- The difference between log-likelihoods of two nested models
- The chi-square statistic from a contingency table
- The F-ratio of between-group to within-group variance
Correct answer: The ratio of the coefficient to its standard error, squared
The Wald statistic is (β̂/SE(β̂))², which follows a chi-square distribution with 1 df under H₀.
MS-DS Master of Data science Statistical Inference & Regression Models 4
LASSO regression differs from ridge regression primarily because LASSO: