MS-DS Master of Data science Master of Data Science: Statistical Inference & Regression Models 1 — Questions and Answers
Question 1: In simple linear regression, which condition does the Gauss-Markov theorem require for the OLS estimator to be BLUE (Best Linear Unbiased Estimator)?
- The error terms must follow a normal distribution
- The error terms must have constant variance and be uncorrelated (Correct answer)
- The predictor variables must be normally distributed
- The response variable must be measured on a ratio scale
Correct answer: The error terms must have constant variance and be uncorrelated
The Gauss-Markov theorem guarantees OLS is BLUE under the conditions of linearity, exogeneity, homoscedasticity (constant error variance), and no autocorrelation. Normality of errors is NOT required for BLUE status—it is only needed for exact inference (t-tests, F-tests) in finite samples.
Question 2: A researcher fits a multiple regression model and finds the F-statistic is highly significant (p < 0.001), yet none of the individual t-tests for the coefficients are significant. What is the most likely explanation?
- The model has too few observations to detect any effect
- Severe multicollinearity is inflating the standard errors of individual coefficients (Correct answer)
- The response variable violates the normality assumption
- The researcher should use logistic regression instead
Correct answer: Severe multicollinearity is inflating the standard errors of individual coefficients
When the overall F-test is significant but individual t-tests are not, severe multicollinearity is the classic culprit. Highly correlated predictors inflate standard errors, making it difficult to isolate each variable's unique contribution, even though collectively they explain significant variance.
Question 3: What does the p-value of 0.03 in a hypothesis test formally represent?
- The probability that the null hypothesis is true
- The probability of observing a test statistic as extreme or more extreme than the one obtained, given that H₀ is true (Correct answer)
- The probability that the alternative hypothesis is true
- The probability of making a Type II error
Correct answer: The probability of observing a test statistic as extreme or more extreme than the one obtained, given that H₀ is true
A p-value is the probability of obtaining a test statistic at least as extreme as the observed one, assuming the null hypothesis is true. It does NOT give the probability that H₀ is true or false—that would require a Bayesian framework.
Question 4: In a regression context, what is the consequence of omitting a variable that is correlated with both the outcome and an included predictor?
- Reduced model R², but unbiased coefficient estimates
- Biased and inconsistent coefficient estimates for the included predictors (Correct answer)
- Inflated standard errors for all included predictors
- The model will fail to converge during estimation
Correct answer: Biased and inconsistent coefficient estimates for the included predictors
Omitting a confounding variable causes omitted variable bias (OVB). Because the omitted variable correlates with an included predictor, the OLS estimator absorbs part of the omitted variable's effect into the included predictor's coefficient, making it biased and inconsistent.
Question 5: Which transformation is most appropriate to stabilize variance (correct heteroscedasticity) when the residual spread increases proportionally with the fitted values?
- Square root transformation of the predictor
- Log transformation of the response variable (Correct answer)
- Standardizing all predictors to z-scores
- Adding a quadratic term for the predictor
Correct answer: Log transformation of the response variable
When variance grows proportionally with the mean (a common pattern in count or ratio data), a log transformation of the response variable compresses large values more than small ones, stabilizing variance and often linearizing a multiplicative relationship.
Question 6: In the context of statistical inference, what does increasing the sample size primarily affect, holding effect size and significance level constant?
- It decreases the probability of a Type I error (α)
- It increases the statistical power of the test (Correct answer)
- It widens confidence intervals for parameter estimates
- It reduces the magnitude of the estimated effect size
Correct answer: It increases the statistical power of the test
Statistical power (1 − β) is the probability of correctly rejecting a false null hypothesis. Larger samples reduce the standard error, making the test statistic larger for any given true effect, which increases power without changing the pre-set α level.
MS-DS Master of Data science Master of Data Science: Statistical Inference & Regression Models 1
In simple linear regression, which condition does the Gauss-Markov theorem require for the OLS estimator to be BLUE (Best Linear Unbiased Estimator)?