Assessment and Evaluation Flashcards

In PROC REG output, which statistic is penalized when predictors that add little explanatory power are included?

Answer: Adjusted R-Square

Adjusted R-Square accounts for the number of predictors and can decrease when a weak variable is added.

When comparing PROC LOGISTIC models on a large sample, which criterion penalizes extra parameters most heavily?

Answer: SC (Schwarz Criterion)

SC uses a penalty of p*ln(n), which exceeds AIC's 2p once n is greater than about 7.

With SELECTION=CP in PROC REG, which candidate model is generally preferred?

Answer: The model whose Cp is closest to p, the number of parameters including the intercept

A model with little bias has Mallows' Cp approximately equal to p.

A PROC REG model with the VIF option shows a predictor with VIF = 14. What does this most likely indicate?

Answer: Serious multicollinearity

VIF values above about 10 indicate that the predictor is highly correlated with the other predictors.

Which diagnostic measures how much all of the fitted values change when one observation is deleted?

Answer: Cook's D

Cook's D combines residual size and leverage to measure an observation's overall influence on the fitted values.

In PROC LOGISTIC's Association of Predicted Probabilities table, the c statistic is equivalent to which measure?

Answer: Area under the ROC curve

The c statistic is the concordance index and equals the ROC AUC.

A PROC LOGISTIC model run with the LACKFIT option returns a Hosmer-Lemeshow p-value of 0.62. What is the correct interpretation?

Answer: There is no evidence of lack of fit

A non-significant Hosmer-Lemeshow test means the observed and expected counts do not differ significantly.