Assessment and Evaluation Flashcards

In a confusion matrix, how is sensitivity calculated?

Answer: TP / (TP + FN)

Sensitivity, the true positive rate, is the share of actual events that are correctly predicted.

A model correctly classifies 870 of 900 actual non-events. What is its specificity?

Answer: 96.7%

Specificity = TN / (TN + FP) = 870/900 ≈ 96.7%.

Which PROC LOGISTIC MODEL statement option produces a classification table across a range of probability cutoffs?

Answer: CTABLE

CTABLE prints a classification table, and PPROB= specifies the cutoff values.

A model has 2% error on the training data and 18% error on the validation data. What does this suggest?

Answer: Overfitting

A large gap between training and validation performance is the classic sign that the model has memorized noise.

Which PROC GLMSELECT statement randomly splits the data into training, validation, and test roles?

Answer: PARTITION

PARTITION FRACTION(VALIDATE= TEST=) randomly assigns observations to roles.

The overall response rate is 5%, but the top decile of model scores has a response rate of 15%. What is the lift for the top decile?

Answer: 3.0

Lift = 15% / 5% = 3.0, so the top decile has three times the baseline response rate.

A confusion matrix shows TP=80, FN=20, FP=30, TN=870. What is the misclassification rate?

Answer: 5.0%

Misclassification = (FN + FP) / Total = 50/1000 = 5%.