Data Science Model Performance and Evaluation 2

Question 1

A spam classifier flags 95 of 100 actual spam emails but also flags 40 legitimate emails as spam. Which metric most directly reflects the cost of those false alarms?

Accepted Answer

Precision

Answer

Precision penalizes false positives, capturing the cost of legitimate emails wrongly flagged as spam.

Question 2

On a dataset where 99% of cases are negative, a model that always predicts 'negative' achieves 99% accuracy. What does this illustrate?

Accepted Answer

The accuracy paradox on imbalanced data

Answer

The accuracy paradox shows accuracy can be misleadingly high on imbalanced data despite a useless model.

Question 3

Which metric is the harmonic mean of precision and recall?

Accepted Answer

F1 score

Answer

The F1 score is the harmonic mean of precision and recall, balancing both.

Question 4

A regression model has a low training error but a much higher test error. This is a sign of:

Accepted Answer

Overfitting

Answer

A large gap with low training error and high test error indicates overfitting (high variance).

Question 5

The area under the ROC curve (AUC) represents the probability that the model ranks a randomly chosen positive higher than a randomly chosen negative. An AUC of 0.5 means:

Accepted Answer

No better than random guessing

Answer

An AUC of 0.5 corresponds to random ranking with no discriminative ability.

Data Science Practice Test

Data Science Practice Test

Data Science Model Performance and Evaluation 2