Data Science Model Evaluation and Validation 2

Question 1

In a binary classifier, what does precision measure?

Accepted Answer

Of all predicted positives, how many were actually positive

Answer

Of all actual positives, how many were predicted positive

Answer

The overall fraction of correct predictions

Answer

The area under the ROC curve

Question 2

What does recall (sensitivity) measure?

Accepted Answer

Of all actual positives, how many were correctly identified

Answer

Of all predicted positives, how many were correct

Answer

The fraction of true negatives identified

Answer

The harmonic mean of precision and accuracy

Question 3

Why is accuracy a poor metric for highly imbalanced datasets?

Accepted Answer

A model predicting the majority class always can score high while missing the minority class

Answer

It is impossible to compute on imbalanced data

Answer

It always equals 50%

Answer

It only works for regression problems

Question 4

The F1 score is best described as the:

Accepted Answer

Harmonic mean of precision and recall

Answer

Arithmetic mean of precision and recall

Answer

Product of accuracy and recall

Answer

Difference between precision and recall

Question 5

What does the ROC curve plot?

Accepted Answer

True positive rate against false positive rate across thresholds

Answer

Precision against recall

Answer

Accuracy against threshold

Answer

Loss against epoch

Question 6

An AUC of 0.5 indicates a classifier that:

Accepted Answer

Performs no better than random guessing

Answer

Is perfect

Answer

Always predicts the positive class

Answer

Has zero false positives

Question 7

In a confusion matrix, a false negative occurs when the model:

Accepted Answer

Predicts negative for an actual positive case

Answer

Predicts positive for an actual negative case

Answer

Predicts positive for an actual positive case

Answer

Predicts negative for an actual negative case