DSE Model Evaluation and Validation 2

Question 1

Which metric is most appropriate when false negatives are far more costly than false positives, such as in cancer screening?

Accepted Answer

Recall

Answer

Recall (sensitivity) measures how many actual positives are correctly identified, minimizing false negatives.

Question 2

What does a Precision-Recall curve's Area Under the Curve (AUC-PR) near 0.5 indicate for a balanced dataset?

Accepted Answer

The model performs at chance level

Answer

For a balanced dataset, a random classifier achieves AUC-PR near 0.5, so scores near that value suggest poor discriminative ability.

Question 3

In stratified k-fold cross-validation, what property is preserved in each fold?

Accepted Answer

The class distribution

Answer

Stratified k-fold ensures each fold reflects the overall class proportions, which is critical for imbalanced datasets.

Question 4

What is the main purpose of a validation set as distinct from both training and test sets?

Accepted Answer

To tune hyperparameters without biasing test evaluation

Answer

The validation set is used for hyperparameter tuning so the test set remains unseen and provides an unbiased generalization estimate.

Question 5

A model achieves 99% accuracy on a dataset where 99% of samples belong to one class. This is an example of:

Accepted Answer

The accuracy paradox

Answer

The accuracy paradox occurs when high accuracy is misleading because a naive baseline (predicting the majority class always) achieves the same score.

DSE - Data Science Practice Test