Data Science with Python Model Evaluation and Validation Questions and Answers

Question 1

A data scientist develops a model to detect a rare but critical disease.
The cost of failing to identify a sick patient (a false negative) is extremely high.
Which evaluation metric should be prioritized to minimize the number of missed cases?

Accepted Answer

Recall (Sensitivity)

Answer

Accuracy

Answer

Precision

Answer

F1-Score

Question 2

What is the primary purpose of using k-fold cross-validation in machine learning?

Accepted Answer

To provide a more robust and stable estimate of a model's performance on unseen data.

Answer

To increase the speed of model training on large datasets.

Answer

To automatically select the best features for the model.

Answer

To completely eliminate the need for a final, held-out test set.

Question 3

A machine learning model performs exceptionally well on the training data with 99% accuracy but its accuracy drops to 70% on the validation data. This phenomenon is a clear indicator of what issue?

Accepted Answer

Overfitting

Answer

Underfitting

Answer

Data Leakage

Answer

High Bias

Question 4

In the context of a binary classification model, which of the following statements best describes the bias-variance tradeoff?

Accepted Answer

Increasing model complexity typically decreases bias but increases variance.

Answer

Increasing model complexity typically increases bias and decreases variance.

Answer

A model with high variance is underfitting the data.

Answer

The ideal model is one that has zero bias and zero variance.

Question 5

A data scientist is building a spam filter. They want to ensure that when an email is flagged as spam, it is very likely to actually be spam, minimizing the chance of legitimate emails ending up in the spam folder. Which metric should they focus on maximizing?

Accepted Answer

Precision

Answer

Recall

Answer

Accuracy

Answer

F1-Score

Question 6

What does the Area Under the ROC Curve (AUC) represent for a classification model?

Accepted Answer

The probability that the model will rank a randomly chosen positive instance higher than a randomly chosen negative instance.

Answer

The model's overall accuracy at the default classification threshold of 0.5.

Answer

The trade-off point where precision and recall are perfectly balanced.

Answer

The total number of correct predictions made by the model.