Data Science with Python Model Evaluation and Validation Questions and Answers

Question 1

A data scientist develops a model to detect a rare but critical disease. The cost of failing to identify a sick patient (a false negative) is extremely high. Which evaluation metric should be prioritized to minimize the number of missed cases?

Accepted Answer

Recall (Sensitivity)

Answer

Recall, also known as Sensitivity or True Positive Rate, measures the model's ability to identify all actual positive cases. It is calculated as TP / (TP + FN). In a medical scenario where missing a case (a False Negative) is highly consequential, maximizing recall is the primary objective.

Question 2

What is the primary purpose of using k-fold cross-validation in machine learning?

Accepted Answer

To provide a more robust and stable estimate of a model's performance on unseen data.

Answer

K-fold cross-validation involves repeatedly training and testing a model on different subsets ('folds') of the data. By averaging the performance scores from each fold, it provides a more reliable and less biased estimate of how the model will generalize to new, independent data compared to a single train-test split.

Question 3

A machine learning model performs exceptionally well on the training data with 99% accuracy but its accuracy drops to 70% on the validation data. This phenomenon is a clear indicator of what issue?

Accepted Answer

Overfitting

Answer

Overfitting occurs when a model learns the training data too well, including its noise and random fluctuations, to the point that it cannot generalize to new, unseen data. A large performance gap between the training set and the validation/test set is the classic symptom of overfitting.

Question 4

In the context of a binary classification model, which of the following statements best describes the bias-variance tradeoff?

Accepted Answer

Increasing model complexity typically decreases bias but increases variance.

Answer

The bias-variance tradeoff is a fundamental concept where bias represents errors from overly simplistic assumptions (underfitting), and variance represents errors from being too sensitive to the training data (overfitting). As a model's complexity increases, its ability to fit the training data improves, thus decreasing bias. However, this increased complexity makes it more likely to model the noise in the training data, thus increasing its variance.

Question 5

A data scientist is building a spam filter. They want to ensure that when an email is flagged as spam, it is very likely to actually be spam, minimizing the chance of legitimate emails ending up in the spam folder. Which metric should they focus on maximizing?

Accepted Answer

Precision

Answer

Precision measures the proportion of positive predictions that were actually correct. It is calculated as TP / (TP + FP). In this scenario, a False Positive (a legitimate email incorrectly marked as spam) is very costly. Maximizing precision directly addresses this by penalizing false positives.

Data Science with Python Certification Practice Test

Data Science with Python Certification Practice Test

Data Science with Python Model Evaluation and Validation Questions and Answers