DSE - Data Science Model Evaluation and Validation Questions and Answers

Question 1

A data scientist is developing a classification model to predict customer churn.
The dataset is highly imbalanced, with only 5% of customers having churned.
Which evaluation metric is most appropriate to assess the model's performance in identifying the minority class (churners)?

Accepted Answer

Recall (Sensitivity)

Answer

Accuracy

Answer

Specificity

Answer

Mean Squared Error (MSE)

Question 2

Which of the following best describes the purpose of K-Fold Cross-Validation?

Accepted Answer

To obtain a more robust estimate of a model's performance on unseen data by using the entire dataset for both training and validation.

Answer

To increase the training data size by creating synthetic samples.

Answer

To reduce the number of features in the dataset before training.

Answer

To split the data into a single training set and a single testing set for final model evaluation.

Question 3

A machine learning model exhibits high variance but low bias. What is the most likely characteristic of this model's performance?

Accepted Answer

The model is highly sensitive to small fluctuations in the training data, likely causing overfitting.

Answer

The model is underfitting the training data.

Answer

The model performs well on both the training and test data.

Answer

The model makes strong, simplistic assumptions about the data.

Question 4

In the context of a binary classification model, what does the Area Under the ROC Curve (AUC) represent?

Accepted Answer

The probability that the model will rank a randomly chosen positive instance higher than a randomly chosen negative instance.

Answer

The model's accuracy at the optimal classification threshold.

Answer

The trade-off between precision and recall at various thresholds.

Answer

The total number of correct predictions made by the model.

Question 5

A team is building a spam filter. The primary goal is to minimize the number of legitimate emails that are incorrectly classified as spam. Which evaluation metric should the team prioritize for optimization?

Accepted Answer

Precision

Answer

Recall

Answer

Accuracy

Answer

F1-Score

Question 6

What is a primary drawback of using accuracy as the sole evaluation metric for a classification model, especially in real-world applications?

Accepted Answer

It can be highly misleading on datasets with a significant class imbalance.

Answer

It is computationally expensive to calculate.

Answer

It cannot be used for multi-class classification problems.

Answer

It only measures the performance on the negative class.