Data Science Data Science Model Performance and Evaluation Questions and Answers 2

Question 1

Which metric is most appropriate for evaluating a classification model when the dataset has a severe class imbalance?

Accepted Answer

Precision-Recall AUC

Answer

Precision-Recall AUC focuses on the minority class performance and is far more informative than accuracy when classes are heavily imbalanced.

Question 2

What does a high variance and low bias in a model typically indicate?

Accepted Answer

The model is overfitting the training data

Answer

High variance with low bias means the model fits training data very closely but fails to generalize, which is the hallmark of overfitting.

Question 3

In k-fold cross-validation with k=10, how many times is each data point used in the test set?

Accepted Answer

1 time

Answer

In k-fold cross-validation, the data is split into k folds, and each fold serves as the test set exactly once across all iterations.

Question 4

What is the primary purpose of a calibration curve (reliability diagram) in model evaluation?

Accepted Answer

To assess whether predicted probabilities match actual outcome frequencies

Answer

A calibration curve plots predicted probabilities against actual frequencies to reveal whether a model's confidence scores are trustworthy.

Question 5

Which evaluation approach is most suitable when you need to assess a time-series forecasting model?

Accepted Answer

Walk-forward validation

Answer

Walk-forward validation respects temporal ordering by always training on past data and testing on future data, preventing data leakage.

Data Science Practice Test