MS-DS Master of Data science MS-DS Master of Data science Model Evaluation and Validation Questions and Answers 2

Question 1

In a stratified k-fold cross-validation, what is preserved in each fold compared to standard k-fold?

Accepted Answer

The proportion of each class label

Answer

Stratified k-fold ensures each fold maintains the same class distribution as the original dataset, which is critical for imbalanced data.

Question 2

Which metric is most appropriate for evaluating a model when the cost of false negatives is significantly higher than false positives?

Accepted Answer

Recall

Answer

Recall measures the proportion of actual positives correctly identified, making it the priority when missing positive cases is costly.

Question 3

What does a high variance and low bias in a model's predictions typically indicate?

Accepted Answer

The model is overfitting the training data

Answer

High variance with low bias means the model fits training data very closely but fails to generalize, which is the hallmark of overfitting.

Question 4

When performing nested cross-validation, what is the purpose of the inner loop?

Accepted Answer

Hyperparameter tuning

Answer

The inner loop of nested cross-validation is used to select the best hyperparameters, while the outer loop provides an unbiased performance estimate.

Question 5

A calibration curve (reliability diagram) plots predicted probabilities against observed frequencies. What does a perfectly calibrated model's curve look like?

Accepted Answer

A diagonal line from (0,0) to (1,1)

Answer

A perfectly calibrated model has predicted probabilities that exactly match observed frequencies, producing a diagonal line.

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Practice Test

MS-DS Master of Data science MS-DS Master of Data science Model Evaluation and Validation Questions and Answers 2