MS-DS Master of Data science MS-DS Master of Data science Model Evaluation and Validation Questions and Answers 3

Question 1

What is the primary advantage of using the Matthews Correlation Coefficient (MCC) over accuracy for binary classification on imbalanced datasets?

Accepted Answer

MCC accounts for all four confusion matrix categories proportionally

Answer

MCC uses all four confusion matrix values (TP, TN, FP, FN) and produces a balanced measure even when classes are highly imbalanced.

Question 2

During model validation, what problem does data leakage most commonly cause?

Accepted Answer

Overly optimistic performance estimates that fail to replicate on new data

Answer

Data leakage allows information from the test set to influence training, resulting in inflated performance metrics that do not generalize.

Question 3

In the context of learning curves, what does convergence of training and validation scores at a low performance level suggest?

Accepted Answer

High bias — the model is too simple for the data

Answer

When both curves converge at low performance, the model lacks the capacity to capture the underlying patterns, indicating underfitting due to high bias.

Question 4

Which validation technique is specifically designed for time-series data to prevent future information from leaking into training?

Accepted Answer

Walk-forward (expanding window) validation

Answer

Walk-forward validation respects temporal ordering by always training on past data and testing on future data, preventing lookahead bias.

Question 5

What does the area under the Precision-Recall curve (AUPRC) measure that AUROC may fail to capture?

Accepted Answer

Model performance on the minority class in imbalanced settings

Answer

AUPRC focuses on the positive (minority) class performance and is more informative than AUROC when class imbalance is severe.

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Practice Test

MS-DS Master of Data science MS-DS Master of Data science Model Evaluation and Validation Questions and Answers 3