Machine Learning Model Evaluation 4

Question 1

What is the Matthews Correlation Coefficient (MCC) particularly useful for?

Accepted Answer

Evaluating classifiers on imbalanced datasets

Answer

MCC accounts for all four cells of the confusion matrix and provides a balanced measure even when classes are severely imbalanced.

Question 2

In a precision-recall curve, what does a model with high area under the curve (AUC-PR) signify?

Accepted Answer

The model maintains high precision while achieving high recall

Answer

High AUC-PR means the model achieves good precision across a range of recall values, indicating strong performance on the positive class.

Question 3

What is 'calibration' in the context of probabilistic classifiers?

Accepted Answer

Ensuring predicted probabilities match observed event frequencies

Answer

A calibrated model produces predicted probabilities that reflect the true likelihood of outcomes (e.g., 70% predictions correct ~70% of the time).

Question 4

Which metric would you use to evaluate a ranking model (e.g., search engine results)?

Accepted Answer

Normalized Discounted Cumulative Gain (NDCG)

Answer

NDCG evaluates the quality of ranked results by considering both relevance and position, discounting lower-ranked items.

Question 5

What is 'stratified k-fold cross-validation' designed to handle?

Accepted Answer

Datasets with imbalanced class distributions

Answer

Stratified k-fold ensures each fold maintains the same class proportion as the full dataset, critical for imbalanced classification.

Machine Learning Practice Test