Machine Learning Model Evaluation 5

Question 1

What is the 'bias-variance trade-off' in model evaluation?

Accepted Answer

The tension between underfitting (high bias) and overfitting (high variance)

Answer

High bias leads to underfitting while high variance leads to overfitting; the trade-off is finding the model complexity that minimizes both.

Question 2

Which cross-validation approach is most appropriate for time-series data?

Accepted Answer

Walk-forward (time-series) cross-validation

Answer

Walk-forward validation respects temporal order by always training on past data and testing on future data, preventing look-ahead bias.

Question 3

What does 'BLEU score' measure in NLP model evaluation?

Accepted Answer

Similarity between machine-generated and reference translations

Answer

BLEU (Bilingual Evaluation Understudy) compares n-gram overlap between machine-generated text and reference translations.

Question 4

What is the 'Brier score' used to evaluate?

Accepted Answer

Probabilistic prediction quality (calibration and resolution)

Answer

The Brier score measures the mean squared difference between predicted probabilities and actual binary outcomes, assessing probabilistic forecasts.

Question 5

When should you prefer macro-averaged F1 over micro-averaged F1 in multi-class classification?

Accepted Answer

When you want each class to contribute equally regardless of size

Answer

Macro-average computes F1 per class then averages them, giving equal weight to all classes regardless of their sample size.

Machine Learning Practice Test