Data Science Model Performance and Evaluation 3

Question 1

For a regression model, which metric penalizes large errors more heavily because it squares the residuals?

Accepted Answer

Mean Squared Error (MSE)

Answer

MSE squares residuals, so larger errors contribute disproportionately more to the total.

Question 2

R-squared of 0.0 in a regression model indicates the model:

Accepted Answer

Explains no more variance than predicting the mean

Answer

An R-squared of 0 means the model does no better than always predicting the target mean.

Question 3

When tuning hyperparameters, why should you use a separate validation set rather than the test set?

Accepted Answer

To avoid leaking test information and overly optimistic estimates

Answer

Tuning on the test set leaks information and inflates the final reported performance.

Question 4

A model's predicted probabilities are systematically too confident. Which technique addresses this?

Accepted Answer

Probability calibration (e.g., Platt scaling)

Answer

Calibration methods like Platt scaling or isotonic regression align predicted probabilities with observed frequencies.

Question 5

In the bias-variance tradeoff, a model that is too simple to capture the underlying pattern exhibits:

Accepted Answer

High bias, low variance

Answer

An overly simple model underfits, showing high bias and low variance.

Data Science Practice Test