FREE MS-DS Master of Data science Machine Learning Model Evaluation Questions and Answers

Question 1

A data scientist has trained a binary classification model to detect fraudulent transactions.
After evaluating the model, they find a very high accuracy (98%) but a significantly lower recall for the 'fraud' class (20%).
The business stakeholders emphasize that minimizing missed fraud cases is paramount, even if it means a higher number of false alarms.
Which evaluation metric should the data scientist prioritize to align with this business objective?

Accepted Answer

Recall (Sensitivity) for the positive class

Answer

Precision for the positive class

Answer

Overall Accuracy

Answer

F1-Score

Question 2

Which of the following statements about cross-validation is FALSE?

Accepted Answer

Cross-validation guarantees that a model will perform identically on any new, unseen data.

Answer

It helps in estimating how well a model will generalize to an independent dataset.

Answer

K-fold cross-validation involves partitioning the dataset into K equally sized folds.

Answer

It primarily helps in detecting and mitigating overfitting by evaluating the model on unseen data.

Question 3

You are evaluating a regression model designed to predict house prices. The model consistently overestimates the prices for lower-priced homes and underestimates the prices for higher-priced homes. Which of the following evaluation plots would most effectively reveal this specific type of systematic bias?

Accepted Answer

A residual plot (residuals vs. predicted values)

Answer

A histogram of the predicted values

Answer

An ROC curve

Answer

A confusion matrix

Question 4

A machine learning engineer is comparing two classification models. Model A has an AUC-ROC of 0.85, and Model B has an AUC-ROC of 0.72. Based solely on these metrics, which conclusion is most appropriate?

Accepted Answer

Model A has a better overall ability to distinguish between positive and negative classes than Model B.

Answer

Model A has higher precision than Model B.

Answer

Model A has lower recall than Model B.

Answer

Model A is less prone to overfitting than Model B.

Question 5

When evaluating a classification model, what does a high Type I error rate (False Positive Rate) signify?

Accepted Answer

The model frequently misclassifies negative instances as positive.

Answer

The model frequently misclassifies positive instances as negative.

Answer

The model accurately identifies most positive instances.

Answer

The model accurately identifies most negative instances.

Question 6

Which of the following is NOT a typical characteristic or purpose of a validation set in machine learning?

Accepted Answer

Provides an unbiased evaluation of the final model's performance.

Answer

Used for hyperparameter tuning and model selection.

Answer

Helps detect and prevent overfitting during the training process.

Answer

Often part of a k-fold cross-validation procedure.