FREE Data Science Model Evaluation and Validation Questions and Answers

Question 1

A data scientist is developing a binary classification model to detect a rare but life-threatening disease.
The consequences of failing to identify a sick patient (a false negative) are far more severe than incorrectly classifying a healthy patient as sick (a false positive).
Which evaluation metric should be prioritized for optimization?

Accepted Answer

Recall (Sensitivity)

Answer

Accuracy

Answer

Precision

Answer

Specificity

Question 2

What is the primary advantage of using K-Fold Cross-Validation compared to a single train-test split for model evaluation?

Accepted Answer

It provides a more robust estimate of the model's performance on unseen data.

Answer

It is computationally much faster to execute.

Answer

It eliminates the need for a separate test set entirely.

Answer

It always results in a higher model accuracy score.

Question 3

A real estate company has built a model to predict house prices. They want to evaluate the model's performance by measuring the average absolute difference between the predicted prices and the actual prices, in the original currency (e.g., dollars). Which metric is most suitable for this purpose?

Accepted Answer

Mean Absolute Error (MAE)

Answer

R-squared (R²)

Answer

Mean Squared Error (MSE)

Answer

Root Mean Squared Error (RMSE)

Question 4

Which of the following best describes a model that is suffering from high bias?

Accepted Answer

The model is too simple and fails to capture the underlying patterns in both the training and test data.

Answer

The model performs well on the training data but poorly on the test data.

Answer

The model is too complex and captures random noise in the training data.

Answer

The model's performance varies significantly with different subsets of training data.

Question 5

A data science team is tuning a model with a very large hyperparameter space and has limited computational resources. They need an efficient method to find a good combination of hyperparameters without exhaustively testing every possibility. Which approach is generally more efficient than Grid Search in this scenario?

Accepted Answer

Random Search

Answer

Leave-One-Out Cross-Validation

Answer

A/B Testing

Answer

Manual Tuning

Question 6

In the context of a binary classification model, what does the Area Under the ROC Curve (AUC) represent?

Accepted Answer

The probability that the model will rank a randomly chosen positive instance higher than a randomly chosen negative instance.

Answer

The model's accuracy when the classification threshold is set to 0.5.

Answer

The trade-off between the number of true positives and false positives at a single, optimal threshold.

Answer

The total area of false positives and false negatives combined.