Data Science with Python Model Evaluation and Validation Flashcards

Which metric is most appropriate for evaluating a model on a severely imbalanced binary classification dataset?

Answer: F1-score

F1-score balances precision and recall, making it far more informative than accuracy when one class heavily dominates.

In scikit-learn, which cross-validation strategy preserves the percentage of samples for each class label?

Answer: StratifiedKFold

StratifiedKFold ensures each fold maintains the same class distribution as the overall dataset.

What does a ROC AUC score of 0.5 indicate about a binary classifier?

Answer: Performance equivalent to random guessing

An AUC of 0.5 means the model has no discriminative ability and performs no better than a random classifier.

Which Python function from sklearn.metrics computes the confusion matrix for a classification model?

Answer: sklearn.metrics.confusion_matrix()

sklearn.metrics.confusion_matrix(y_true, y_pred) returns the matrix of true vs. predicted class counts.

When using GridSearchCV in scikit-learn, what does the refit=True parameter do?

Answer: Refits the best estimator on the entire training set after search

refit=True causes GridSearchCV to retrain the best-found model on the complete training data, making it ready for prediction.

Which of the following best describes the purpose of a validation set (as distinct from a test set)?

Answer: Data used to tune hyperparameters during development

The validation set is used to select hyperparameters and model architecture, while the test set provides the final unbiased evaluation.

What is the primary risk of performing feature selection on the entire dataset before applying cross-validation?

Answer: Data leakage leading to overly optimistic performance estimates

Selecting features using the full dataset leaks information from the validation folds into training, inflating performance metrics.