Data Science with Python Certification Data Science with Python Model Evaluation and Validation 2 — Questions and Answers
Question 1: Which metric is most appropriate for evaluating a model on a severely imbalanced binary classification dataset?
- Accuracy
- F1-score (Correct answer)
- Mean Squared Error
- R-squared
Correct answer: F1-score
F1-score balances precision and recall, making it far more informative than accuracy when one class heavily dominates.
Question 2: In scikit-learn, which cross-validation strategy preserves the percentage of samples for each class label?
- KFold
- ShuffleSplit
- StratifiedKFold (Correct answer)
- LeaveOneOut
Correct answer: StratifiedKFold
StratifiedKFold ensures each fold maintains the same class distribution as the overall dataset.
Question 3: What does a ROC AUC score of 0.5 indicate about a binary classifier?
- Perfect classification
- Better than random chance
- Performance equivalent to random guessing (Correct answer)
- Complete misclassification
Correct answer: Performance equivalent to random guessing
An AUC of 0.5 means the model has no discriminative ability and performs no better than a random classifier.
Question 4: Which Python function from sklearn.metrics computes the confusion matrix for a classification model?
- sklearn.metrics.classification_report()
- sklearn.metrics.confusion_matrix() (Correct answer)
- sklearn.metrics.roc_auc_score()
- sklearn.metrics.accuracy_score()
Correct answer: sklearn.metrics.confusion_matrix()
sklearn.metrics.confusion_matrix(y_true, y_pred) returns the matrix of true vs. predicted class counts.
Question 5: When using GridSearchCV in scikit-learn, what does the refit=True parameter do?
- Refits the model on each fold separately
- Refits the best estimator on the entire training set after search (Correct answer)
- Runs cross-validation twice
- Saves all fitted models to disk
Correct answer: Refits the best estimator on the entire training set after search
refit=True causes GridSearchCV to retrain the best-found model on the complete training data, making it ready for prediction.
Question 6: Which of the following best describes the purpose of a validation set (as distinct from a test set)?
- Final unbiased estimate of model performance
- Data used to tune hyperparameters during development (Correct answer)
- Data used to train model weights
- Data used for feature engineering only
Correct answer: Data used to tune hyperparameters during development
The validation set is used to select hyperparameters and model architecture, while the test set provides the final unbiased evaluation.
Question 7: What is the primary risk of performing feature selection on the entire dataset before applying cross-validation?
- Underfitting the model
- Data leakage leading to overly optimistic performance estimates (Correct answer)
- Slower training time
- Increased model variance
Correct answer: Data leakage leading to overly optimistic performance estimates
Selecting features using the full dataset leaks information from the validation folds into training, inflating performance metrics.
Data Science with Python Certification Data Science with Python Model Evaluation and Validation 2
Which metric is most appropriate for evaluating a model on a severely imbalanced binary classification dataset?