Data Science with Python Certification Data Science with Python Model Evaluation and Validation 5 — Questions and Answers
Question 1: Which technique can be used to visualize the tradeoff between precision and recall across different classification thresholds?
- ROC curve
- Precision-Recall curve (Correct answer)
- Learning curve
- Residual plot
Correct answer: Precision-Recall curve
The Precision-Recall curve plots precision vs. recall at every possible threshold, especially useful for imbalanced datasets.
Question 2: What is the purpose of a learning curve in model evaluation?
- To tune hyperparameters automatically
- To diagnose whether a model suffers from high bias or high variance (Correct answer)
- To measure feature importance
- To visualize decision boundaries
Correct answer: To diagnose whether a model suffers from high bias or high variance
Learning curves plot training and validation scores as a function of training set size, revealing whether the bottleneck is bias (underfitting) or variance (overfitting).
Question 3: In time-series model evaluation, why is standard k-fold cross-validation inappropriate?
- It requires too much memory
- It allows future data to leak into training, violating temporal ordering (Correct answer)
- It does not support regression tasks
- It always produces overfitting
Correct answer: It allows future data to leak into training, violating temporal ordering
Standard k-fold randomly shuffles data, so validation folds may contain earlier timestamps than training folds, constituting look-ahead bias.
Question 4: What does sklearn.metrics.mean_absolute_percentage_error (MAPE) measure?
- Average squared error as a percentage
- Average absolute percent deviation between predicted and actual values (Correct answer)
- Percentage of predictions within one standard deviation
- Root mean squared error normalized by mean
Correct answer: Average absolute percent deviation between predicted and actual values
MAPE computes the average of |actual - predicted| / |actual| * 100%, expressing error as a percentage of the true value.
Question 5: Which of the following statements about the R-squared (R²) metric is TRUE?
- R² can never be negative
- R² of 1.0 always indicates a good model
- R² measures the proportion of variance in the target explained by the model (Correct answer)
- R² is equivalent to the Pearson correlation coefficient
Correct answer: R² measures the proportion of variance in the target explained by the model
R² represents the fraction of target variance explained by the model, ranging from -∞ (worse than mean baseline) to 1.0 (perfect fit).
Question 6: When using RandomizedSearchCV instead of GridSearchCV, what is the primary tradeoff?
- RandomizedSearchCV always finds better hyperparameters
- RandomizedSearchCV is faster but may miss the optimal hyperparameter combination (Correct answer)
- RandomizedSearchCV requires more data
- RandomizedSearchCV does not support cross-validation
Correct answer: RandomizedSearchCV is faster but may miss the optimal hyperparameter combination
RandomizedSearchCV samples a fixed number of parameter combinations randomly, reducing computation at the cost of potentially not evaluating the global optimum.
Question 7: What is the macro-average F1-score, and when is it preferable to the weighted-average F1-score?
- Average F1 weighted by class support; preferable for balanced datasets
- Unweighted average F1 across all classes; preferable when all classes are equally important regardless of frequency (Correct answer)
- F1 score for the majority class only; preferable for imbalanced datasets
- Geometric mean of precision and recall; always preferred
Correct answer: Unweighted average F1 across all classes; preferable when all classes are equally important regardless of frequency
Macro-average computes F1 per class and averages without weighting by support, giving equal importance to rare and common classes.
Data Science with Python Certification Data Science with Python Model Evaluation and Validation 5
Which technique can be used to visualize the tradeoff between precision and recall across different classification thresholds?