Machine Learning Model Evaluation 2 — Questions and Answers
Question 1: Which metric is most appropriate when the cost of false negatives is much higher than false positives, such as in cancer screening?
- Precision
- Recall (Correct answer)
- Specificity
- Accuracy
Correct answer: Recall
Recall (sensitivity) measures the proportion of actual positives correctly identified, minimizing missed cases (false negatives).
Question 2: What does a ROC curve plot on its axes?
- Precision vs Recall
- True Positive Rate vs False Positive Rate (Correct answer)
- Accuracy vs Loss
- F1 Score vs Threshold
Correct answer: True Positive Rate vs False Positive Rate
A ROC curve plots True Positive Rate (sensitivity) on the y-axis against False Positive Rate (1-specificity) on the x-axis.
Question 3: If a model achieves AUC-ROC of 0.5, what does this indicate?
- Perfect classification
- Better than random guessing
- Performance equivalent to random guessing (Correct answer)
- The model always predicts the negative class
Correct answer: Performance equivalent to random guessing
An AUC of 0.5 means the model has no discriminative ability and performs no better than random chance.
Question 4: In k-fold cross-validation with k=5, what percentage of data is used for training in each fold?
- 20%
- 50%
- 80% (Correct answer)
- 95%
Correct answer: 80%
With k=5, the data is split into 5 folds; each iteration uses 4 folds (80%) for training and 1 fold (20%) for validation.
Question 5: What is the purpose of a confusion matrix?
- To visualize decision boundaries
- To display counts of true/false positives and negatives (Correct answer)
- To plot learning curves
- To measure training speed
Correct answer: To display counts of true/false positives and negatives
A confusion matrix tabulates TP, TN, FP, and FN counts to give a detailed breakdown of classification performance.
Question 6: Which evaluation strategy is most suitable when you have very limited data?
- Hold-out validation
- Leave-One-Out Cross-Validation (LOOCV) (Correct answer)
- Train-test split 90/10
- Bootstrap with 1000 samples
Correct answer: Leave-One-Out Cross-Validation (LOOCV)
LOOCV uses every sample as a test set exactly once, maximizing training data usage and is ideal for small datasets.
Question 7: What does the F1 score represent?
- The arithmetic mean of precision and recall
- The harmonic mean of precision and recall (Correct answer)
- The geometric mean of accuracy and AUC
- The ratio of true positives to total predictions
Correct answer: The harmonic mean of precision and recall
The F1 score is the harmonic mean of precision and recall, balancing both metrics and penalizing extreme values in either.
Machine Learning Model Evaluation 2
Which metric is most appropriate when the cost of false negatives is much higher than false positives, such as in cancer screening?