Troubleshooting & Problem Resolution Flashcards

Stakeholders do not trust a high-performing black-box model's predictions. What is the best resolution step?

Answer: Provide interpretability such as feature importance or a simpler surrogate model

Explainability tools build trust and help validate that the model uses sensible signals.

A scheduled scoring job runs out of memory on a large dataset. Which fix is most appropriate?

Answer: Process the data in batches or chunks

Batch processing limits memory use without changing the results.

Sampling for a churn model produced a training set that does not represent the full customer base. What is this problem called?

Answer: Sampling bias

A non-representative sample biases the model and harms generalization.

When diagnosing any failed data mining project, which step should come FIRST?

Answer: Reproduce and isolate the problem, checking data and process logs

Systematic troubleshooting begins by reproducing and isolating the issue before changing anything.

A regression model's residuals fan out as predicted values increase. Which assumption is violated?

Answer: Homoscedasticity (constant variance)

Fanning residuals indicate heteroscedasticity, often fixed by transforming the target.

Precision is high but recall is very low for a disease-screening model. How could the issue be addressed?

Answer: Lower the classification threshold to catch more positives

Lowering the threshold increases recall at the cost of some precision.

After fixing a bug in a data pipeline, what should be done to prevent recurrence?

Answer: Document the root cause and add validation checks or monitoring

Root-cause documentation and automated checks prevent the same failure from repeating.