Data Analysis & Reporting Flashcards
7 cards from real AML practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 Data Analysis & Reporting flashcards as text
What is the purpose of cross-validation in model evaluation reporting?
Answer: To provide an unbiased estimate of model performance on unseen data
Cross-validation estimates generalization performance by rotating held-out folds, reducing reliance on a single train-test split that may be lucky or unlucky.
A Kolmogorov-Smirnov (KS) test in credit scoring reports the maximum separation between which two curves?
Answer: Cumulative distribution functions of good and bad borrowers
The KS statistic measures the maximum vertical distance between the CDFs of positive and negative classes, quantifying a model's discriminatory power.
When reporting model drift in production, which type of drift refers to changes in the input feature distribution?
Answer: Covariate shift
Covariate shift occurs when the marginal distribution of input features P(X) changes while the conditional P(Y|X) remains stable.
In cohort analysis, what does retention rate measure?
Answer: The fraction of users from an initial cohort who remain active in a later period
Retention rate tracks what fraction of users who joined in a given cohort period are still active N periods later.
Which pandas function is most efficient for computing group-level summary statistics across a large DataFrame?
Answer: df.groupby().agg()
groupby().agg() leverages vectorized operations internally, making it far more efficient than row-wise apply or iterrows for group aggregations.
When evaluating a multi-class classifier, which averaging strategy for F1 score weights each class by its support (number of true instances)?
Answer: Weighted average
Weighted F1 computes F1 per class then averages them weighted by each class's support, accounting for class imbalance in multi-class settings.
A Pareto chart in a root-cause analysis report shows that 3 out of 15 defect categories account for 80% of failures. What principle does this illustrate?
Answer: The 80/20 rule (Pareto principle)
The Pareto principle states that roughly 80% of effects come from 20% of causes, guiding prioritization to fix the most impactful issues first.