Quality Assurance & Improvement Flashcards
7 cards from real AML practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 Quality Assurance & Improvement flashcards as text
Disparate impact in a ML model is formally defined as:
Answer: The ratio of positive outcome rates between a protected group and a reference group being below 0.8
The 4/5ths (80%) rule defines disparate impact as occurring when the selection rate for a protected group is less than 80% of the highest-performing group.
Equalized odds as a fairness constraint requires that a model:
Answer: Achieve equal true positive and false positive rates across groups
Equalized odds requires both true positive rate (sensitivity) and false positive rate to be equal across protected attribute groups.
SHAP (SHapley Additive exPlanations) values are guaranteed to satisfy which key property that makes them useful for QA?
Answer: Efficiency: SHAP values for all features sum to the difference between the prediction and the global mean
The efficiency axiom ensures SHAP values fully decompose each prediction, making them a complete local explanation that sums to the prediction gap from baseline.
When performing a fairness audit, you discover that a model achieves demographic parity but not equalized odds. This means:
Answer: Equal prediction rates across groups mask unequal error rates between groups
Demographic parity only checks if positive prediction rates are equal, while equalized odds reveals whether the model makes different types of errors for different groups.
In a model card for a production ML system, the 'Intended Use' section primarily serves to:
Answer: Define the scope within which the model's QA guarantees hold
The Intended Use section documents the conditions under which the model was validated, alerting users when deployment context exceeds QA coverage.
Counterfactual explanations in XAI are most useful for ML QA because they:
Answer: Reveal the minimum input change needed to flip a model's decision
Counterfactuals show which feature perturbations would change an outcome, helping QA teams identify decision boundaries and potential unfairness.
Which technique helps detect if a model has learned spurious correlations by testing it on out-of-distribution examples with controlled variations?
Answer: Behavioral testing with invariance and directional expectation tests (CheckList)
CheckList-style behavioral testing defines expected model behaviors under transformations (invariance) and targeted perturbations (direction tests) to expose learned shortcuts.