CAP CAP Model Deployment & Lifecycle Management 1 — Questions and Answers
Question 1: Which deployment pattern runs the new model alongside the old model and gradually shifts traffic to the new one?
- Big-bang deployment
- Canary / gradual rollout (Correct answer)
- Rollback deployment
- Blue-green deployment
Correct answer: Canary / gradual rollout
Canary deployment incrementally shifts a small percentage of traffic to the new model, limiting risk before full cutover.
Question 2: Model monitoring in production should track which of the following to detect data drift?
- Server CPU usage only
- Distribution of input features over time (Correct answer)
- Number of database tables
- Color scheme of dashboards
Correct answer: Distribution of input features over time
Monitoring the statistical distribution of input features reveals data drift, which can degrade model performance over time.
Question 3: What does 'model staleness' mean in the CAP lifecycle management context?
- The model has too many parameters
- Model performance has degraded because the real-world environment has changed (Correct answer)
- The model was never deployed
- The model uses outdated visualization libraries
Correct answer: Model performance has degraded because the real-world environment has changed
Model staleness occurs when concept drift causes a previously accurate model to become less effective as the underlying patterns change.
Question 4: Which artifact MUST be version-controlled alongside model code to ensure reproducibility?
- Dashboard color themes
- Training data schema and model hyperparameters (Correct answer)
- Email correspondence
- Project management ticket IDs
Correct answer: Training data schema and model hyperparameters
Reproducibility requires version-controlling the training data schema, hyperparameters, and code so the exact model state can be recreated.
Question 5: A/B testing in model deployment is used to:
- Train two models on the same data simultaneously
- Compare two model versions using live traffic and business metrics (Correct answer)
- Replace unit testing
- Avoid data collection
Correct answer: Compare two model versions using live traffic and business metrics
A/B testing exposes different user groups to each model version and measures real business outcomes to determine the superior model.
Question 6: Which document formally describes a model's intended use, limitations, and performance across subgroups?
- Data dictionary
- Model card (Correct answer)
- Entity-relationship diagram
- Sprint retrospective
Correct answer: Model card
A model card is a standardized documentation artifact that captures intended use cases, performance benchmarks, and fairness considerations.
CAP CAP Model Deployment & Lifecycle Management 1
Which deployment pattern runs the new model alongside the old model and gradually shifts traffic to the new one?