CAP CAP Model Deployment & Lifecycle Management 2 — Questions and Answers
Question 1: What is the primary purpose of a model registry in an MLOps pipeline?
- To store raw input data
- To centrally track model versions, metadata, and deployment status (Correct answer)
- To replace the data warehouse
- To schedule data ingestion jobs
Correct answer: To centrally track model versions, metadata, and deployment status
A model registry provides a single source of truth for model versions, lineage, and stage transitions (staging, production, archived).
Question 2: Which metric directly measures concept drift in a deployed classification model?
- Server response time
- Accuracy or AUC degradation over time on live labels (Correct answer)
- Number of API calls per minute
- Disk I/O throughput
Correct answer: Accuracy or AUC degradation over time on live labels
Tracking predictive performance metrics (accuracy, AUC) on incoming labeled data over time directly reveals concept drift.
Question 3: Shadow mode deployment means:
- Deploying a model only at night
- Running a new model in parallel without using its outputs for decisions (Correct answer)
- Hiding model predictions from users
- Encrypting model weights
Correct answer: Running a new model in parallel without using its outputs for decisions
In shadow mode, the new model receives real inputs and generates predictions, but only the existing model's outputs drive actual decisions.
Question 4: Which trigger should initiate a model retraining cycle?
- A scheduled calendar reminder regardless of performance
- Statistically significant performance degradation detected by monitoring (Correct answer)
- A request from the marketing team for a new dashboard
- Completion of a quarterly budget cycle
Correct answer: Statistically significant performance degradation detected by monitoring
Retraining should be triggered by evidence of performance degradation, not arbitrary schedules, to ensure resources are spent when needed.
Question 5: Feature stores in production analytics primarily serve to:
- Store raw unprocessed data permanently
- Provide consistent, reusable feature definitions across training and serving (Correct answer)
- Replace relational databases
- Generate model visualizations
Correct answer: Provide consistent, reusable feature definitions across training and serving
Feature stores centralize computed feature logic so training and serving pipelines use identical transformations, preventing training-serving skew.
Question 6: Which practice reduces training-serving skew in deployed models?
- Using different preprocessing code in training and inference pipelines
- Sharing a single feature transformation pipeline for both training and serving (Correct answer)
- Ignoring validation datasets
- Retraining on test data
Correct answer: Sharing a single feature transformation pipeline for both training and serving
Sharing the same feature transformation pipeline ensures training and serving see identically processed features, eliminating skew.
CAP CAP Model Deployment & Lifecycle Management 2
What is the primary purpose of a model registry in an MLOps pipeline?