AML AML Model Deployment & MLOps 2 — Questions and Answers
Question 1: What is a feature store in an MLOps architecture?
- A cloud storage bucket for raw datasets
- A centralized repository for storing, sharing, and serving precomputed features for ML models (Correct answer)
- A tool for automated feature selection
- A replica of the training database used for inference
Correct answer: A centralized repository for storing, sharing, and serving precomputed features for ML models
Feature stores provide consistent, reusable features across training and serving pipelines, eliminating training-serving skew.
Question 2: What is the purpose of model versioning in MLOps?
- To label different datasets used to train a model
- To track distinct model iterations with metadata to enable rollback and comparison (Correct answer)
- To increment model complexity with each training run
- To compress model weights for faster inference
Correct answer: To track distinct model iterations with metadata to enable rollback and comparison
Model versioning logs each trained model artifact with its configuration, metrics, and data lineage to support governance and rollback.
Question 3: Which metric category is most critical when monitoring a deployed classification model in production?
- Training loss from the most recent training run
- Prediction confidence distribution and accuracy on live data (Correct answer)
- Number of API calls per second
- Model file size on disk
Correct answer: Prediction confidence distribution and accuracy on live data
Monitoring live prediction confidence and accuracy reveals model drift and performance degradation before it significantly impacts business outcomes.
Question 4: What is shadow deployment in machine learning operations?
- Running a model only during off-peak hours to save resources
- Running a new model in parallel with the production model without using its predictions for actual decisions (Correct answer)
- Deploying a model to a private server without public access
- Testing a model with synthetic data before live deployment
Correct answer: Running a new model in parallel with the production model without using its predictions for actual decisions
Shadow deployment routes live traffic to both the current and new model, logging new model outputs for evaluation without affecting end users.
Question 5: What is the primary purpose of model explainability tools like SHAP in production ML systems?
- To speed up model inference at scale
- To explain individual predictions and ensure model transparency for stakeholders and auditors (Correct answer)
- To compress model artifacts for faster deployment
- To automate feature engineering pipelines
Correct answer: To explain individual predictions and ensure model transparency for stakeholders and auditors
SHAP attributes each feature's contribution to individual predictions, supporting transparency and regulatory compliance in production.
Question 6: What is canary deployment in ML production systems?
- Testing new models in a completely isolated validation environment
- Gradually rolling out a new model to a small subset of users before a full release (Correct answer)
- Deploying a model only for internal QA testing
- Using a lightweight fallback model when the primary model fails
Correct answer: Gradually rolling out a new model to a small subset of users before a full release
Canary deployment incrementally increases traffic to a new model version, allowing real-world validation with minimal risk exposure.
AML AML Model Deployment & MLOps 2
What is a feature store in an MLOps architecture?