AIF-C01 Machine Learning Operations (MLOps) 2 — Questions and Answers
Question 1: What is the key difference between a shadow deployment and a canary deployment?
- Shadow deployment sends real traffic to the new model but discards its predictions (Correct answer)
- Shadow deployment routes all traffic to the new model without user awareness
- Canary deployment tests only synthetic data while shadow uses real data
- There is no meaningful difference between the two strategies
Correct answer: Shadow deployment sends real traffic to the new model but discards its predictions
In shadow deployment, real production requests are mirrored to the new model, but only the existing model's responses are served to users, allowing safe comparison without risk.
Question 2: Amazon SageMaker Feature Store serves what primary purpose in an MLOps workflow?
- Storing pre-trained model weights for foundation models
- Providing a centralized repository for storing, sharing, and reusing ML features (Correct answer)
- Managing hyperparameter configurations across training runs
- Archiving historical model inference logs
Correct answer: Providing a centralized repository for storing, sharing, and reusing ML features
SageMaker Feature Store provides a centralized, reusable repository of ML features that ensures consistency between training and inference, and enables feature sharing across teams.
Question 3: What is concept drift in the context of ML model monitoring?
- The gradual decrease in model accuracy due to infrastructure issues
- A change in the underlying relationship between input features and the target variable (Correct answer)
- When the input feature distribution shifts away from the training distribution
- When a model is retrained with outdated data
Correct answer: A change in the underlying relationship between input features and the target variable
Concept drift occurs when the statistical relationship between inputs and outputs changes over time, meaning the patterns the model learned are no longer valid.
Question 4: In Amazon SageMaker, what is a 'production variant' used for?
- Defining different IAM permission sets for model endpoints
- Splitting traffic between multiple model versions on a single endpoint (Correct answer)
- Specifying different training configurations for the same algorithm
- Creating duplicate backups of a model artifact in S3
Correct answer: Splitting traffic between multiple model versions on a single endpoint
A production variant allows you to deploy multiple model versions behind a single SageMaker endpoint and control what percentage of traffic each variant receives.
Question 5: Which SageMaker capability automatically detects bias in training data and model predictions?
- SageMaker Debugger
- SageMaker Clarify (Correct answer)
- SageMaker Data Wrangler
- SageMaker Experiments
Correct answer: SageMaker Clarify
SageMaker Clarify provides bias detection for both pre-training data and post-training model predictions, as well as feature importance and model explainability reports.
Question 6: What triggers are commonly used to initiate automatic model retraining in an MLOps pipeline?
- Only manual approvals from a data scientist
- New data availability, performance degradation, or scheduled time intervals (Correct answer)
- Changes to the AWS region or availability zone configuration
- Updates to the SageMaker SDK version
Correct answer: New data availability, performance degradation, or scheduled time intervals
Automated retraining is typically triggered by data drift alerts, model performance falling below a threshold, scheduled time-based intervals, or new labeled data becoming available.
Question 7: What is the role of SageMaker Experiments in an MLOps workflow?
- Running A/B tests on live production endpoints
- Tracking, organizing, and comparing multiple ML training runs and their metrics (Correct answer)
- Automatically deploying the best-performing model to production
- Managing user access controls for SageMaker notebooks
Correct answer: Tracking, organizing, and comparing multiple ML training runs and their metrics
SageMaker Experiments records the inputs, parameters, configurations, and results of every training run, enabling teams to reproduce and compare experiments systematically.
AIF-C01 Machine Learning Operations (MLOps) 2
What is the key difference between a shadow deployment and a canary deployment?