AI AI Engineer: MLOps and Model Deployment 2

Question 1

What is the role of an A/B test in an MLOps deployment workflow?

Accepted Answer

To statistically compare two model versions using live traffic

Answer

A/B testing splits live traffic between two model versions and uses statistical analysis to determine which performs better.

Question 2

Which metric is most useful for detecting concept drift in a classification model?

Accepted Answer

Prediction distribution shift over time

Answer

Monitoring the distribution of model predictions over time reveals concept drift because the model's output distribution changes as real-world patterns evolve.

Question 3

What is 'shadow mode' deployment in MLOps?

Accepted Answer

Sending production traffic to a new model without using its outputs to serve users

Answer

Shadow mode runs a new model on real traffic in parallel with the live model, comparing outputs without impacting users.

Question 4

Which tool is commonly used for experiment tracking in ML, allowing teams to log parameters, metrics, and artifacts?

Accepted Answer

MLflow

Answer

MLflow is an open-source platform for tracking experiments, packaging code, and managing model lifecycle.

Question 5

What is 'online serving' in ML deployment as opposed to 'batch inference'?

Accepted Answer

Generating predictions in real time for individual requests

Answer

Online serving generates low-latency predictions on demand for individual requests, unlike batch inference which processes large datasets periodically.

AI - Engineer Practice Test

AI - Engineer Practice Test

AI AI Engineer: MLOps and Model Deployment 2