AML Project Planning & Execution 2 — Questions and Answers
Question 1: During the scoping phase of an ML project, a team discovers the labeled dataset has only 500 samples for a 10-class classification task. What is the MOST appropriate first response?
- Proceed with training and apply heavy regularization
- Re-evaluate feasibility and explore data augmentation or semi-supervised approaches (Correct answer)
- Switch immediately to a simpler binary classification task
- Deploy a pre-trained model without any fine-tuning
Correct answer: Re-evaluate feasibility and explore data augmentation or semi-supervised approaches
Re-evaluating feasibility and exploring augmentation or semi-supervised learning addresses the core data scarcity problem before committing to a training approach.
Question 2: A stakeholder requests a real-time fraud detection system with sub-10ms latency. Which planning consideration is MOST critical to address early?
- Choosing between gradient boosting and neural networks
- Defining the latency budget across all pipeline components including inference, pre-processing, and I/O (Correct answer)
- Selecting a cloud provider for model hosting
- Determining the number of training epochs
Correct answer: Defining the latency budget across all pipeline components including inference, pre-processing, and I/O
End-to-end latency budgeting must account for all pipeline stages, not just model inference, and must be validated early to avoid architectural rework.
Question 3: Which artifact BEST serves as the single source of truth for tracking ML experiment reproducibility across a team?
- A shared Google Doc with hyperparameter tables
- An MLflow or similar experiment tracking registry logging code version, data hash, params, and metrics (Correct answer)
- Slack messages with attached screenshots of training curves
- A README file updated manually by each team member
Correct answer: An MLflow or similar experiment tracking registry logging code version, data hash, params, and metrics
Experiment tracking platforms like MLflow capture code version, data lineage, hyperparameters, and metrics in a queryable, reproducible format.
Question 4: A project manager wants to estimate compute costs for training a large language model. Which factor has the GREATEST impact on total cost?
- The number of evaluation metrics used
- Model parameter count, dataset size, and number of training steps combined (Correct answer)
- The programming language used for the training script
- The number of stakeholder review meetings scheduled
Correct answer: Model parameter count, dataset size, and number of training steps combined
Compute cost scales with the product of model size, data volume, and training steps, making these the dominant cost drivers.
Question 5: When defining the success criteria for an ML project, what is the PRIMARY risk of using accuracy as the sole metric for an imbalanced dataset?
- Accuracy is too slow to compute at scale
- A model predicting only the majority class can achieve high accuracy while failing on the minority class (Correct answer)
- Accuracy cannot be used with multi-class problems
- Stakeholders are unfamiliar with the accuracy metric
Correct answer: A model predicting only the majority class can achieve high accuracy while failing on the minority class
On imbalanced datasets, a trivial majority-class predictor can achieve misleadingly high accuracy, masking complete failure on the minority class.
Question 6: In ML project execution, what does 'data versioning' primarily help prevent?
- Overfitting during model training
- Silent dataset drift causing irreproducible experiments when the underlying data changes (Correct answer)
- Slow SQL queries on the feature store
- High memory usage during batch inference
Correct answer: Silent dataset drift causing irreproducible experiments when the underlying data changes
Data versioning creates immutable snapshots so that experiments reference the exact dataset used, preventing silent reproducibility failures when source data is updated.
Question 7: A team is planning sprint tasks for an ML project. Which task type should be placed in the EARLIEST sprint to reduce project risk?
- Writing the final model documentation
- Building a baseline end-to-end pipeline that touches all system components (Correct answer)
- Designing the production UI for model outputs
- Scheduling the external model audit
Correct answer: Building a baseline end-to-end pipeline that touches all system components
An early end-to-end baseline pipeline exposes integration risks, latency issues, and data quality problems before significant investment is made in modeling.
AML Project Planning & Execution 2
During the scoping phase of an ML project, a team discovers the labeled dataset has only 500 samples for a 10-class classification task.
What is the MOST appropriate first response?