CAIC Software Development Lifecycle 2 — Questions and Answers
Question 1: Which SDLC phase is MOST critical for identifying AI model bias before a system goes to production?
- Requirements gathering
- Testing and validation (Correct answer)
- Deployment
- Maintenance
Correct answer: Testing and validation
Testing and validation is the phase where bias, fairness, and accuracy issues are systematically identified and corrected before release.
Question 2: In an AI project using Scrum, what artifact BEST captures the evolving data requirements for model training?
- Sprint burndown chart
- Product backlog (Correct answer)
- Definition of Done
- Velocity chart
Correct answer: Product backlog
The product backlog holds all prioritized work items, including data acquisition and preprocessing tasks, and is continuously refined.
Question 3: A team discovers that their AI model performs well in development but poorly in production. This is BEST described as:
- Underfitting
- Data leakage
- Train-serve skew (Correct answer)
- Overfitting
Correct answer: Train-serve skew
Train-serve skew occurs when the data distribution or feature engineering in production differs from the training environment.
Question 4: Which practice BEST supports continuous integration in an AI development pipeline?
- Manual model approval gates only
- Automated testing of data pipelines and model performance (Correct answer)
- Deploying directly to production after training
- Using a single shared development environment
Correct answer: Automated testing of data pipelines and model performance
Automated testing of data pipelines and model metrics ensures that each code or data change is validated before merging.
Question 5: When defining the 'Definition of Done' for an AI feature, which criterion is MOST relevant?
- The feature has been discussed in standup
- Model accuracy meets the agreed threshold on a held-out test set (Correct answer)
- The developer has completed their local testing
- The feature is merged to the main branch
Correct answer: Model accuracy meets the agreed threshold on a held-out test set
A measurable performance threshold on unseen data provides an objective, agreed criterion for completion of an AI feature.
Question 6: In the AI SDLC, 'data versioning' primarily serves to:
- Reduce cloud storage costs
- Enable reproducibility of experiments and audits (Correct answer)
- Speed up model inference
- Simplify feature engineering
Correct answer: Enable reproducibility of experiments and audits
Data versioning allows teams to reproduce any past experiment by associating model versions with the exact dataset snapshot used.
Question 7: A company wants to adopt MLOps. Which SDLC stage does MLOps MOST directly extend?
- Requirements analysis
- Design
- Deployment and operations (Correct answer)
- Feasibility study
Correct answer: Deployment and operations
MLOps applies DevOps principles to the deployment and ongoing operations of ML models, automating retraining, monitoring, and delivery.
CAIC Software Development Lifecycle 2
Which SDLC phase is MOST critical for identifying AI model bias before a system goes to production?