CAIC Software Development Lifecycle 3 — Questions and Answers
Question 1: Which type of technical debt is UNIQUE to AI systems compared to traditional software?
- Undocumented APIs
- Dependency on stale training data (Correct answer)
- Missing unit tests
- Hardcoded configuration values
Correct answer: Dependency on stale training data
AI systems accumulate data debt when training datasets become stale, causing model performance to degrade as the real world changes.
Question 2: During sprint retrospective, a team notes that model retraining takes too long. The BEST solution within an Agile SDLC is:
- Skip retraining in future sprints
- Add automated retraining pipelines as backlog items (Correct answer)
- Move to a waterfall model
- Reduce the size of the training dataset permanently
Correct answer: Add automated retraining pipelines as backlog items
Creating backlog items for pipeline automation is the Agile approach to addressing process inefficiencies iteratively.
Question 3: A feature store is BEST described as:
- A repository for storing trained model weights
- A centralized system for managing and sharing ML features across teams (Correct answer)
- A bug tracking system for AI projects
- A cloud service for deploying models
Correct answer: A centralized system for managing and sharing ML features across teams
A feature store centralizes the computation, storage, and serving of ML features, ensuring consistency between training and inference.
Question 4: In an AI project, a 'shadow mode' deployment is used to:
- Hide model logic from end users
- Run a new model in parallel with the old one without affecting users (Correct answer)
- Test the model on synthetic data only
- Deploy a model to a subset of users for A/B testing
Correct answer: Run a new model in parallel with the old one without affecting users
Shadow mode runs the new model alongside the production model, capturing its outputs for evaluation without impacting user experience.
Question 5: Which gate is MOST appropriate to include in an AI CI/CD pipeline to prevent data quality regressions?
- Code style linting check
- Automated data validation and schema checks (Correct answer)
- Manual stakeholder sign-off
- UI smoke testing
Correct answer: Automated data validation and schema checks
Automated data validation checks enforce schema, null rates, and distribution bounds so pipeline failures catch data quality issues immediately.
Question 6: When managing an AI project with regulatory compliance requirements, which SDLC artifact is MOST important for an audit?
- Daily standup notes
- Model card documenting training data, performance, and limitations (Correct answer)
- Sprint velocity history
- UX wireframes
Correct answer: Model card documenting training data, performance, and limitations
A model card provides structured documentation regulators need: training data provenance, evaluation results, intended use, and known limitations.
Question 7: In the context of AI SDLC, 'concept drift' requires which operational response?
- Redesigning the user interface
- Monitoring model performance metrics and triggering retraining (Correct answer)
- Migrating to a new cloud provider
- Refactoring the codebase to remove legacy modules
Correct answer: Monitoring model performance metrics and triggering retraining
Concept drift means the statistical relationship between inputs and outputs has changed, requiring performance monitoring and model retraining.
CAIC Software Development Lifecycle 3
Which type of technical debt is UNIQUE to AI systems compared to traditional software?