CAIC System Architecture & Design 3 — Questions and Answers
Question 1: A company wants to serve multiple AI use cases (recommendation, fraud detection, NLP) from a single platform. Which architectural approach best enables this?
- Separate siloed stacks per use case
- An AI platform with shared infrastructure, common SDKs, and modular ML pipelines (Correct answer)
- A single mega-model that handles all tasks
- Manual deployment scripts maintained per team
Correct answer: An AI platform with shared infrastructure, common SDKs, and modular ML pipelines
A unified AI platform with shared infrastructure and tooling reduces duplication, standardizes best practices, and enables teams to focus on their specific models rather than platform concerns.
Question 2: What distinguishes an online learning system from a batch retraining system in AI architecture?
- Online learning uses GPUs; batch retraining uses CPUs
- Online learning updates the model incrementally with each new data point or mini-batch in production (Correct answer)
- Online learning requires labeled data; batch retraining uses unlabeled data
- Online learning is only suitable for classification tasks
Correct answer: Online learning updates the model incrementally with each new data point or mini-batch in production
Online learning continuously updates model parameters using a stream of incoming examples, allowing the model to adapt to distribution shifts without full retraining cycles.
Question 3: Which pattern addresses the challenge of serving recommendations to millions of users with sub-100ms latency?
- Running full neural network inference per request in real time
- Two-stage retrieval: fast approximate nearest-neighbor retrieval followed by a lightweight re-ranker (Correct answer)
- Storing all possible recommendations pre-computed in a relational database
- Using synchronous blocking API calls to a remote model server
Correct answer: Two-stage retrieval: fast approximate nearest-neighbor retrieval followed by a lightweight re-ranker
Two-stage systems use efficient ANN search to narrow candidates quickly, then apply a more accurate but lightweight model to rank only those candidates, achieving both speed and quality.
Question 4: In a microservices-based AI system, what is the primary risk of tight coupling between the prediction service and the data preprocessing service?
- Increased network throughput between services
- Changes to preprocessing logic can break the prediction service without clear version boundaries (Correct answer)
- Reduced container image sizes for both services
- Simplified monitoring with a single log stream
Correct answer: Changes to preprocessing logic can break the prediction service without clear version boundaries
Tight coupling means that changes in one service's interface or behavior directly impact dependent services, violating the independent deployability principle of microservices.
Question 5: Which database type is most architecturally suited as a vector store for similarity search in a RAG (Retrieval-Augmented Generation) system?
- Relational database with B-tree indexes
- Time-series database optimized for write throughput
- Purpose-built vector database with ANN index support like Pinecone or Weaviate (Correct answer)
- Key-value store with consistent hashing
Correct answer: Purpose-built vector database with ANN index support like Pinecone or Weaviate
Purpose-built vector databases use specialized ANN indexes (HNSW, IVF) that enable efficient high-dimensional similarity search, which is the core operation in RAG retrieval.
Question 6: When an AI system must maintain a 99.99% SLA, which architectural pattern is essential for the model serving layer?
- Single high-performance server with premium hardware
- Active-active multi-region deployment with circuit breakers and fallback logic (Correct answer)
- Scheduled maintenance windows for model updates
- Synchronous retraining triggered by prediction failures
Correct answer: Active-active multi-region deployment with circuit breakers and fallback logic
A 99.99% SLA (~52 minutes downtime/year) requires active-active redundancy across failure domains plus circuit breakers to degrade gracefully rather than fail completely.
Question 7: What is the primary purpose of a model registry in an MLOps architecture?
- Storing raw training datasets for reproducibility
- Centralizing model artifact versioning, metadata, and stage promotion across the model lifecycle (Correct answer)
- Scheduling distributed training jobs across GPU clusters
- Monitoring live model performance and triggering alerts
Correct answer: Centralizing model artifact versioning, metadata, and stage promotion across the model lifecycle
A model registry tracks model versions, associated metadata (metrics, lineage, parameters), and manages promotion stages (staging → production) to enable governed model lifecycle management.
CAIC System Architecture & Design 3
A company wants to serve multiple AI use cases (recommendation, fraud detection, NLP) from a single platform.
Which architectural approach best enables this?