SRE Chaos Engineering & Resilience 5 — Questions and Answers
Question 1: What is the 'Swiss cheese model' as it applies to SRE resilience design?
- A caching strategy that layers multiple cache tiers to prevent origin overload
- A layered defense model where multiple imperfect safeguards combine so no single failure causes a full outage (Correct answer)
- A deployment model that uses multiple cloud providers simultaneously
- A testing framework that validates each microservice in isolation before integration
Correct answer: A layered defense model where multiple imperfect safeguards combine so no single failure causes a full outage
The Swiss cheese model illustrates that layered defenses each have 'holes' (weaknesses), but when stacked, the holes rarely align, preventing complete system failure.
Question 2: What distinguishes a 'chaos engineering experiment' from random destructive testing?
- Chaos experiments always occur in production; random tests are confined to staging
- Chaos experiments have defined hypotheses, controlled scope, and measurable outcomes; random tests do not (Correct answer)
- Chaos experiments use automated tooling; random tests are always performed manually
- Chaos experiments target only stateless services; random tests target databases
Correct answer: Chaos experiments have defined hypotheses, controlled scope, and measurable outcomes; random tests do not
Legitimate chaos engineering is disciplined and scientific — it starts with a hypothesis, limits blast radius, and measures specific outcomes to learn from each experiment.
Question 3: What is 'dependency mapping' and why is it a prerequisite for effective chaos experiments?
- Tracking which teams own which microservices to assign on-call rotations
- Identifying all upstream and downstream service relationships so experiments target the right failure points (Correct answer)
- Documenting API contracts between services in an OpenAPI specification
- Mapping database schemas to application models for ORM configuration
Correct answer: Identifying all upstream and downstream service relationships so experiments target the right failure points
Without knowing service dependencies, you cannot predict the blast radius of an experiment or design meaningful hypotheses about failure propagation.
Question 4: Which type of chaos experiment specifically validates that a service's circuit breaker trips correctly under sustained error conditions?
- CPU saturation injection
- Dependency failure simulation (Correct answer)
- DNS resolution failure
- Disk I/O throttling
Correct answer: Dependency failure simulation
Simulating a dependency returning errors or timing out verifies that the calling service's circuit breaker opens correctly and stops cascading the failures downstream.
Question 5: What does 'graceful degradation' mean in resilience engineering?
- Shutting down non-critical features during high load so core functionality remains available (Correct answer)
- Gradually rolling back a bad deployment using percentage-based traffic shifting
- Reducing infrastructure costs by terminating idle instances automatically
- Archiving old data to cold storage to keep primary databases performant
Correct answer: Shutting down non-critical features during high load so core functionality remains available
Graceful degradation ensures that when parts of a system fail, the most critical features keep working even if secondary features become unavailable.
Question 6: In chaos engineering maturity models, what characterizes a 'Level 3' or advanced chaos practice?
- Running chaos experiments once per quarter in staging only
- Fully automated continuous chaos integrated into CI/CD with real-time abort controls and self-healing validation (Correct answer)
- Manually injecting failures during scheduled maintenance windows with team approval
- Chaos experiments limited to network-layer faults in development environments
Correct answer: Fully automated continuous chaos integrated into CI/CD with real-time abort controls and self-healing validation
Mature chaos programs run continuously and automatically as part of delivery pipelines, with automated safeguards and validation of self-healing mechanisms.
Question 7: What is the key difference between 'high availability' (HA) and 'fault tolerance' in system design?
- HA focuses on data durability; fault tolerance focuses on request speed
- HA minimizes downtime through redundancy and failover; fault tolerance allows a system to continue operating correctly even during active failures (Correct answer)
- HA is achieved through geographic distribution; fault tolerance requires on-premises hardware only
- HA applies only to databases; fault tolerance applies only to compute layers
Correct answer: HA minimizes downtime through redundancy and failover; fault tolerance allows a system to continue operating correctly even during active failures
HA aims to reduce outage duration via fast failover, while fault tolerance means the system continues operating (possibly at reduced capacity) without any perceptible disruption even while a component is failing.
SRE Chaos Engineering & Resilience 5
What is the 'Swiss cheese model' as it applies to SRE resilience design?