SRE Reliability Principles & Service Level Objectives 2 — Questions and Answers
Question 1: Why does Google's SRE model argue that having a separate SRE team, rather than embedding reliability work in development teams, is beneficial?
- A dedicated SRE team creates structural incentives to prioritize reliability — developers are motivated to ship features, while SREs are specifically accountable for stability and operational excellence (Correct answer)
- SREs have unique technical skills that cannot be learned by developers
- Having a separate team reduces the headcount needed in the development organization
- Regulatory requirements mandate that operations and development teams be separated
Correct answer: A dedicated SRE team creates structural incentives to prioritize reliability — developers are motivated to ship features, while SREs are specifically accountable for stability and operational excellence
Google's SRE model creates a team whose primary incentive and success metric is reliability, counterbalancing development teams whose primary incentive is feature velocity. The structural separation aligns incentives with the organization's reliability goals.
In traditional ops/dev separations, operations teams often had little influence over the code quality they had to run. Google's SRE model is different: SREs are software engineers who choose to work on operations, they co-own reliability with development teams, and they use the error budget mechanism to enforce a balance between feature velocity and reliability. Without this structural balance, organizational pressure typically favors feature velocity because features are visible and exciting while reliability is only noticed in its absence. The SRE team's explicit accountability for uptime creates a counterweight. Importantly, SREs spend at most 50% of their time on operational work (the rest on engineering) to prevent the role from becoming a traditional ops silo.
Question 2: An SLA promises customers 99.5% monthly availability with a 10% service credit for each additional 0.5% of downtime. The service experienced 5 hours of downtime in a 30-day month. Was the SLA breached?
- Yes — 99.5% of 43,200 minutes allows only 216 minutes (3.6 hours) of downtime; 5 hours (300 minutes) exceeds this (Correct answer)
- No — 5 hours is within the 99.5% threshold for a 30-day month
- Yes — any downtime in a month automatically breaches a 99.5% SLA
- No — SLA calculations exclude weekends and after-hours periods by default
Correct answer: Yes — 99.5% of 43,200 minutes allows only 216 minutes (3.6 hours) of downtime; 5 hours (300 minutes) exceeds this
30 days × 24 hours × 60 minutes = 43,200 minutes. 0.5% × 43,200 = 216 minutes (3.6 hours) allowed. 5 hours = 300 minutes, which exceeds 216 minutes, so the SLA is breached.
SLA breach calculation: A 99.5% availability SLA over 30 days means 0.5% of 43,200 minutes = 216 minutes of allowed downtime. 5 hours = 300 minutes. Since 300 > 216, the SLA is breached by 84 minutes. The breach amount is 84/43,200 ≈ 0.19% beyond the budget, which in this SLA's structure would trigger the first 10% credit tier. It's important to know the exact SLA window (calendar month, trailing 30 days), whether planned maintenance counts, and how downtime is measured (full outage vs. partial degradation). These specifics are defined in the SLA document and determine both breach assessment and credit calculations.
Question 3: What is the key difference between an SLO and an SLA?
- SLOs are internal targets that guide engineering decisions; SLAs are external contractual commitments with financial or legal consequences for breach (Correct answer)
- SLOs apply to latency only; SLAs apply to availability only
- SLOs are set by customers; SLAs are set by the engineering team
- SLOs measure real-time performance; SLAs measure historical performance over a quarter
Correct answer: SLOs are internal targets that guide engineering decisions; SLAs are external contractual commitments with financial or legal consequences for breach
SLOs are internal targets that teams use to drive reliability work and gate releases. SLAs are contracts with customers — if the SLA is breached, there are consequences (credits, contract termination). SLOs are typically set stricter than SLAs to provide a buffer.
The SLI/SLO/SLA hierarchy works as follows: SLIs are the raw measurements, SLOs are the targets for those measurements used internally by engineering and SRE teams, and SLAs are the external-facing commitments made to customers with consequences for non-compliance. Best practice is to set the SLO stricter than the SLA — e.g., internal SLO of 99.9% while the SLA commits to 99.5%. This gives the team a buffer: if the internal SLO is breached, the team escalates and acts before the SLA is at risk. Teams that set their SLO equal to their SLA have no warning before they owe customer credits. SLOs also guide error budget policies independently of legal commitments.
Question 4: A reliability review finds that a service achieves 99.97% availability — well above its 99.9% SLO. The SRE team proposes loosening the SLO to 99.95%. What is the BEST argument in favor of this change?
- The gap between actual reliability and the SLO target suggests the service is over-engineered for its current needs; loosening the SLO frees error budget for feature velocity without meaningfully impacting users (Correct answer)
- Loosening the SLO reduces engineering accountability and makes the team's job easier
- A looser SLO will attract fewer customer complaints because expectations are lower
- The SLO should be loosened to match the SLA to reduce confusion
Correct answer: The gap between actual reliability and the SLO target suggests the service is over-engineered for its current needs; loosening the SLO frees error budget for feature velocity without meaningfully impacting users
If a service consistently and significantly exceeds its SLO, it may be over-engineered relative to user needs, consuming engineering resources that could be used for features. Adjusting the SLO to reflect the actual user-acceptable threshold releases that budget.
One of SRE's key insights is that excessive reliability is as problematic as insufficient reliability. If a team maintains 99.97% availability when users only need 99.9%, the team has been spending engineering effort on unnecessary redundancy and over-cautious change management that could have been invested in new features. The SLO should reflect the actual minimum reliability users find acceptable — informed by user research, competitive analysis, and business impact assessments. Loosening the SLO from 99.97% to 99.95% (still above the hypothetical user need) creates more error budget for safe deployments, experiments, and feature work. The caveat: always validate the new SLO against real user impact data and customer commitments before changing it.
Question 5: Which of the following best describes 'toil' in the SRE context?
- Manual, repetitive, automatable operational work that scales linearly with service load and does not produce lasting improvement (Correct answer)
- Any work that is performed outside of business hours, including on-call rotations
- Technical debt accumulated from architectural shortcuts taken during initial development
- Customer support tickets that require engineering intervention to resolve
Correct answer: Manual, repetitive, automatable operational work that scales linearly with service load and does not produce lasting improvement
Toil is the specific category of work that is manual, repetitive, tactical (no enduring improvement), reactive, and scales proportionally with service growth — the opposite of engineering work that reduces future burden.
Google's SRE book defines toil with six properties: manual (requires human action), repetitive (done multiple times with no improvement), automatable (a computer could do it), tactical (reactive fire-fighting), devoid of enduring value (doing it doesn't improve the future state), and O(n) with service growth (as the service grows, the work grows proportionally). Examples: manually restarting a pod when its memory use exceeds a threshold, running the same SQL query weekly to generate a capacity report, manually approving routine deployments. The SRE principle is that toil should be capped at 50% of team time — if it exceeds this, the team is failing to engineer systemic improvements and is stuck in a maintenance treadmill. Reducing toil is a primary SRE objective.
Question 6: Why should SLOs be set based on what users actually need rather than on what the system can currently achieve?
- Setting SLOs based on current capability locks in the status quo and may over-invest in reliability that users don't value, while user-need-based SLOs create the right reliability incentives (Correct answer)
- User surveys are the only valid input for SLO setting because engineers cannot estimate reliability
- System-capability-based SLOs are always lower than user-need SLOs, leading to SLA breaches
- User-need-based SLOs are required by industry compliance frameworks like ISO 27001
Correct answer: Setting SLOs based on current capability locks in the status quo and may over-invest in reliability that users don't value, while user-need-based SLOs create the right reliability incentives
SLOs calibrated to current capability lock in technical debt and over-engineering simultaneously. User-need-based SLOs create a clear target: meet the minimum reliability users require, invest the rest in innovation.
Capability-based SLOs have two failure modes: if set at current high capability, they consume engineering effort maintaining unnecessary reliability; if set at current low capability, they legitimize poor service. User-need-based SLOs — determined through customer research, competitive benchmarking, and business impact analysis — set the correct target: the minimum reliability below which users churn, complain, or experience meaningful harm. Above that threshold, additional reliability investment has diminishing returns. The SRE framework creates space for teams to ask: 'What's the least reliable we can be while still satisfying users?' This is not a goal to reduce quality — it is a goal to invest reliability effort where it creates the most user value, and direct remaining capacity toward features.
SRE Reliability Principles & Service Level Objectives 2
Why does Google's SRE model argue that having a separate SRE team, rather than embedding reliability work in development teams, is beneficial?