DOL Metrics and KPIs 3 — Questions and Answers
Question 1: What does a 'toil' metric measure in the SRE and DevOps context?
- Total code lines written per sprint
- Manual, repetitive operational work that scales with service growth (Correct answer)
- Number of on-call escalations per engineer
- Time spent in planning meetings
Correct answer: Manual, repetitive operational work that scales with service growth
Toil is manual, repetitive, automatable work that grows proportionally with service traffic, and reducing it is a key SRE objective.
Question 2: A DevOps leader sees high deployment frequency but rising MTTR. What does this combination most likely indicate?
- The team is shipping stable, well-tested code
- Deployments are frequent but insufficiently tested, causing more incidents (Correct answer)
- MTTR improvements lag deployment frequency by design
- The observability stack is misconfigured
Correct answer: Deployments are frequent but insufficiently tested, causing more incidents
Deploying often while MTTR rises suggests quality gates are weak — incidents are happening more often and taking longer to resolve.
Question 3: Which leading indicator would a DevOps leader monitor to predict future deployment failures before they happen?
- Post-release MTTR
- Test coverage trends and flaky test rate (Correct answer)
- Customer churn rate
- Number of support tickets closed
Correct answer: Test coverage trends and flaky test rate
Declining test coverage or rising flaky test rates are leading indicators that code quality is degrading and production failures are more likely.
Question 4: What is the relationship between 'lead time for changes' and team cognitive load in DevOps?
- Longer lead time always means less cognitive load for engineers
- Long lead times often indicate large batch sizes that increase cognitive load and risk (Correct answer)
- Lead time and cognitive load are unrelated metrics
- Short lead times increase cognitive load by requiring constant context switching
Correct answer: Long lead times often indicate large batch sizes that increase cognitive load and risk
Long lead times typically reflect large, complex batches of change that are harder to understand, review, and roll back — all of which raise cognitive load.
Question 5: A DevOps team uses 'cycle time' as a KPI. What does cycle time measure?
- Time from first commit to production deployment
- Time from work item started to production delivery (Correct answer)
- The duration of a single CI build
- Time from production incident to post-mortem completion
Correct answer: Time from work item started to production delivery
Cycle time measures from when work is actively begun (picked up from backlog) to when it reaches production, reflecting end-to-end delivery speed.
Question 6: Which statistical measure is most appropriate for analyzing latency distributions in production systems?
- Mean (average) latency
- Percentile metrics such as P95 and P99 (Correct answer)
- Mode of all response times
- Standard deviation alone
Correct answer: Percentile metrics such as P95 and P99
Percentile metrics like P95/P99 capture tail latency that affects the worst-served users, which averages hide by being pulled down by fast requests.
Question 7: How does 'value stream mapping' help a DevOps leader identify metric improvement opportunities?
- It shows which engineers are most productive
- It visualizes end-to-end workflow to expose bottlenecks and wait times between steps (Correct answer)
- It generates automated performance test reports
- It calculates infrastructure cost per deployment
Correct answer: It visualizes end-to-end workflow to expose bottlenecks and wait times between steps
Value stream mapping charts every step from idea to production delivery, making handoff delays and process bottlenecks visible so teams know where to focus improvement efforts.
DOL Metrics and KPIs 3
What does a 'toil' metric measure in the SRE and DevOps context?