CAE Agile Metrics & Reporting 3 — Questions and Answers
Question 1: Which agile metric would MOST directly help a team identify recurring process inefficiencies across multiple sprints?
- Sprint velocity
- Retrospective action item completion rate (Correct answer)
- Number of user stories in the backlog
- Stakeholder attendance at sprint reviews
Correct answer: Retrospective action item completion rate
Tracking whether retrospective action items are actually completed shows if the team is systematically addressing and resolving recurring process issues.
Question 2: In a Kanban system, what does a high 'cycle time standard deviation' tell you about the team's process?
- The team completes work faster than average
- Work item delivery is unpredictable and inconsistent (Correct answer)
- The team is meeting all service level expectations
- Story point estimates are well-calibrated
Correct answer: Work item delivery is unpredictable and inconsistent
A high standard deviation in cycle time means delivery times vary widely, indicating an unpredictable process that is difficult to forecast.
Question 3: A team wants to use metrics to forecast when a set of backlog items will be complete. Which technique is MOST appropriate?
- Monte Carlo simulation using historical throughput data (Correct answer)
- Multiplying average velocity by the number of sprints remaining
- Asking the product owner to set a firm deadline
- Counting total story points and dividing by team size
Correct answer: Monte Carlo simulation using historical throughput data
Monte Carlo simulation uses historical throughput data to generate probabilistic delivery forecasts, accounting for natural variability in team performance.
Question 4: Which metric is considered a 'vanity metric' in agile contexts because it rarely correlates with actual business outcomes?
- Customer net promoter score (NPS)
- Number of features shipped per quarter (Correct answer)
- Defect escape rate to production
- Time to market for new features
Correct answer: Number of features shipped per quarter
Feature count is a vanity metric because shipping many features doesn't mean customers find value in them or that business goals are achieved.
Question 5: During a sprint review, a stakeholder asks why the sprint burndown chart showed completed work dropping suddenly on the last day. What is the MOST likely cause?
- The team deleted incomplete stories
- Stories were only marked 'done' at the end of the sprint rather than continuously (Correct answer)
- The tool auto-reset the chart at midnight
- Team members forgot to attend the daily scrum
Correct answer: Stories were only marked 'done' at the end of the sprint rather than continuously
A 'cliff' pattern on a burndown chart typically indicates that the team batched their 'done' updates instead of closing stories incrementally.
Question 6: What does an agile team's 'escaped defect rate' measure?
- The number of bugs found during sprint testing
- The percentage of defects that reach production despite team testing (Correct answer)
- The ratio of rework hours to total development hours
- How quickly the QA team finds and closes bugs
Correct answer: The percentage of defects that reach production despite team testing
Escaped defect rate tracks how many defects slip past the team's testing and are discovered by users in production, indicating testing effectiveness.
Question 7: A product owner is preparing an agile metrics dashboard for senior leadership. Which combination of metrics provides the MOST balanced view of team health and delivery?
- Daily stand-up duration, sprint length, and team headcount
- Velocity, customer satisfaction, and defect escape rate (Correct answer)
- Number of meetings held, lines of code written, and backlog size
- Sprint name, release date, and Scrum master name
Correct answer: Velocity, customer satisfaction, and defect escape rate
Combining velocity (output), customer satisfaction (value), and defect escape rate (quality) gives a balanced view of productivity, effectiveness, and product quality.
CAE Agile Metrics & Reporting 3
Which agile metric would MOST directly help a team identify recurring process inefficiencies across multiple sprints?