SRE Alerting Strategy and Noise Reduction

Question 1

What is 'alert fatigue' in the context of SRE?

Accepted Answer

A state where on-call engineers become desensitized to alerts due to excessive or low-quality notifications

Answer

A monitoring system failure caused by too many simultaneous metrics being tracked

Answer

The gradual degradation of alert threshold accuracy over time as services scale

Answer

An architectural pattern where alerts are batched to reduce notification frequency

Question 2

Which alerting philosophy does Google SRE recommend as a best practice for page-worthy alerts?

Accepted Answer

Alert on symptoms that directly indicate user-visible impact

Answer

Alert on causes so engineers understand the root issue immediately

Answer

Alert on any metric that deviates more than two standard deviations from baseline

Answer

Alert on resource saturation before symptoms appear to enable proactive response

Question 3

What is a 'dead man's switch' (watchdog) alert in SRE monitoring?

Accepted Answer

An alert that fires when it STOPS receiving expected periodic signals from a system

Answer

An alert that fires when a critical service exceeds its error budget

Answer

An automated rollback trigger activated when a deployment health check fails

Answer

An alert generated when a primary monitoring system loses connection to a secondary

Question 4

In SRE alerting, what does 'precision' of an alert measure?

Accepted Answer

The proportion of alert firings that correspond to actual incidents requiring action

Answer

The percentage of real incidents that triggered at least one alert

Answer

The average time between when an alert fires and when an engineer acknowledges it

Answer

The accuracy of the alert threshold relative to the SLO violation boundary

Question 5

What is the primary distinction between a 'page' alert and a 'ticket' alert in SRE practice?

Accepted Answer

Page alerts require immediate human action; ticket alerts can be addressed during business hours

Answer

Page alerts go to the primary on-call engineer; ticket alerts go to the team lead

Answer

Page alerts are triggered by SLO violations; ticket alerts are triggered by infrastructure metrics

Answer

Page alerts use push notifications; ticket alerts use email-based delivery systems

Question 6

What is 'alert flapping' and why is it problematic?

Accepted Answer

When an alert repeatedly transitions between firing and resolved states in a short time window

Answer

When an alert fires simultaneously across multiple monitoring regions, creating duplicate notifications

Answer

When alert thresholds are automatically adjusted by ML systems without human approval

Answer

When an alert's notification channel is misconfigured and messages fail to deliver

Question 7

Which of the following is a key indicator that an alert should be removed or tuned down in severity?

Accepted Answer

The alert consistently fires and resolves without any engineer taking action

Answer

The alert fires more than once per quarter during business hours

Answer

The alert requires more than 15 minutes of investigation before resolution

Answer

The alert fires during deployments but not during steady-state operations