alert fatigue

jargon

ops culturesecurity

The numbness that sets in when monitoring pages so often that real alerts drown among the false ones.

Every ignored alarm trains the on-call brain that alarms do not matter - right up until one does. The cure is ruthless: every alert must be actionable, or it gets tuned, aggregated, or deleted.

Alert fatigue is the failure mode where a monitoring system works exactly as designed and stops protecting anything. Every alert is technically correct, and there are far too many, so the humans receiving them learn to dismiss without reading. The system is healthy on paper; the response capability is gone.

It is a design problem rather than a discipline problem, which is the part organizations get wrong when they respond by telling analysts to be more careful. Attention is finite and habituation is automatic. A queue that produces hundreds of low-value notifications trains exactly the behaviour that will miss the one that mattered, and blaming the person for that outcome guarantees it recurs.

The fixes are unglamorous. Alert on conditions that require a decision rather than on facts that are merely true. Give every alert an owner and a documented action, and delete the ones that have neither. Track how often each rule fires and what fraction was actionable, then tune or remove the worst offenders. A short queue of alerts people actually read protects more than a comprehensive one they have learned to ignore.

Also known as: pager fatigue, alarm fatigue

All glossary entries