it's always DNS

networkingops culture

The troubleshooting proverb that whatever the symptom, the root cause will turn out to be DNS.

Half joke, half statistics: name resolution touches everything, fails in creative ways, and hides behind symptoms that look like anything else. The haiku version is canon: it's not DNS - there's no way it's DNS - it was DNS.

It's always DNS is the standard joke of incident response, deployed the moment a problem is inexplicable, and its persistence is diagnostic rather than merely funny. DNS genuinely is implicated in a disproportionate share of outages.

The reason is structural. Almost every action begins with a name resolution, so a DNS failure presents as everything being broken simultaneously rather than as a DNS error, which is exactly the shape that makes a problem hard to localize. Caching means the symptom appears at unpredictable times and in unpredictable places, and a change to a record propagates on a schedule set by TTLs somebody chose months ago. Failures are also frequently partial, affecting one resolver's view and not another's, which produces the impossible reports where a service is down for half the staff.

The corollary experienced engineers actually use is that DNS should therefore be the first thing eliminated rather than the first thing blamed. Resolving the name from several vantage points, checking whether the answer matches expectation, and confirming which resolver answered takes under a minute and either finds the problem or removes a large branch of the search tree. The joke is a heuristic, and it earns its place.

Also known as: it was DNS

← All glossary entries