Dyn - managed DNS, and the day it went down

A distributed denial-of-service attack against Dyn's infrastructure made a long list of major services unreachable at once - not because those services were down, but because the names that pointed at them could not be resolved. The applications were healthy and the internet could not find them.

The lesson generalised beyond one company: authoritative DNS is a single point of failure that does not look like one on an architecture diagram, because it is drawn as a lookup rather than as a dependency. The response across the industry was secondary providers and multiple authoritative services, which is a design decision made in the shape of a bill.

The specifics deserve recording, because the general lesson above is easy to nod at and the particulars are what make it stick. The attack began at about 11:10 UTC on Friday 21 October 2016 against Dyn's managed DNS in the US-East region, in three waves, two of which stopped the service outright. Something over 100,000 devices took part and the traffic was estimated at 1.2 terabits per second, among the largest ever measured at the time. Twitter, Netflix, Reddit, Spotify, GitHub and PayPal became unreachable for millions of people across the United States and Europe, all of them running normally behind names nobody could resolve.

The weapon was Mirai, and its biography is the uncomfortable part. It scanned the entire IPv4 internet for anything with telnet open and logged in using a list of sixty-two factory-default passwords. It used no amplification and no reflection - only the raw volume of a great many small machines. Its authors were three college students whose interest was hosting Minecraft servers, and three weeks before the Dyn attack one of them published the complete source code on a forum, so that by 21 October anyone could assemble the same army. They pleaded guilty in 2017 and were sentenced in 2018 to probation and community service. A worm built by students to win a game still appears in record-setting attacks a decade later, because the condition it exploits has not changed.

What it was made of matters to a networking catalogue. The devices were consumer security cameras and video recorders, cheap, unattended, and never updated; a Chinese component maker recalled the boards inside many of them within days. These are the population the access-control article describes as the hard case - equipment that cannot run an agent and often cannot be told apart from the furniture - and the incident is the proof that the cheapest device on the network can be the one that decides whether the most expensive service stays reachable.

Dyn itself was absorbed into Oracle in the weeks that followed. Whether that was an endorsement of its importance or a consequence of its exposure can be argued either way; what is not arguable is that the industry drew the design conclusion recorded above, and that a single authoritative provider has been treated as a risk to be priced ever since.