underlay / overlay

networkingcloud

The two-layer view of modern networks: the underlay is the physical routed plumbing; the overlay is the virtual topology tunneled on top of it.

When the overlay misbehaves, the underlay is where you troubleshoot; conflating the two is the classic modern-network confusion.

An underlay is the physical network that actually moves packets; an overlay is a virtual network built on top of it by encapsulating traffic so that endpoints appear adjacent when they are not. VXLAN, GRE, IPsec tunnels and SD-WAN all build overlays, and the underlay is whatever carries them.

The separation buys independence, which is the whole point. An overlay can present a consistent addressing and policy model across data centres, clouds and offices whose underlying networks are owned by different people and cannot be changed, and workloads can move without renumbering. That is why virtualization and multi-cloud designs depend on it entirely.

The cost is that troubleshooting now requires two mental models at once. Overlay reachability can be perfect while the underlay is dropping packets, and vice versa, and the classic symptom is a tunnel that establishes correctly and then fails on large transfers because encapsulation overhead exceeded the underlay's MTU. Anyone diagnosing an overlay problem needs to establish which layer is broken before anything else, because the tools and the owners differ.

← All glossary entries