The TCP Proxy: What a Layer 4 Middlebox Does and Does Not See

A proxy is any intermediary that sits between a client and a server and speaks to each on the other's behalf. The lowest useful place to do that is Layer 4, the transport layer, where the unit of work is a TCP connection rather than an HTTP request or a TLS record. A TCP proxy is the simplest full proxy there is, and understanding exactly what it can and cannot see makes every higher-layer proxy easier to reason about.

Origins: the load balancer that could not read

Layer 4 proxying exists because the first load balancers had to work without understanding anything. In the 1990s a device in front of a server farm had one job - spread connections - and the traffic above the transport was somebody else's protocol. Reading it would have meant implementing it, and implementing it would have meant breaking whenever it changed.

So the design settled on the smallest possible commitment: accept a connection, choose a member, open a second connection, and move bytes between them without interpretation. That refusal to look is the whole architecture, and it is why the same device handles a protocol that did not exist when it was built.

The trade has not changed since. Layer 4 is fast, protocol-agnostic and cheap in state; it cannot make a decision that depends on what the bytes mean.

Two connections, spliced

The defining move of a TCP proxy is that it does not forward packets; it terminates one connection and originates another. The client completes a full TCP handshake (SYN, SYN-ACK, ACK) with the proxy and believes it is talking to the server. The proxy, in turn, completes its own separate handshake with the real server. From then on the proxy relays bytes between the two: it reads from one socket and writes to the other, in both directions, until one side closes. This is called connection splicing or a full proxy at Layer 4.

Because there are two independent connections, they can differ in almost every transport detail. The client-side and server-side connections have their own sequence numbers, their own window sizes, their own TCP options, and their own congestion state. A slow, lossy client link and a fast server link no longer share one end-to-end control loop; the proxy buffers between them and lets each side run at its own pace. That decoupling is a large part of why a full proxy improves performance and resilience, and it is invisible to both endpoints.

What Layer 4 can see, and what it cannot

A TCP proxy operates on the transport header: source and destination IP address, source and destination port, and the TCP flags and options. That is enough to make a forwarding decision, to load-balance across a pool of servers, to enforce connection limits, and to apply source and destination network address translation. It is not enough to see anything about the application. HTTP methods, URLs, headers, hostnames, and any TLS handshake details all live in the payload above the transport header, and a pure Layer 4 proxy treats that payload as an opaque stream of bytes it copies without parsing.

This is the key trade-off. A Layer 4 proxy is fast and protocol-agnostic precisely because it does not parse the payload; it will happily proxy HTTP, TLS, SSH (Secure Shell), a database protocol, or anything else, because it does not care what the bytes mean. But it also cannot make decisions that depend on application content, cannot route by URL, and cannot inspect an encrypted stream. When you need those things you move up to a Layer 7 proxy (see the article on HTTP proxies), and you pay for the extra visibility with the cost of parsing.

There is one important nuance for encrypted traffic. A Layer 4 proxy cannot read a TLS session, but the very first bytes of a TLS ClientHello are sent in the clear, and they include the Server Name Indication (SNI). A Layer 4 proxy can peek at that one field to route a connection to the right backend by hostname without decrypting anything, a common pattern for TLS passthrough. It is still not decryption; it is reading a plaintext label on an otherwise opaque stream.

The disappearing source address

Because the proxy originates a brand-new connection to the server, the server sees the proxy's address as the source, not the client's. From the server's point of view every connection comes from the handful of proxy IPs. This is usually deliberate (it is how source NAT and connection pooling work) but it destroys the one piece of information many servers most want: who the client actually was. Access logs show the proxy; IP-based access rules see the proxy; geolocation sees the proxy.

Two mechanisms put the client identity back. At Layer 7, HTTP proxies add an X-Forwarded-For header, but that only works for HTTP and only after the payload is parsed. At Layer 4, the answer is the Proxy Protocol, a tiny header the proxy prepends to the very start of the server-side TCP stream, before any application data, carrying the original source and destination address and port. The server (if it understands the Proxy Protocol) reads that header first, records the real client address, and then treats the rest of the stream normally. It is the Layer 4 way to preserve client identity through a connection that has been re-originated, and it is protocol-agnostic because it sits below the application entirely.

What it cannot do, and what people try anyway

It cannot route by content. A path, a hostname in a header, a cookie or a method are all invisible. Any requirement phrased as "send /api to this pool" is a layer 7 requirement wearing a layer 4 question.

It cannot retry a failed request. If a member dies mid-connection, the proxy can only reset; it has no idea whether the client's request was safely repeatable, because it never parsed one.

It cannot inspect what it forwards. Encrypted or not, the payload is opaque by design - which is why layer 4 is the honest choice for traffic you have decided not to inspect, and the wrong one for traffic you have to.

It multiplies connections. Two sockets per session, held for the session's life, which turns into a table sized by concurrency rather than by request rate. Long-lived idle connections are the usual cause of exhaustion, and the fix is timeouts that match the application rather than the default.

The persistence problem

Because there is no cookie and no header to key on, stickiness at layer 4 is usually by source address. That works until the clients arrive behind one address - a corporate gateway, a mobile carrier, a CGNAT deployment - and then a thousand users pin to one member, which looks like a broken balancing algorithm and is actually the algorithm working on the information available.

The landscape, by category

• Kernel and open-source proxies - IPVS, HAProxy in TCP mode, NGINX stream, and the eBPF-based load balancers behind large cloud fabrics.
• Application delivery controllers - F5, Citrix, A10 and similar, where layer 4 is the fast path and layer 7 the option you turn on per virtual server.
• Cloud network load balancers, sold explicitly on the layer 4 trade: high throughput, low latency, no content awareness, and often preserving the client address where the layer 7 product does not.
• Anything doing TLS passthrough, which is layer 4 by necessity because the decision has been made not to hold the key.

Where you meet TCP proxies

Any load balancer or ADC running in a fast Layer 4 mode is a TCP proxy. On F5 BIG-IP, for example, the FastL4 profile drives a lightweight Layer 4 path (with hardware acceleration on supported platforms) for exactly the cases where you want throughput and connection handling without Layer 7 parsing; a Standard virtual server, by contrast, engages the full TCP proxy and can layer HTTP and TLS processing on top. Cloud load balancers expose the same split as "network" (Layer 4) versus "application" (Layer 7) load balancers. The pattern is identical everywhere: decide whether you need to see the application, and if you do not, stay at Layer 4 where it is cheapest and most general.