SYN cookies answer when the queue overflows

Sat, 30 Oct 2027 00:00:00 +0000

A Linux TCP listener distinguishes pending handshake requests from established connections waiting for accept(). These are commonly called the SYN queue and accept queue. Their capacity limits and overflow paths differ, so diagnosing a connection failure requires identifying which stage is under pressure.

SYN cookies reduce retained per-request state while a handshake is incomplete. The server encodes recoverable information in its SYN-ACK and reconstructs connection state after a valid reply. This saves pending-request memory; it does not make processing free, remove established-connection state, or guarantee admission and normal application performance.

SYN cookies concern the handshake. Verification does not bypass the accept queue or increase application acceptance capacity.

Two queues, two different symptoms

On the ordinary stateful path, the listener remembers a request while awaiting the final ACK. Linux documents tcp_max_syn_backlog as a per-listener limit on remembered requests; the documented memory-dependent default and request-size estimate are not universal allocation measurements. The application’s listen() backlog, capped by somaxconn, bounds established connections waiting for acceptance. Linux has defaulted somaxconn to 4096 since 5.4; actual settings can differ.

Cookies can help when pending-handshake state is exhausted, but do not guarantee connection completion. A full accept queue can prevent creation or admission of the established child. With tcp_abort_on_overflow=0, some stateful paths ignore the ACK and retain the pending request, allowing later recovery. A client may already report connect success; other attempts may time out. The earlier 400-client account is not reproduced or bound to retained raw results in this review, so it cannot establish an exact loss count.

Linux mode 1 uses cookies as an overflow fallback when CONFIG_SYN_COOKIES is enabled; mode 2 generates them unconditionally for testing. In the pinned Linux 6.12 IPv4 implementation, the sequence number combines keyed hashes, the client initial sequence number, a time counter and an MSS-table index. The raw high bits of the returned acknowledgment are not simply a readable counter.

That implementation advances its counter every sixty seconds and accepts only the current or preceding counter slot before further validation. A seventy-second-old cookie can cross two boundaries and fail; elapsed time alone does not fix the outcome. Validation also binds connection fields and checks the recovered index. These checks resist blind spoofing but provide neither application authentication nor a universal one-in-2²⁴ security guarantee.

Option support depends on the implementation and handshake. Linux 6.12 can encode window-scale, SACK and ECN information in timestamp bits and recover it from the timestamp echo. Without that timestamp path, options can be lost. The documentation warns about extension and performance limitations, but does not justify saying every cookie SYN-ACK has no options or every admitted connection is slower.

The pinned IPv4 implementation chooses among four MSS-table entries, rather than preserving an arbitrary advertisement exactly. If scaling is absent, a 65535-byte receive window can constrain a long-RTT transfer; that is a conditional window limit, not measured cookie-mode throughput. Missing SACK changes recovery information without forcing retransmission of every later segment. Capture the actual options before attributing a slowdown to cookies.

When it breaks

Cookies can reduce visible pending-request depth while pressure continues. Monitor changes in SyncookiesSent alongside queue occupancy, connection attempts, CPU load, warnings and application outcomes. A rising counter is not proof of an attack: legitimate overload or unconditional test mode can also produce cookies. Application failures remain possible and useful evidence; cookies do not make all flooding invisible.

Cookies do not add accept-queue or application-processing capacity. Investigate ListenOverflows, acceptance rate and resource limits together before choosing a remedy. A larger backlog absorbs some bursts but does not increase sustained processing capacity. Faster acceptance or additional workers helps only when that stage is the bottleneck; changing cookie mode does not repair a stalled application.

Check it quickly

These read-only Linux commands inspect configuration, cumulative counters and listening queues; they do not reproduce a flood, validate cookies or measure packet delivery:

sysctl net.ipv4.tcp_syncookies net.ipv4.tcp_max_syn_backlog net.core.somaxconn
nstat -az | grep -E 'Syncookies|ListenOverflow|ListenDrops'
ss -lnt

Take comparable snapshots in the relevant network namespace and examine deltas. A healthy current service can have nonzero historical counters, and a brief nonempty accept queue is normal. For a listening TCP socket, Recv-Q shows established connections waiting for accept() and Send-Q the backlog limit. Sustained occupancy near that limit, new overflows and request failures together are stronger evidence of acceptance pressure than any single total.

Sources

RFC 4987 TCP SYN flood mitigation, the cookie construction and its trade-offs, checked 2026-09-16.
RFC 9293 TCP, the three-way handshake and the notion of a backlog, checked 2026-09-16.
Linux kernel ip-sysctl, tcp_syncookies, tcp_max_syn_backlog, somaxconn and tcp_abort_on_overflow, checked 2026-09-16.
Linux 6.12 IPv4 SYN cookies, keyed cookie construction, MSS table and timestamp-carried options, checked 2026-10-19.
Linux 6.12 TCP cookie timing, sixty-second counter and two-slot age test, checked 2026-10-19.
Linux 6.12 pending request handling, conditional overflow handling and retained requests, checked 2026-10-19.

Handshake · XB Field Notes

SYN cookies answer when the queue overflows

Two queues, two different symptoms

The cookie is the state you refused to store

What a cookie cannot carry

When it breaks

Check it quickly

Sources