Protocols · XB Field Notes

Gossip converges under explicit assumptions

Mon, 20 Sep 2027 00:00:00 +0000

Gossip spreads information through repeated peer contact. It can tolerate some lost exchanges because surviving copies can be sent again. Eventual convergence still needs assumptions: relevant peers must become reachable, updates must remain available, and the merge rule must handle the states that can arise. Random contact alone does not guarantee agreement by a fixed deadline.

Separate dissemination from reconciliation

A node can periodically push state, pull another node’s state, or exchange in both directions. These are protocol choices, not a requirement that every gossip message be symmetrical. A bounded number of contacts per node can spread a small update rapidly through a well-connected population. Logarithmic dissemination estimates describe particular probabilistic models; they are not a worst-case timer derived from one network round trip.

Membership gossip also does not make application writes consistent by itself. Dynamo uses gossip for membership, vector clocks for object versions, and Merkle-tree anti-entropy for repairing replicas. Those mechanisms solve different problems. A membership protocol’s incarnation number distinguishes states of one member; it is not automatically a causal version for arbitrary business data.

The model below intentionally assumes one writer per key and increasing versions. Selecting the larger version is enough for its single update. Equal versions containing different values expose an undefined conflict policy: retaining whichever value arrived first makes the merge order-dependent. Production reconciliation needs a declared policy for concurrency, duplicate delivery, deletion, and restart identity.

When it breaks

A lasting partition prevents an update from crossing between disconnected components. Repetition inside each component cannot repair that missing path. After connectivity returns, reconciliation can resume if the necessary state still exists. A deleted item also needs a deletion representation and retention policy; expiring tombstones too early can let stale data return. Version vectors and tombstones are complementary mechanisms, not interchangeable guarantees against lost writes.

A slow or paused member can resemble a failed one. SWIM separates failure detection from infection-style dissemination and adds suspicion so a member can refute a suspected failure with a newer incarnation. This reduces false declarations under its assumptions; it does not provide a perfect failure detector during arbitrary delays. Re-replication and ownership changes are decisions of the surrounding system, not inevitable consequences of every gossip suspicion.

Repair consumes resources too. Full-state exchanges, aggressive intervals, and simultaneous rejoins can overload links or receivers. Bounded fanout, incremental exchange, and controlled recovery help, but costs depend on state size as well as the number of contacted peers. Restart versioning must follow the implementation’s identity rules: resetting a counter can make new announcements appear stale or create ambiguity; a lower counter does not inherently overwrite a higher one.

A missing path blocks gossip. Source-based mechanism illustration. Model values and source-reported observations are not a new runtime test.

Check it quickly

Save this as check_gossip.py and run python3 check_gossip.py. It uses only Python’s standard library and no sockets. Twenty dictionaries model peers; each sweep visits nodes sequentially and performs push-pull merges. A fixed seed reproduces the random choices in this runtime. Knowledge can cross several contacts during one sweep, so this is not a synchronized network-round simulator.

import random

NODES, ROUNDS, SEED = 20, 12, 7

def merge(a, b):
 # One writer per key, with strictly increasing versions in this fixture.
 merged = dict(a)
 for key, (version, value) in b.items():
 if key not in merged or version > merged[key][0]:
 merged[key] = (version, value)
 return merged

def spread(peer_for, nodes=None):
 rng = random.Random(SEED)
 if nodes is None:
 nodes = [{} for _ in range(NODES)]
 nodes[0]["config"] = (1, "blue")
 history = []
 for _ in range(ROUNDS):
 # Sequential contacts: knowledge can travel several hops in one sweep.
 for i in range(NODES):
 peer = peer_for(i, rng)
 if peer == i:
 continue
 united = merge(nodes[i], nodes[peer])
 nodes[i], nodes[peer] = dict(united), dict(united)
 history.append(sum(state.get("config") == (1, "blue") for state in nodes))
 return nodes, history

random_peer = lambda i, rng: rng.randrange(NODES)
nodes, history = spread(random_peer)
assert history[-1] == NODES
assert all(a <= b for a, b in zip(history, history[1:]))
print("random contacts:", history)

# A fixed hub is connected; fixed contacts do not inherently prevent convergence.
_, hub = spread(lambda i, rng: 0)
assert hub[0] == NODES
print("fixed hub after one sweep:", hub[0])

# Two disconnected peer sets retain different views until contacts can cross.
partitioned, split = spread(lambda i, rng: (i // 10) * 10 + rng.randrange(10))
assert split[-1] == 10 and all(not node for node in partitioned[10:])
_, healed = spread(random_peer, partitioned)
assert healed[-1] == NODES
print("partition:", split[-1], "after healing:", healed[-1])

# Dissemination alone cannot repair an ambiguous concurrent-write merge rule.
a, b = {"config": (2, "red")}, {"config": (2, "green")}
assert merge(a, b) != merge(b, a)
print("equal-version conflict is order-dependent: not a multi-writer merge")
print("PASS: dissemination, topology, partition, and merge controls")

The random-contact history is 5, 14, 19, then 20 informed peers, with later sweeps staying at 20. That reproduces the original single-update observation, which was recorded with CPython 3.12 on macOS. It is evidence about this input and schedule, not a probability bound for other seeds or deployed clusters. The current model adds explicit counterexamples rather than inferring reliability from that successful run alone.

Every node contacting the same live hub spreads the update to all twenty in one sweep. Fixed partners therefore do not necessarily stall dissemination; connectivity and scheduling matter. The partition control confines contact to two groups of ten, leaving the second group uninformed. Restoring unrestricted contacts eventually informs everyone in this deterministic run.

Finally, merging equal-version red and green values in opposite orders produces different results. That assertion demonstrates the fixture’s single-writer limitation instead of silently calling it general conflict resolution. No network loss, suspicion timeout, tombstone lifecycle, bandwidth budget, or real membership implementation is exercised. Measure those separately before choosing operational convergence targets.

Sources

Dynamo: Amazon’s highly available key-value store, gossip-based membership and anti-entropy with version reconciliation, checked 2026-09-12.
SWIM: Scalable Weakly-consistent Infection-style process group Membership protocol, suspicion mechanism and incarnation numbers for failure detection, checked 2026-09-12.
Cassandra internode communications (gossip), peer-to-peer state exchange with versioned gossip messages, checked 2026-09-12.

The TCP handshake establishes sequence spaces, not synchronized clocks

Fri, 10 Sep 2027 00:00:00 +0000

The TCP three-way handshake establishes the two directions of a byte stream. Each endpoint supplies an initial sequence number and acknowledges the other endpoint’s number. Those are independent sequence spaces, not a shared wall clock. Applications do not need synchronized machine clocks to connect. Later sequence numbers identify bytes and support ordering, duplicate handling, acknowledgments, and retransmission within that connection.

Ordinary setup and the controlled loss case are shown; 32-bit sequence numbers wrap. Connect success does not prove application readiness.

When it breaks

A client sends SYN with its initial sequence number. The server returns SYN-ACK containing its own initial number and acknowledging the client’s SYN. The client acknowledges that response. SYN consumes one position in sequence space, so its acknowledgment advances by one even when the segment contains no application data. Treating these values as timestamps produces incorrect packet interpretations and can send an investigation toward unrelated clock synchronization.

The two endpoints do not change state at exactly the same instant. A client’s connect call can succeed after receiving SYN-ACK while its final acknowledgment is lost before reaching the server. The server can remain in SYN-RECEIVED, displayed as SYN-RECV by Linux tools. Subsequent data carrying an appropriate acknowledgment can also complete the exchange, so a missing standalone third packet does not inevitably cause permanent failure.

Successful TCP setup establishes neither application health nor peer authentication. A service might accept connections but never produce a valid response. A proxy may complete one TCP connection while failing to establish its separate upstream connection. Likewise, sequence-number selection helps protect connection incarnations and resist prediction attacks, but it is not an authentication protocol and does not encrypt the stream.

Check it quickly

Save this as check-tcp-handshake.py and run python3 check-tcp-handshake.py with Docker. The disposable container uses Python 3.11.16, iproute2 6.1.0, nftables 1.0.6, and tcpdump 4.99.3, tested with Linux 6.12.76. NET_ADMIN and SYS_ADMIN support its nested network namespaces. A newly created veth joins the client and target, with no host interfaces or published ports involved:

The environment and numerical outcomes below are inherited article reports, not independently reproduced here. The image does not pin the host kernel or package availability.

The revised reader retains byte chunks from unbuffered pipes and consumes complete buffered lines before waiting for readiness. One deadline covers each line, including fragmented arrivals; EOF, malformed text and oversized lines fail explicitly. Synthetic tests cover queued and delayed statuses, timeouts and cleanup calls. They do not establish actual pipe scheduling, process cleanup or network behavior: an authorized integrated rerun is still required. A child-status timeout alone is not a TCP diagnosis.

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,uuid
recipe='FROM python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84\nRUN apt-get update && apt-get install -y --no-install-recommends iproute2=6.1.0-3 iputils-ping=3:20221126-1+deb12u1 nftables=1.0.6-2+deb12u2 tcpdump=4.99.3-1 traceroute=1:2.1.2-1 procps=2:4.0.2-3 && rm -rf /var/lib/apt/lists/*\nCMD ["python3"]\n'
program=r"""
from contextlib import ExitStack,contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import json,os,select,signal,subprocess,time

def run(*args,check=True,input=None,timeout=8):
 result=subprocess.run(args,input=input,capture_output=True,text=True,timeout=timeout)
 if check:assert result.returncode==0,(args,result.returncode,result.stdout,result.stderr)
 return result
def ns(name,*args,**kwargs):return run('ip','netns','exec',name,*args,**kwargs)
def ip(name,*args,**kwargs):return ns(name,'ip',*args,**kwargs)
def stop(process):
 if process.poll() is None:
 process.terminate()
 try:process.wait(timeout=2)
 except subprocess.TimeoutExpired:process.kill();process.wait(timeout=2)
class DeadlineLines:
 def __init__(self,read_chunk,wait_readable,now,limit=65536):
 self.read_chunk=read_chunk;self.wait_readable=wait_readable;self.now=now
 self.pending=bytearray();self.eof=False;self.limit=limit
 def read(self,timeout):
 deadline=self.now()+timeout
 while True:
 end=self.pending.find(b'\n')
 if end>=0:
 if end+1>self.limit:raise ValueError('child status line too long')
 value=bytes(self.pending[:end+1]);del self.pending[:end+1]
 return value.decode('utf-8')
 if len(self.pending)>self.limit:raise ValueError('child status line too long')
 if self.eof:raise EOFError(('child status EOF before newline',bytes(self.pending[:160])))
 remaining=deadline-self.now()
 if remaining<=0 or not self.wait_readable(remaining):
 raise TimeoutError('child status deadline exceeded; this is not a TCP diagnosis')
 chunk=self.read_chunk(4096)
 if not chunk:self.eof=True
 else:self.pending.extend(chunk)
def reader(stream):
 # Only use this with an unbuffered binary pipe, with no other reader.
 return DeadlineLines(stream.read,lambda timeout:bool(select.select([stream],[],[],timeout)[0]),time.monotonic)
def line(process,timeout=3):return process.status.read(timeout)
def spawn(stack,command):
 process=subprocess.Popen(command,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE,bufsize=0)
 for stream in (process.stdin,process.stdout,process.stderr):stack.callback(stream.close)
 stack.callback(stop,process);process.status=reader(process.stdout);return process
@contextmanager
def lab(names):
 with TemporaryDirectory(prefix='net-path-') as directory,ExitStack() as stack:
 root=Path(directory);proc=root/'proc';proc.mkdir();run('mount','-t','proc','proc',str(proc))
 stack.callback(run,'umount',str(proc))
 for name in names:
 run('ip','netns','add',name);stack.callback(run,'ip','netns','delete',name,check=False)
 ip(name,'link','set','lo','up')
 def setting(name,key,value):
 assert name in names
 path=proc/'sys'/key.replace('.','/')
 return ns(name,'python3','-c',f'from pathlib import Path; p=Path({str(path)!r}); p.write_text({str(value)!r}); print(p.read_text().strip())').stdout.strip()
 yield root,stack,setting
def wire(left,left_if,right,right_if,left_addresses,right_addresses):
 run('ip','link','add',left_if,'type','veth','peer','name',right_if)
 for name,interface,addresses in [(left,left_if,left_addresses),(right,right_if,right_addresses)]:
 run('ip','link','set',interface,'netns',name)
 for address in addresses:ip(name,'address','add',address,'dev',interface,*(['nodad'] if ':' in address else []))
 ip(name,'link','set',interface,'up')
def nft(name,script):return ns(name,'nft','-f','-',input=script)
def counters(name):
 data=json.loads(ns(name,'nft','-j','list','ruleset').stdout);found={}
 for entry in data['nftables']:
 rule=entry.get('rule',{})
 if 'comment' in rule:
 for expression in rule['expr']:
 if 'counter' in expression:found[rule['comment']]=expression['counter']['packets']
 return found


def capture(stack,root,name,interface,expression,label):
 path=root/(label+'.pcap')
 process=subprocess.Popen(['ip','netns','exec',name,'tcpdump','--immediate-mode','-Z','root','-U','-n','-i',interface,'-w',str(path),expression],
 stdout=subprocess.DEVNULL,stderr=subprocess.PIPE,bufsize=0)
 stack.callback(process.stderr.close);stack.callback(stop,process);messages=reader(process.stderr)
 deadline=time.monotonic()+3
 while time.monotonic()<deadline:
 assert process.poll() is None,'tcpdump exited before listening'
 if 'listening on' in messages.read(max(0,deadline-time.monotonic())):return process,path
 raise AssertionError('tcpdump did not become ready')
def packets(process,path):
 import struct,socket
 process.send_signal(signal.SIGINT);_,diagnostic=process.communicate(timeout=3)
 assert process.returncode==0,diagnostic
 raw=path.read_bytes();assert raw[:4]==bytes.fromhex('d4c3b2a1') and struct.unpack_from('<I',raw,20)[0]==1
 offset=24;result=[]
 while offset<len(raw):
 seconds,micros,size,original=struct.unpack_from('<IIII',raw,offset);offset+=16
 frame=raw[offset:offset+size];offset+=size
 if frame[12:14]!=b'\x08\x00' or frame[23]!=6:continue
 ihl=(frame[14]&15)*4;length=struct.unpack_from('!H',frame,16)[0];tcp=14+ihl
 sport,dport,seq,ack=struct.unpack_from('!HHII',frame,tcp);header=(frame[tcp+12]>>4)*4
 opts={};cursor=tcp+20
 while cursor<tcp+header:
 kind=frame[cursor]
 if kind==0:break
 if kind==1:cursor+=1;continue
 option_length=frame[cursor+1];data=frame[cursor+2:cursor+option_length];assert option_length>=2
 if kind==2:opts['mss']=struct.unpack('!H',data)[0]
 elif kind==3:opts['wscale']=data[0]
 elif kind==4:opts['sack_ok']=True
 elif kind==5:opts['sack']=[list(pair) for pair in struct.iter_unpack('!II',data)]
 cursor+=option_length
 body=frame[tcp+header:14+length]
 result.append({'t':seconds+micros/1e6,'src':socket.inet_ntoa(frame[26:30]),'dst':socket.inet_ntoa(frame[30:34]),
 'sport':sport,'dport':dport,'seq':seq,'ack':ack,'flags':frame[tcp+13],'window':struct.unpack_from('!H',frame,tcp+14)[0],
 'length':len(body),'options':opts,'preview':body[:24].hex()})
 return result
with lab(['client','target']) as (root,stack,setting):
 wire('client','c0','target','t0',['192.0.2.1/30'],['192.0.2.2/30'])
 server_code=r'''
import socketserver
class Handler(socketserver.StreamRequestHandler):
 def handle(self):
 data=self.rfile.readline();self.wfile.write(b'echo:'+data);self.wfile.flush()
class Server(socketserver.ThreadingTCPServer):allow_reuse_address=True;daemon_threads=True
server=Server(('192.0.2.2',18080),Handler);print('ready',flush=True);server.serve_forever()
'''
 service=spawn(stack,['ip','netns','exec','target','python3','-u','-c',server_code]);assert line(service).strip()=='ready'
 tap,path=capture(stack,root,'client','c0','tcp port 18080','handshake')
 normal=ns('client','python3','-c',
 'import socket; s=socket.socket();s.settimeout(2);s.bind(("192.0.2.1",40100));s.connect(("192.0.2.2",18080));s.sendall(b"normal\\n");print(s.makefile("rb").readline().decode().strip());s.close()')
 assert normal.stdout.strip()=='echo:normal'
 nft('client','''table ip missing_ack {
 chain output { type filter hook output priority 0; policy accept;
 tcp dport 18080 tcp flags & (syn|ack|rst|fin) == ack counter drop comment "missing_final_ack"
 }
}
''')
 client_code=r'''
import json,socket,sys
s=socket.socket();s.settimeout(.25);s.bind(('192.0.2.1',40101));s.connect(('192.0.2.2',18080))
print(json.dumps({'connect_returned':True}),flush=True);s.sendall(b'deferred\n')
try:s.recv(64);raise AssertionError('unexpected application response')
except TimeoutError:print(json.dumps({'application_timeout':True}),flush=True)
assert sys.stdin.readline().strip()=='restore';s.settimeout(5)
received=b''
while not received.endswith(b'\n'):
 chunk=s.recv(64);assert chunk,'premature EOF';received+=chunk
assert received==b'echo:deferred\n',received
print(json.dumps({'recovered':received.decode().strip()}),flush=True);s.close()
'''
 client=spawn(stack,['ip','netns','exec','client','python3','-u','-c',client_code])
 connected=json.loads(line(client));failure=json.loads(line(client));assert connected['connect_returned'] and failure['application_timeout']
 pending=ns('target','ss','-nHt','state','syn-recv').stdout;assert '192.0.2.1:40101' in pending,pending
 blocked=counters('client');assert blocked['missing_final_ack']>0
 nft('client','delete table ip missing_ack');assert client.stdin.write(b'restore\n')==8;client.stdin.flush()
 recovery=json.loads(line(client,6));assert recovery['recovered']=='echo:deferred'
 client.wait(timeout=2);assert client.returncode==0
 observed=packets(tap,path);flow=[p for p in observed if 40100 in (p['sport'],p['dport'])]
 syn=next(p for p in flow if p['flags']&0x12==0x02)
 synack=next(p for p in flow if p['flags']&0x12==0x12)
 third=next(p for p in flow if p['sport']==40100 and p['flags']&0x12==0x10)
 assert synack['ack']==(syn['seq']+1)%2**32
 assert third['seq']==(syn['seq']+1)%2**32 and third['ack']==(synack['seq']+1)%2**32
 print(json.dumps({'three_messages':[syn,synack,third],'normal_reply':normal.stdout.strip(),
 'client_connect_returned':connected['connect_returned'],'application_timeout':failure['application_timeout'],
 'server_still_syn_recv':pending.strip(),'drop_counters':blocked,'after_ack_filter_removed':recovery}))
"""
name='xbcatm-tcpflow-'+uuid.uuid4().hex[:10]
def docker(*args,**kwargs):return subprocess.run(['docker',*args],capture_output=True,text=True,timeout=300,**kwargs)
try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'Dockerfile').write_text(recipe)
 build=docker('build','-t',name,str(root));assert build.returncode==0,build.stderr
 result=docker('run','--rm','-i','--name',name,'--network','none','--cap-add','NET_ADMIN','--cap-add','SYS_ADMIN',
 '--memory','256m','--pids-limit','128',name,'python3','-',input=program)
 assert result.returncode==0,result.stderr
 print(json.dumps(json.loads(result.stdout),sort_keys=True))
finally:
 docker('rm','-f',name);docker('image','rm',name)

The first connection uses client port 40100 and target port 18080. It sends a newline-terminated request and receives the expected echo. A packet capture supplies the actual SYN, SYN-ACK, and final ACK values. Assertions check that SYN-ACK acknowledges the client’s sequence plus one and that the third message acknowledges the server’s sequence plus one, using modulo arithmetic for the 32-bit sequence space.

The second connection uses client port 40101. An output filter in the client namespace drops ACK-bearing packets without SYN, FIN, or RST, including the final acknowledgment and subsequent data. Blocking that data matters: otherwise a later data segment could acknowledge the SYN-ACK and repair the intentionally missing third message. The example therefore makes the intended asymmetric state observable instead of relying on a narrow timing accident.

The client reports that connect returned successfully, but its application read reaches a deadline. At the same point, the server’s socket listing shows that exact client tuple in SYN-RECV. A rule counter confirms dropped packets. These independent observations establish the distinction between the client’s local API result, the server’s protocol state, and the failed application exchange.

Removing the filter lets the same connection finish the exchange and deliver the queued request. The client receives echo:deferred; it does not create a replacement socket to hide the original failure. Framed reads accumulate the response rather than assuming one recv call always returns the entire application message. The test verifies both the temporary failure and recovery after its controlled cause is removed.

Diagnose setup separately from useful service

Record the four-tuple, namespace, capture point, and state on each endpoint. A reset, repeated SYN, repeated SYN-ACK, and a completed handshake followed by an application timeout suggest different next checks. Capture both directions when possible: an outgoing acknowledgment visible locally is not proof that the remote endpoint received it. The fixture’s counter and remote socket state supply that missing context.

For service validation, follow the transport connection with a meaningful request and inspect its response. For identity and confidentiality, use the appropriate authenticated protocol above TCP. This exercise covers an ordinary connection without Fast Open or simultaneous open, and does not claim every valid connection begins with exactly three separately visible packets. Its central check is the sequence and state relationship, not a simplistic packet-count rule.

Sources

RFC 9293 TCP, sequence numbers, connection states and the three-way handshake, checked 2026-09-12.
Linux tcp(7), TCP socket behavior and Linux-specific controls, checked 2026-09-12.
Linux ss(8), examining established and SYN-RECV sockets, checked 2026-09-12.
Python 3.11 subprocess, unbuffered binary pipes; real process integration remains untested, checked 2026-10-19.

Well-known endpoints provide a discovery convention

Tue, 22 Jun 2027 00:00:00 +0000

A well-known URI gives software a conventional place to discover information associated with an origin. The reserved /.well-known/ prefix reduces collisions between independently designed protocols, and registered suffixes identify their defining specifications. The prefix is a naming mechanism, not an implementation or a guarantee of trustworthy content. A server must still route the selected endpoint correctly, and a client must interpret the response according to that endpoint’s own contract.

Follow the suffix’s specification

RFC 8615 reserves the namespace for supported URI schemes. It does not require every site to publish every registered resource, and it does not define a common response schema for all suffixes. Consult the IANA registration and its linked specification before deciding the media type, authentication, redirect handling, caching, or transport requirements. A familiar path cannot substitute for these protocol-specific rules.

For web-based vulnerability contact information, RFC 9116 places security.txt at /.well-known/security.txt and requires HTTPS retrieval. A legacy /security.txt location can redirect to the well-known location. The document is UTF-8 plain text with defined fields, including at least one Contact and a single Expires field. Contact gives a reporting channel; it does not prove that a mailbox is staffed or that a submitted report has been accepted.

The document’s Expires field uses a timestamp to limit the age of the information. It is distinct from an HTTP Expires response header. A transport-level 200 does not make stale contact metadata current. Keep the address and policy maintained, and renew the document before it becomes stale. If optional fields such as Canonical or Policy are used, apply their specification rather than treating arbitrary URLs as trusted declarations.

When it breaks

A broad application fallback can return an HTML login page or a site’s home page with status 200 for every unknown path. That looks reachable to a shallow health check but supplies no usable protocol document. Preserve the exact route and inspect both media type and content. An accidental redirect to an unrelated origin can also change the trust context instead of merely moving a file.

The namespace itself authenticates nothing. HTTPS validation establishes the connection to the intended host, while document validation and any additional verification requirements establish whether its contents are usable. Do not infer permission to scan or attack a service from the presence of a security contact. Nor should a deployment blindly expose an entire directory because one standardized resource is intended to be public.

Returning 404 for an unimplemented suffix is different from silently inventing a generic successful document. Each supported endpoint should have an owner and a check suited to its protocol. A registry entry describes a convention; it is not evidence that a particular origin has implemented it or that its current response passed validation.

Well-known discovery: mechanism and evidence boundaries.

Check it quickly

Save this as well-known.py and run it with Python 3.14 using python3 well-known.py. The fixture serves only loopback HTTP, with an explicitly fictitious contact at example.invalid. It tests routing and selected document fields. Because it does not use HTTPS, it is not a complete RFC 9116 deployment or a transport-security test.

Validation (2026-10-17): Retained macOS arm64 Python 3.14.4 records match both complete fence payloads and report success with assertions enabled and loopback guards. This source review did not rerun them. Do not use -O or PYTHONOPTIMIZE: the unimplemented-route fetch is inside an assertion. Guards recorded connection attempts, not packet-capture request counts.

from datetime import datetime, timedelta, timezone
from http.client import HTTPConnection
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
import json

now = datetime.now(timezone.utc)
future = (now + timedelta(days=30)).isoformat().replace('+00:00', 'Z')
past = (now - timedelta(days=1)).isoformat().replace('+00:00', 'Z')
content = {'text': f'Contact: mailto:security@example.invalid\nExpires: {future}\n'}


class Handler(BaseHTTPRequestHandler):
 def do_GET(self):
 status = 200 if self.path == '/.well-known/security.txt' else 301 if self.path == '/security.txt' else 404
 data = content['text'].encode() if status == 200 else b''
 self.send_response(status)
 if status == 301:
 self.send_header('Location', '/.well-known/security.txt')
 self.send_header('Content-Type', 'text/plain; charset=utf-8')
 self.send_header('Cache-Control', 'no-store')
 self.send_header('Content-Length', str(len(data)))
 self.end_headers()
 self.wfile.write(data)

 def log_message(self, *args):
 pass


def check_contact_and_expiry(text):
 fields = dict(line.split(': ', 1) for line in text.splitlines())
 if not fields.get('Contact', '').startswith('mailto:'):
 raise ValueError('missing demonstration contact')
 expires = datetime.fromisoformat(fields['Expires'])
 if expires <= now:
 raise ValueError('expired metadata')


server = ThreadingHTTPServer(('127.0.0.1', 0), Handler)
thread = Thread(target=server.serve_forever)
thread.start()
try:
 def fetch(path):
 connection = HTTPConnection('127.0.0.1', server.server_port, timeout=5)
 try:
 connection.request('GET', path)
 response = connection.getresponse()
 return {'status': response.status, 'body': response.read().decode(),
 'location': response.getheader('Location'),
 'content_type': response.getheader('Content-Type')}
 finally:
 connection.close()

 valid = fetch('/.well-known/security.txt')
 assert valid['status'] == 200 and valid['content_type'] == 'text/plain; charset=utf-8'
 check_contact_and_expiry(valid['body'])
 legacy = fetch('/security.txt')
 assert legacy['status'] == 301 and legacy['location'] == '/.well-known/security.txt'
 assert fetch('/.well-known/unimplemented')['status'] == 404
 rejected = []
 for text in (content['text'].replace(future, past), f'Expires: {future}\n'):
 content['text'] = text
 response = fetch('/.well-known/security.txt')
 assert response['status'] == 200
 try:
 check_contact_and_expiry(response['body'])
 except ValueError as error:
 rejected.append(str(error))
 assert rejected == ['expired metadata', 'missing demonstration contact']
 print(json.dumps({'valid_status': valid['status'], 'legacy_status': legacy['status'],
 'unimplemented_status': 404, 'rejected_200_documents': rejected,
 'https_transport_tested': False}))
finally:
 server.shutdown()
 thread.join()
 server.server_close()

The well-known route first returns 200 with the expected plain-text media type, a contact, and expiry thirty days ahead. The legacy path returns 301 to the standard location, and an unimplemented suffix returns 404. The script then serves two invalid documents at the same successful route: one expired and one missing its contact. Both still return HTTP 200 but fail the field checks.

The parser handles only the demonstration format: its dictionary collapses duplicate fields, and it accepts only mailto contacts. It compares against startup time, not a refreshed observation time. It is not a complete parser for repeated fields, signatures, canonical URLs, redirects or URI syntax. These are deliberate limits, not full RFC compliance. Validate actual transport, content and freshness before publishing a real endpoint.

Sources

RFC 8615: Well-Known URIs, the reserved namespace and registration model, checked 2026-09-13.
RFC 9116: security.txt, location, HTTPS transport, contact fields, and expiry, checked 2026-09-13.
IANA: Well-Known URIs registry, registered suffixes and their defining specifications, checked 2026-09-13.
Python assert statement, optimization removes assertions and their expressions, checked 2026-10-18.

DNS needs a working TCP path when a UDP answer is truncated

Thu, 19 Nov 2026 00:00:00 +0000

DNS supports both UDP and TCP. A server sets the TC flag when a response is truncated for the available message space. A client receiving that flag must handle the incomplete result, commonly by retrying over TCP. This is about the DNS response limit in use, not proof that the physical network cannot carry any larger UDP datagram.

TC marks an incomplete answer even when records are present. A working TCP path recovers the complete set; a UDP-only endpoint cannot. The article also demonstrates separate message framing and connection reuse, without encryption guarantees.

When it breaks

A network that permits UDP port 53 but blocks TCP port 53 can pass small lookups and fail larger ones. DNSSEC data and larger record sets make the difference visible, but TCP is not reserved for those cases. General-purpose DNS implementations need TCP support, and clients may choose it directly. A successful small UDP query therefore does not establish that the DNS service’s complete transport path works.

A truncated answer can still contain records. Ignoring TC and using those records as the complete set can omit valid destinations or other required data. NOERROR does not cancel the truncation flag. Capture the flags and complete question alongside answer counts, especially when an application appears to see fewer records than a direct authoritative query through a different transport.

TCP also does not remove all DNS message-size limits. Each DNS message on a TCP stream is preceded by a two-byte length, which limits that message to 65,535 bytes. A zone transfer can use multiple DNS messages. Stream reads may split either the length prefix or the message, so code must read the required number of bytes rather than assuming one recv call returns a whole answer.

Check it quickly

Save this as check-dns-tcp.py and run python3 check-dns-tcp.py with Docker. The pinned BIND 9.18.49 environment serves a four-record TXT set inside an isolated container. The initial query explicitly disables EDNS to exercise the traditional 512-byte UDP limit. Later stages use a real TCP connection and a deliberately UDP-only relay:

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,uuid
recipe='FROM python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84\nRUN apt-get update && apt-get install -y --no-install-recommends bind9=1:9.18.49-1~deb12u2 bind9-dnsutils=1:9.18.49-1~deb12u2 dnsmasq-base=2.90-4~deb12u2 python3-dnspython=2.3.0-1 && rm -rf /var/lib/apt/lists/*\nCMD ["/usr/bin/python3"]\n'
program=r"""
from contextlib import ExitStack,contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import json,re,signal,subprocess,time
import dns.exception,dns.message,dns.query,dns.rcode,dns.rdatatype

def run(*args,check=True):
 result=subprocess.run(args,capture_output=True,text=True,timeout=15)
 if check:assert result.returncode==0,(args,result.returncode,result.stdout,result.stderr)
 return result
def query(name,kind='A',port=1053):
 return dns.query.udp(dns.message.make_query(name,kind),'127.0.0.1',port=port,timeout=1)
def values(response,kind='A'):
 wanted=dns.rdatatype.from_text(kind)
 return [item.to_text() for rrset in response.answer if rrset.rdtype==wanted for item in rrset]
def show(response):
 def section(records):return [{'name':str(rr.name),'type':dns.rdatatype.to_text(rr.rdtype),'ttl':rr.ttl,'data':[item.to_text() for item in rr]} for rr in records]
 return {'rcode':dns.rcode.to_text(response.rcode()),'answer':section(response.answer),'authority':section(response.authority)}
def wait(predicate,timeout=15):
 deadline=time.monotonic()+timeout
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except (OSError,dns.exception.Timeout):pass
 time.sleep(.05)
 raise AssertionError('DNS fixture condition did not become true')
def stop(process):
 if process.poll() is None:
 process.terminate()
 try:process.wait(timeout=5)
 except subprocess.TimeoutExpired:process.kill();process.wait(timeout=5)

@contextmanager
def lab(records,soa_ttl=8,minimum=4,extra=''):
 with TemporaryDirectory(prefix='xbcatm-dns-') as directory,ExitStack() as stack:
 root=Path(directory);zone=root/'zone';conf=root/'named.conf';log=root/'named.log';serial=1
 def save(body):
 zone.write_text(f'$ORIGIN lab.test.\n$TTL 30\n@ {soa_ttl} IN SOA ns.lab.test. hostmaster.lab.test. ({serial} 60 60 3600 {minimum})\n@ IN NS ns.lab.test.\nns IN A 127.0.0.1\n'+body+'\n')
 run('named-checkzone','lab.test',str(zone))
 save(records)
 conf.write_text(f'''options {{ directory "{root}"; listen-on port 1053 {{127.0.0.1;}}; listen-on-v6 {{none;}};
recursion no; dnssec-validation no; notify no; querylog yes; max-cache-size 8m; pid-file "{root}/named.pid"; session-keyfile none;
{extra} }};
controls {{}};
zone "lab.test" {{ type primary; file "{zone}"; }};
''')
 run('named-checkconf',str(conf))
 output=stack.enter_context(log.open('w'))
 authority=subprocess.Popen(['named','-g','-c',str(conf),'-n','1','-U','1'],stdout=output,stderr=subprocess.STDOUT)
 stack.callback(stop,authority)
 def ready():
 assert authority.poll() is None,log.read_text()
 response=query('lab.test.','SOA');return response.rcode()==0 and response.answer[0][0].serial==serial
 wait(ready)
 def update(body):
 nonlocal serial
 serial+=1;save(body);authority.send_signal(signal.SIGHUP);wait(ready)
 def cache(port=1054,upstream=1053,size=150,options=()):
 logfile=root/('cache-'+str(port)+'.log');stream=stack.enter_context(logfile.open('w'))
 command=['dnsmasq','--no-daemon','--conf-file=/dev/null','--no-resolv','--no-hosts','--bind-interfaces',
 '--listen-address=127.0.0.1','--port='+str(port),'--server=127.0.0.1#'+str(upstream),
 '--cache-size='+str(size),'--log-queries','--log-facility=-',*options]
 process=subprocess.Popen(command,stdout=stream,stderr=subprocess.STDOUT);stack.callback(stop,process)
 def cache_ready():
 assert process.poll() is None,logfile.read_text()
 return query('lab.test.','SOA',port).rcode()==0
 wait(cache_ready);return process,logfile
 def queries(name,kind='A'):
 return sum(bool(re.search(r'query: '+re.escape(name.rstrip('.'))+r' IN '+kind+r'\b',line)) for line in log.read_text().splitlines())
 yield root,update,cache,queries
import dns.flags,socket,struct,threading
texts=[f'{i}-'+('x'*200) for i in range(4)]
records='small 30 IN A 127.0.0.2\n'+'\n'.join('large 30 IN TXT "'+s+'"' for s in texts)
def exact(stream,size):
 data=b''
 while len(data)<size:
 part=stream.recv(size-len(data));assert part,'unexpected TCP EOF';data+=part
 return data
def description(response):return {'wire_bytes':len(response.to_wire()),'TC':bool(response.flags&dns.flags.TC),'answer_records':sum(len(rr) for rr in response.answer)}
with lab(records) as (_,update,cache,queries):
 q=dns.message.make_query('large.lab.test.','TXT',use_edns=False)
 udp=dns.query.udp(q,'127.0.0.1',port=1053,timeout=2)
 assert udp.flags&dns.flags.TC and len(udp.to_wire())<=512
 tcp=dns.query.tcp(q,'127.0.0.1',port=1053,timeout=2)
 assert not tcp.flags&dns.flags.TC and set(values(tcp,'TXT'))=={'"'+s+'"' for s in texts}
 assert len(tcp.to_wire())>512
 with socket.create_connection(('127.0.0.1',1053),timeout=2) as stream:
 lengths=[]
 for name,kind in [('large.lab.test.','TXT'),('small.lab.test.','A')]:
 request=dns.message.make_query(name,kind);wire=request.to_wire();stream.sendall(struct.pack('!H',len(wire))+wire)
 size=struct.unpack('!H',exact(stream,2))[0];reply=dns.message.from_wire(exact(stream,size));assert reply.id==request.id and reply.rcode()==0
 assert values(reply,kind);lengths.append(size)
 with socket.socket() as reservation,socket.socket(socket.AF_INET,socket.SOCK_DGRAM) as relay:
 reservation.bind(('127.0.0.1',0));port=reservation.getsockname()[1];relay.bind(('127.0.0.1',port));relay.settimeout(3)
 errors=[]
 def one_datagram():
 try:
 data,peer=relay.recvfrom(4096);answer=dns.query.udp(dns.message.from_wire(data),'127.0.0.1',port=1053,timeout=2);relay.sendto(answer.to_wire(),peer)
 except Exception as error:errors.append(type(error).__name__)
 thread=threading.Thread(target=one_datagram);thread.start()
 limited=dns.query.udp(q,'127.0.0.1',port=port,timeout=2);assert limited.flags&dns.flags.TC
 thread.join(timeout=4);assert not thread.is_alive() and not errors
 try:dns.query.tcp(q,'127.0.0.1',port=port,timeout=1);raise AssertionError('UDP-only endpoint unexpectedly accepted TCP')
 except OSError as error:failure=type(error).__name__
 print(json.dumps({'UDP':description(udp),'TCP':description(tcp),'two_messages_one_connection_lengths':lengths,
 'UDP_only_path':{'TC_received':True,'TCP_error':failure}},sort_keys=True))
"""
name='xbcatm-dns-'+uuid.uuid4().hex[:10]
def docker(*args,**kwargs):return subprocess.run(['docker',*args],capture_output=True,text=True,timeout=300,**kwargs)
try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'Dockerfile').write_text(recipe)
 built=docker('build','-t',name,str(root));assert built.returncode==0,built.stderr
 result=docker('run','--rm','-i','--name',name,'--network','none','--memory','384m','--pids-limit','128',name,
 '/usr/bin/python3','-',input=program)
 assert result.returncode==0,result.stderr
 print(json.dumps(json.loads(result.stdout),sort_keys=True))
finally:
 docker('rm','-f',name);docker('image','rm',name)

The historical UDP result reserializes to 462 bytes, has TC set, and contains only two of the four TXT records. The TCP response is 892 bytes, has TC clear, and contains all four expected values. The script compares complete record contents. Its wire_bytes fields use len(response.to_wire()): reserialized DNS-message lengths, not captured packet lengths. The later reused-stream lengths instead come directly from TCP framing.

The program next opens one TCP connection and sends two DNS queries through it, first for the large TXT answer and then for a small A answer. It reads each two-byte length prefix and the corresponding message body, obtaining message lengths 892 and 48 in this run. It also checks response IDs. The same stream serves both exchanges, demonstrating that a fresh handshake is not required for every query.

A separate endpoint forwards one UDP question to the real authority and returns its truncated response. On that same endpoint number, a reserved but non-listening TCP socket prevents a TCP listener from existing. The UDP query succeeds with TC, while the TCP attempt fails with ConnectionRefusedError. This reproduces the transport mismatch without changing firewall rules or relying on an unrelated public resolver.

Diagnose truncation and connection failure separately

Check the response flags, advertised EDNS capacity if present, TCP connection result, and final DNS contents. A TC response followed by a refused connection has a different cause from an unanswered UDP packet or an authoritative NXDOMAIN. The relay is intentionally limited to one exchange; it is a controlled failure fixture, not a general DNS proxy implementation intended for deployment.

Permit the required TCP path through the client, resolver, server listener, and intervening access rules. Keep connection deadlines and resource limits appropriate to the resolver workload. Reuse can reduce repeated setup work, but idle connections, connection limits, and server behavior still matter. Do not convert the extra handshake observed for one new connection into a fixed latency surcharge for every TCP lookup.

EDNS can allow a larger UDP response and avoid truncation in many cases, but it does not eliminate the need for TCP. Larger datagrams can encounter path-MTU and fragmentation problems before the client ever receives a useful TC signal. Test the configured UDP size alongside a complete TCP exchange, and avoid treating an unusually large advertised payload as a substitute for a working second transport.

Finally, TCP provides a reliable byte stream, not DNS confidentiality or authenticated record data. Plain DNS over TCP remains plaintext. TLS-based DNS transport and DNSSEC address separate security properties. Keep those claims separate from this experiment, whose evidence establishes truncation handling, complete record recovery, stream framing, connection reuse, and a concrete failure when only UDP is available.

Sources

dnspython Message API, to_wire serializes DNS messages; the unchanged fixture measures its returned bytes, checked 2026-10-18.
RFC 7766 DNS over TCP, TCP support, connection reuse, and DNS transport requirements, checked 2026-09-12.
RFC 1035 domain names, TC, traditional UDP size, and two-byte TCP framing, checked 2026-09-12.
RFC 5936 AXFR, zone transfers spanning multiple DNS messages, checked 2026-09-12.

IPv6 resolves neighbors with ICMPv6 before the TCP connection can work

Mon, 16 Nov 2026 00:00:00 +0000

An IPv6 host on an Ethernet link needs a link-layer destination before it can send a frame to a neighbor. Neighbor Discovery performs that address-resolution work using ICMPv6 Neighbor Solicitation and Neighbor Advertisement messages. It also covers functions beyond IPv4 ARP, including router discovery and neighbor reachability detection. Calling it merely “ARP with longer addresses” hides dependencies that matter when configuring an IPv6 network.

When it breaks

A valid IPv6 address and route do not prove that the next hop’s Ethernet address can be resolved. If the required solicitation or advertisement is filtered, a TCP service can be listening correctly while clients fail before exchanging application data. Opening only the application’s TCP port does not repair this lower-layer dependency. Inspect the neighbor table and relevant ICMPv6 traffic alongside the service listener.

Cached state can conceal the error. A host with a usable neighbor entry may continue communicating after a damaging rule is installed, while a fresh client or an expired entry exposes the failure. Conversely, a STALE entry is not automatically broken: Neighbor Discovery can use existing link-layer information and subsequently check reachability. Interpret the state machine rather than requiring every healthy entry to remain REACHABLE forever.

Another common confusion is resolving the remote application’s address on the local Ethernet segment. For an off-link destination, the host normally resolves its selected next-hop router. Confirm routing before deciding which neighbor should appear in the table. A capture on the wrong interface, or a cached entry requiring no new solicitation, can also explain an empty trace without proving that discovery is absent.

Conceptual ICMPv6 dependency. Neighbor resolution neither authenticates ownership nor by itself tests SLAAC, Router Advertisement processing or Duplicate Address Detection. No network lab was rerun for this drawing.

Check it quickly

This retained lab was previously reported with Python 3.11.16, iproute2 6.1.0, nftables 1.0.6 and tcpdump 4.99.3; it was not rerun here. Save it as check-neighbor-discovery.py only for a separately authorized disposable Linux VM. Containers share its kernel: NET_ADMIN, SYS_ADMIN, proc mounting and namespace firewall changes require that isolated boundary. The build downloads packages; --network none applies only to the later container run. Keep assertions enabled and never relax host security to bypass setup failure. The veth, neighbor and firewall changes are intended to remain inside the lab:

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,uuid
recipe='FROM python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84\nRUN apt-get update && apt-get install -y --no-install-recommends iproute2=6.1.0-3 iputils-ping=3:20221126-1+deb12u1 nftables=1.0.6-2+deb12u2 tcpdump=4.99.3-1 traceroute=1:2.1.2-1 procps=2:4.0.2-3 && rm -rf /var/lib/apt/lists/*\nCMD ["python3"]\n'
program=r"""
from contextlib import ExitStack,contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import json,os,select,signal,subprocess,time

def run(*args,check=True,input=None,timeout=8):
 result=subprocess.run(args,input=input,capture_output=True,text=True,timeout=timeout)
 if check:assert result.returncode==0,(args,result.returncode,result.stdout,result.stderr)
 return result
def ns(name,*args,**kwargs):return run('ip','netns','exec',name,*args,**kwargs)
def ip(name,*args,**kwargs):return ns(name,'ip',*args,**kwargs)
def stop(process):
 if process.poll() is None:
 process.terminate()
 try:process.wait(timeout=2)
 except subprocess.TimeoutExpired:process.kill();process.wait(timeout=2)
def line(process,timeout=3):
 assert select.select([process.stdout],[],[],timeout)[0],'child response deadline exceeded'
 value=process.stdout.readline();assert value,'child exited without a response'
 return value
def spawn(stack,command):
 process=subprocess.Popen(command,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE,text=True)
 stack.callback(stop,process);return process
@contextmanager
def lab(names):
 with TemporaryDirectory(prefix='net-path-') as directory,ExitStack() as stack:
 root=Path(directory);proc=root/'proc';proc.mkdir();run('mount','-t','proc','proc',str(proc))
 stack.callback(run,'umount',str(proc))
 for name in names:
 run('ip','netns','add',name);stack.callback(run,'ip','netns','delete',name,check=False)
 ip(name,'link','set','lo','up')
 def setting(name,key,value):
 assert name in names
 path=proc/'sys'/key.replace('.','/')
 return ns(name,'python3','-c',f'from pathlib import Path; p=Path({str(path)!r}); p.write_text({str(value)!r}); print(p.read_text().strip())').stdout.strip()
 yield root,stack,setting
def wire(left,left_if,right,right_if,left_addresses,right_addresses):
 run('ip','link','add',left_if,'type','veth','peer','name',right_if)
 for name,interface,addresses in [(left,left_if,left_addresses),(right,right_if,right_addresses)]:
 run('ip','link','set',interface,'netns',name)
 for address in addresses:ip(name,'address','add',address,'dev',interface,*(['nodad'] if ':' in address else []))
 ip(name,'link','set',interface,'up')
def nft(name,script):return ns(name,'nft','-f','-',input=script)
def counters(name):
 data=json.loads(ns(name,'nft','-j','list','ruleset').stdout);found={}
 for entry in data['nftables']:
 rule=entry.get('rule',{})
 if 'comment' in rule:
 for expression in rule['expr']:
 if 'counter' in expression:found[rule['comment']]=expression['counter']['packets']
 return found

def server(stack,name,address,port=18080,label='server'):
 code=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
import json,socket,sys
address,port,label=sys.argv[1],int(sys.argv[2]),sys.argv[3]
class Handler(BaseHTTPRequestHandler):
 def do_GET(self):
 body=json.dumps({'server':label,'peer':self.client_address,'path':self.path}).encode()
 self.send_response(200);self.send_header('Content-Length',str(len(body)));self.end_headers();self.wfile.write(body)
 def log_message(self,*args):pass
class Server(ThreadingHTTPServer):
 address_family=socket.AF_INET6 if ':' in address else socket.AF_INET
 def server_bind(self):
 if self.address_family==socket.AF_INET6:self.socket.setsockopt(socket.IPPROTO_IPV6,socket.IPV6_V6ONLY,1)
 super().server_bind()
service=Server((address,port),Handler);print('ready',flush=True);service.serve_forever()
'''
 process=spawn(stack,['ip','netns','exec',name,'python3','-u','-c',code,address,str(port),label])
 assert line(process).strip()=='ready';return process
def request(name,address,port=18080):
 code=r'''
import http.client,json,sys
client=http.client.HTTPConnection(sys.argv[1],int(sys.argv[2]),timeout=.7)
try:
 client.request('GET','/lab-visible-message');response=client.getresponse();body=json.loads(response.read())
 assert response.status==200;print(json.dumps({'ok':True,'status':response.status,'body':body}))
except OSError as error:print(json.dumps({'ok':False,'type':type(error).__name__,'errno':error.errno}))
finally:client.close()
'''
 return json.loads(ns(name,'python3','-c',code,address,str(port)).stdout)

def capture(stack,root,name,interface,expression,label):
 path=root/(label+'.pcap')
 process=subprocess.Popen(['ip','netns','exec',name,'tcpdump','--immediate-mode','-Z','root','-U','-n','-i',interface,'-w',str(path),expression],
 stdout=subprocess.DEVNULL,stderr=subprocess.PIPE,text=True)
 stack.callback(stop,process)
 deadline=time.monotonic()+3
 while time.monotonic()<deadline:
 assert process.poll() is None,'tcpdump exited before listening'
 if select.select([process.stderr],[],[],.1)[0] and 'listening on' in process.stderr.readline():return process,path
 raise AssertionError('tcpdump did not become ready')
def captured(process,path,ascii=False):
 process.send_signal(signal.SIGINT);_,diagnostic=process.communicate(timeout=3)
 assert process.returncode==0,diagnostic
 result=run('tcpdump','-nn','-vv',*(['-A'] if ascii else []),'-r',str(path))
 return result.stdout

with lab(['client','target']) as (root,stack,setting):
 wire('client','c0','target','t0',['192.0.2.1/30','2001:db8:1::1/64'],['192.0.2.2/30','2001:db8:1::2/64'])
 server(stack,'target','2001:db8:1::2')
 ip('client','-4','neigh','flush','dev','c0');ip('target','-4','neigh','flush','dev','t0')
 arp_process,arp_file=capture(stack,root,'client','c0','arp','arp')
 ns('client','ping','-4','-n','-c','1','-W','1','192.0.2.2')
 arp=captured(arp_process,arp_file);assert 'ARP,' in arp and 'Request who-has' in arp and 'Reply' in arp,arp
 ip('client','-6','neigh','flush','dev','c0');ip('target','-6','neigh','flush','dev','t0')
 nd_process,nd_file=capture(stack,root,'client','c0','icmp6 or arp','nd')
 ns('client','ping','-6','-n','-c','1','-W','1','2001:db8:1::2')
 nd=captured(nd_process,nd_file)
 assert 'neighbor solicitation' in nd and 'neighbor advertisement' in nd and 'ARP,' not in nd,nd
 mac=json.loads(ip('target','-j','link','show','t0').stdout)[0]['address']
 neighbors=json.loads(ip('client','-6','-j','neigh','show','2001:db8:1::2').stdout)
 assert len(neighbors)==1 and neighbors[0]['lladdr']==mac
 healthy=request('client','2001:db8:1::2');assert healthy['ok']
 setting('client','net.ipv6.neigh.c0.retrans_time_ms',200);setting('client','net.ipv6.neigh.c0.mcast_solicit',3)
 nft('client','''table inet block_nd {
 chain input { type filter hook input priority 0; policy accept;
 icmpv6 type nd-neighbor-advert counter drop comment "blocked_advertisements"
 }
}
''')
 ip('client','-6','neigh','flush','dev','c0')
 failed=ns('client','ping','-6','-n','-c','1','-W','1','2001:db8:1::2',check=False);assert failed.returncode!=0
 broken=request('client','2001:db8:1::2');assert not broken['ok']
 blocked=counters('client');assert blocked['blocked_advertisements']>0
 failed_neighbors=json.loads(ip('client','-6','-j','neigh','show','2001:db8:1::2').stdout)
 assert failed_neighbors and all('lladdr' not in entry for entry in failed_neighbors)
 nft('client','delete table inet block_nd');ip('client','-6','neigh','flush','dev','c0')
 recovered=request('client','2001:db8:1::2');assert recovered['ok']
 print(json.dumps({'ipv4_arp_capture':arp,'ipv6_capture':nd,'resolved_neighbor':neighbors,'target_mac':mac,
 'healthy_http':healthy,'blocked_ping_exit':failed.returncode,'blocked_http':broken,'blocked_counters':blocked,
 'unresolved_neighbor':failed_neighbors,'recovered_http':recovered}))
"""
name='xbcatm-netpath-'+uuid.uuid4().hex[:10]
def docker(*args,**kwargs):return subprocess.run(['docker',*args],capture_output=True,text=True,timeout=300,**kwargs)
try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'Dockerfile').write_text(recipe)
 build=docker('build','-t',name,str(root));assert build.returncode==0,build.stderr
 result=docker('run','--rm','-i','--name',name,'--network','none','--cap-add','NET_ADMIN','--cap-add','SYS_ADMIN',
 '--memory','256m','--pids-limit','128',name,'python3','-',input=program)
 assert result.returncode==0,result.stderr
 print(json.dumps(json.loads(result.stdout),sort_keys=True))
finally:
 docker('rm','-f',name);docker('image','rm',name)

The script assigns IPv4 and IPv6 documentation addresses to both endpoints. It first captures a real IPv4 ARP request and reply as a positive control for the capture setup. It then clears the client’s IPv6 neighbor cache and sends an IPv6 echo request. The trace contains Neighbor Solicitation, Neighbor Advertisement, and echo traffic, with no ARP frames in this IPv6 exchange.

The solicitation for 2001:db8:1::2 is sent to solicited-node multicast address ff02::1:ff00:2. The reply provides the target’s link-layer address, which the script compares with the target interface’s actual MAC address and the client’s neighbor entry. An HTTP request then records the server body; the fixture asserts status 200 and success, not the body identity. This joins the control exchange, learned information, and application outcome instead of relying on configuration text alone.

Next, the client drops incoming neighbor advertisements. Its cache is cleared so an older mapping cannot hide the change. Discovery retransmission settings are shortened only inside the client namespace to keep the test bounded. The historical report records ping failure, HTTP EHOSTUNREACH (errno 113), and FAILED neighbor state. The code asserts failed requests and absence of a link-layer address, not that exact errno or state label. The filter counter confirms that advertisements reached the drop rule.

Removing that rule and clearing the failed entry restores HTTP success. The target service and address stay unchanged throughout the comparison. This isolates address-resolution failure from a stopped application or a missing TCP allow rule. The exact timing and packet count depend on the configured discovery retries, so the example checks the failure state and recovery rather than prescribing one universal timeout.

Keep discovery, addressing, and authentication separate

The fixture uses static IPv6 addresses with duplicate-address detection disabled for setup. It demonstrates neighbor resolution, not SLAAC, Router Advertisement processing, or Duplicate Address Detection. Those functions require their own observations and policy review. Passing this small test cannot certify every ICMPv6 dependency in a production network; RFC 4890 discusses filtering requirements in broader operational contexts.

An advertisement also should not be treated as authenticated proof of ownership merely because it supplies a MAC address. The protocol’s local-link checks limit where messages can originate but do not make every node on that link trustworthy. When debugging, record interface, selected next hop, neighbor state, filters, and the corresponding application result. That evidence distinguishes an unresolved neighbor from an unrelated service failure.

Sources

RFC 4861 Neighbor Discovery, solicitation, advertisement, address resolution and reachability states, checked 2026-10-15.
Linux ip-neighbour(8), inspecting and flushing IPv6 neighbor entries, checked 2026-10-15.
RFC 4890 ICMPv6 filtering, preserving ICMPv6 functions required by IPv6 networks, checked 2026-10-15.

UDP preserves datagrams; recovery and timing belong to the application

Wed, 11 Nov 2026 00:00:00 +0000

UDP carries discrete datagrams between transport endpoints. Its eight-byte header contains source port, destination port, length, and checksum fields. The protocol does not add TCP’s reliable ordered byte stream or its congestion-control behavior. This gives applications room to choose their own delivery policy, but it does not promise lower latency for every workload. A missing datagram can be more costly than a handshake when the application needs its contents.

The source sends A, BC and an empty datagram, receiving lengths 1, 2 and 0. Eight bytes abcdefgh read with a 3-byte buffer produce abc plus MSG_TRUNC; defgh is discarded, a subsequent receive times out, and a new next datagram succeeds. A deliberately programmed relay changes 1,2,3 to 3,1,1; set deduplication leaves 3,1 without recovering 2 or restoring order. Source exercise illustration, not rerun; the relay is fault injection, not a network-loss statistic or latency guarantee.

When it breaks

A successful send does not prove that the destination application received or processed a message. UDP supplies no built-in acknowledgment, retransmission, duplicate suppression, or ordering guarantee. A timeout therefore cannot tell a payment client whether an operation happened. If retries are allowed, the application needs a protocol that resolves repeated requests and uncertain outcomes rather than blindly interpreting silence as failure.

Receive buffers create another failure mode even on a healthy local path. A datagram larger than the application’s supplied buffer can be truncated. The missing suffix is not kept for the next stream-style read. Linux can report MSG_TRUNC through recvmsg, allowing the application to detect this condition. Accepting the shortened bytes as a complete command or measurement can silently corrupt meaning.

UDP’s lack of built-in congestion control is a responsibility, not permission to send without limits. Applications must choose appropriate pacing, congestion behavior, message sizes, and deadlines for their deployment. Fragmentation can make a large datagram less robust, and obsolete data may be worthless after a deadline. A protocol such as QUIC can provide reliable streams over UDP; the carrier alone does not determine application reliability.

Check it quickly

Save this as check-udp.py and run python3 check-udp.py with Docker. The pinned Python 3.11.16 Linux container has no external network path or published ports. The first part exercises actual socket receive semantics. The second uses a deliberately programmed relay to withhold, repeat, and reorder messages; it does not claim to measure spontaneous packet loss:

The environment and numerical outcomes below are inherited article reports, not independently reproduced here. The image does not pin the host kernel or package availability.

import json,subprocess,uuid
image="python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84"
program=r"""
import json,socket

with socket.socket(socket.AF_INET,socket.SOCK_DGRAM) as sender,socket.socket(socket.AF_INET,socket.SOCK_DGRAM) as receiver:
 receiver.bind(('127.0.0.1',0));receiver.settimeout(.25);sender.settimeout(1);target=receiver.getsockname()
 sender.connect(target)
 for body in [b'A',b'BC',b'']:sender.send(body)
 boundaries=[receiver.recvfrom(16)[0] for _ in range(3)]
 assert boundaries==[b'A',b'BC',b'']
 sender.send(b'abcdefgh');data,ancillary,flags,peer=receiver.recvmsg(3)
 assert data==b'abc' and flags&socket.MSG_TRUNC
 try:receiver.recvfrom(16);raise AssertionError('truncated suffix survived')
 except TimeoutError:tail='discarded'
 sender.send(b'next');assert receiver.recvfrom(16)[0]==b'next'

 # A deliberate application relay changes delivery; this is not measured network loss.
 received=[];applied=[];seen=set();forwarded=[]
 with socket.socket(socket.AF_INET,socket.SOCK_DGRAM) as relay,socket.socket(socket.AF_INET,socket.SOCK_DGRAM) as output:
 relay.bind(('127.0.0.1',0));relay.settimeout(1);output.settimeout(1)
 sender.connect(relay.getsockname())
 for seq in [1,2,3]:sender.send(str(seq).encode())
 inputs=[relay.recvfrom(16)[0] for _ in range(3)]
 for index in [2,0,0]:
 output.sendto(inputs[index],target);forwarded.append(int(inputs[index]))
 value=int(receiver.recvfrom(16)[0]);received.append(value)
 if value not in seen:seen.add(value);applied.append(value)
 assert received==[3,1,1] and applied==[3,1] and 2 not in seen
 try:receiver.recvfrom(16);raise AssertionError('missing message arrived by itself')
 except TimeoutError:no_retransmit=True
 print(json.dumps({'datagram_lengths':[len(x) for x in boundaries],'empty_datagram_is_message':boundaries[2]==b'',
 'short_buffer':data.decode(),'truncation_flag':bool(flags&socket.MSG_TRUNC),'unread_tail':tail,
 'relay_inputs':[int(x) for x in inputs],'relay_outputs':forwarded,'receiver':received,'deduplicated':applied,
 'missing_sequence':2,'no_automatic_retransmission_during_deadline':no_retransmit}))
"""
name='xbcatm-socket-'+uuid.uuid4().hex[:10]
try:
 result=subprocess.run(['docker','run','--rm','-i','--name',name,'--network','none',
 '--memory','128m','--pids-limit','64',image,'python3','-'],input=program,
 capture_output=True,text=True,timeout=60)
 assert result.returncode==0,result.stderr
 print(json.dumps(json.loads(result.stdout),ensure_ascii=False,sort_keys=True))
finally:
 subprocess.run(['docker','rm','-f',name],capture_output=True,text=True,timeout=10)

Three sends contain A, BC, and an empty payload. Three receives return lengths one, two, and zero. Each receive consumes one datagram rather than combining adjacent sends into a stream. The empty result is a valid zero-length message, not evidence that the sender closed a connection. The sender uses UDP connect to select its peer; that API call does not turn UDP into TCP.

Next, the sender transmits eight bytes, abcdefgh, while the receiver supplies a three-byte buffer. It obtains abc and MSG_TRUNC. A subsequent receive times out instead of returning defgh; after a new next datagram is sent, that complete new message arrives. This positive control distinguishes discarded data from a receiver that simply stopped working after its first call.

The relay receives sequence identifiers 1, 2, and 3. It deliberately forwards 3, then 1, then another 1, and omits 2. The downstream socket actually receives that chosen sequence. A small application set suppresses the duplicate, leaving applied identifiers 3 and 1, but it neither restores ordering nor reconstructs the absent message. Duplicate handling solves only one of these separate problems.

Afterward, the receiver waits for another datagram and reaches its deadline. UDP has not automatically retransmitted identifier 2. The relay, rather than the kernel, manufactured this omission, so the observation should not be described as a network reliability statistic. It is a reproducible way to test how the receiving application reacts to delivery patterns the transport does not rule out.

Choose the missing semantics deliberately

A live measurement may discard an old sample and display the newest sequence, while a file transfer needs missing data recovered. An application acknowledgment can report receipt or a durable operation result, but those meanings must be specified. Define identifier scope, retry deadlines, duplicate retention, and restart behavior before using retries for actions with side effects. This fixture implements none of that broader transaction protocol.

Budget receive capacity and datagram size together, and record truncation or application-level loss separately from malformed payloads. A larger socket queue cannot repair a receive call whose buffer is too short for one message. Likewise, an increased send rate may worsen congestion rather than improve freshness. Measure useful messages delivered before their deadlines, not just bytes accepted by the local send call.

UDP is useful when its message model fits the protocol being built and the missing guarantees have explicit owners. Assess actual end-to-end timing, failure recovery, and resource use before choosing it for a latency claim. Preserved boundaries simplify some parsing, but successful application behavior still depends on what happens when a complete datagram never arrives, arrives twice, or arrives too late.

Sources

RFC 768 User Datagram Protocol, UDP header and delivery properties, checked 2026-09-12.
RFC 8085 UDP usage guidelines, congestion control, message sizing and application responsibilities, checked 2026-09-12.
Linux udp(7), datagram reception, truncation and connected UDP sockets, checked 2026-09-12.

EDNS advertises UDP capacity without guaranteeing path delivery

Wed, 28 Oct 2026 00:00:00 +0000

EDNS extends DNS through an OPT pseudo-record. Among other information, a request can advertise the UDP payload size its sender is prepared to receive. This allows answers beyond the traditional 512-byte DNS limit. It is an advertisement within a DNS exchange, not a network path-MTU discovery protocol or a promise that every intervening device will deliver a datagram of that size.

The illustrated values are the article’s explicit loopback configuration and recorded DNS-message sizes, not universal defaults or a path-MTU measurement. OPT advertises capacity; the client must interpret TC and retain TCP access.

When it breaks

A client advertising 4096 bytes does not compel the server to send a 4096-byte response. The server can apply a smaller outgoing limit. If the required answer does not fit, it can return TC and the client still needs a working TCP path. Increasing the advertised number cannot fix a server policy that deliberately uses a smaller response budget.

Conversely, a response that fits the advertised capacity can still be lost along the path. Fragmentation, tunnels, packet filters, and local network constraints are separate from what the DNS participants advertise. If the client receives no response, it may never see a TC signal. Treat payload settings as an interoperability choice to verify, not a universal performance knob whose maximum value is always best.

EDNS also carries information that must not be mistaken for ordinary cached DNS data. OPT is a pseudo-record describing the exchange. Its fields encode such things as payload capacity, version, extended response code, and flags; its TTL-shaped field is not a normal record lifetime. The DNSSEC OK flag, DO, asks for relevant DNSSEC records but does not prove that anyone validated the answer.

Check it quickly

Save this as check-edns.py and run python3 check-edns.py with Docker. The pinned BIND 9.18.49 authority runs on isolated loopback networking. Its edns-udp-size and max-udp-size are explicitly set to 1232. The zone contains a medium TXT set with four records and a larger set with eight, allowing capacity and truncation to be observed independently:

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,uuid
recipe='FROM python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84\nRUN apt-get update && apt-get install -y --no-install-recommends bind9=1:9.18.49-1~deb12u2 bind9-dnsutils=1:9.18.49-1~deb12u2 dnsmasq-base=2.90-4~deb12u2 python3-dnspython=2.3.0-1 && rm -rf /var/lib/apt/lists/*\nCMD ["/usr/bin/python3"]\n'
program=r"""
from contextlib import ExitStack,contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import json,re,signal,subprocess,time
import dns.exception,dns.message,dns.query,dns.rcode,dns.rdatatype

def run(*args,check=True):
 result=subprocess.run(args,capture_output=True,text=True,timeout=15)
 if check:assert result.returncode==0,(args,result.returncode,result.stdout,result.stderr)
 return result
def query(name,kind='A',port=1053):
 return dns.query.udp(dns.message.make_query(name,kind),'127.0.0.1',port=port,timeout=1)
def values(response,kind='A'):
 wanted=dns.rdatatype.from_text(kind)
 return [item.to_text() for rrset in response.answer if rrset.rdtype==wanted for item in rrset]
def show(response):
 def section(records):return [{'name':str(rr.name),'type':dns.rdatatype.to_text(rr.rdtype),'ttl':rr.ttl,'data':[item.to_text() for item in rr]} for rr in records]
 return {'rcode':dns.rcode.to_text(response.rcode()),'answer':section(response.answer),'authority':section(response.authority)}
def wait(predicate,timeout=15):
 deadline=time.monotonic()+timeout
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except (OSError,dns.exception.Timeout):pass
 time.sleep(.05)
 raise AssertionError('DNS fixture condition did not become true')
def stop(process):
 if process.poll() is None:
 process.terminate()
 try:process.wait(timeout=5)
 except subprocess.TimeoutExpired:process.kill();process.wait(timeout=5)

@contextmanager
def lab(records,soa_ttl=8,minimum=4,extra=''):
 with TemporaryDirectory(prefix='xbcatm-dns-') as directory,ExitStack() as stack:
 root=Path(directory);zone=root/'zone';conf=root/'named.conf';log=root/'named.log';serial=1
 def save(body):
 zone.write_text(f'$ORIGIN lab.test.\n$TTL 30\n@ {soa_ttl} IN SOA ns.lab.test. hostmaster.lab.test. ({serial} 60 60 3600 {minimum})\n@ IN NS ns.lab.test.\nns IN A 127.0.0.1\n'+body+'\n')
 run('named-checkzone','lab.test',str(zone))
 save(records)
 conf.write_text(f'''options {{ directory "{root}"; listen-on port 1053 {{127.0.0.1;}}; listen-on-v6 {{none;}};
recursion no; dnssec-validation no; notify no; querylog yes; max-cache-size 8m; pid-file "{root}/named.pid"; session-keyfile none;
{extra} }};
controls {{}};
zone "lab.test" {{ type primary; file "{zone}"; }};
''')
 run('named-checkconf',str(conf))
 output=stack.enter_context(log.open('w'))
 authority=subprocess.Popen(['named','-g','-c',str(conf),'-n','1','-U','1'],stdout=output,stderr=subprocess.STDOUT)
 stack.callback(stop,authority)
 def ready():
 assert authority.poll() is None,log.read_text()
 response=query('lab.test.','SOA');return response.rcode()==0 and response.answer[0][0].serial==serial
 wait(ready)
 def update(body):
 nonlocal serial
 serial+=1;save(body);authority.send_signal(signal.SIGHUP);wait(ready)
 def cache(port=1054,upstream=1053,size=150,options=()):
 logfile=root/('cache-'+str(port)+'.log');stream=stack.enter_context(logfile.open('w'))
 command=['dnsmasq','--no-daemon','--conf-file=/dev/null','--no-resolv','--no-hosts','--bind-interfaces',
 '--listen-address=127.0.0.1','--port='+str(port),'--server=127.0.0.1#'+str(upstream),
 '--cache-size='+str(size),'--log-queries','--log-facility=-',*options]
 process=subprocess.Popen(command,stdout=stream,stderr=subprocess.STDOUT);stack.callback(stop,process)
 def cache_ready():
 assert process.poll() is None,logfile.read_text()
 return query('lab.test.','SOA',port).rcode()==0
 wait(cache_ready);return process,logfile
 def queries(name,kind='A'):
 return sum(bool(re.search(r'query: '+re.escape(name.rstrip('.'))+r' IN '+kind+r'\b',line)) for line in log.read_text().splitlines())
 yield root,update,cache,queries
import dns.flags
records='\n'.join(f'{name} 30 IN TXT "{i}-'+('x'*200)+'"' for name,count in [('medium',4),('large',8)] for i in range(count))
def ask(name,payload=None,version=0,flags=0):
 q=dns.message.make_query(name+'.lab.test.','TXT',use_edns=False)
 if payload is not None:q.use_edns(edns=version,ednsflags=flags,payload=payload)
 return dns.query.udp(q,'127.0.0.1',port=1053,timeout=2)
def description(response):return {'wire_bytes':len(response.to_wire()),'TC':bool(response.flags&dns.flags.TC),
 'rcode':dns.rcode.to_text(response.rcode()),'EDNS_version':response.edns,'advertised_payload':response.payload,
 'DO':bool(response.ednsflags&dns.flags.DO),'AD':bool(response.flags&dns.flags.AD),'answer_records':sum(len(rr) for rr in response.answer)}
with lab(records,extra='edns-udp-size 1232; max-udp-size 1232;') as (_,update,cache,queries):
 classic=ask('medium');extended=ask('medium',1232);capped=ask('large',4096);do=ask('medium',1232,flags=dns.flags.DO);version=ask('medium',1232,version=1)
 assert classic.flags&dns.flags.TC and len(classic.to_wire())<=512 and classic.edns==-1
 assert not extended.flags&dns.flags.TC and 512<len(extended.to_wire())<=1232 and len(values(extended,'TXT'))==4
 assert extended.edns==0 and extended.opt is not None
 assert capped.flags&dns.flags.TC and len(capped.to_wire())<=1232
 assert do.ednsflags&dns.flags.DO and not do.flags&dns.flags.AD
 assert not any(rr.rdtype==dns.rdatatype.RRSIG for rr in do.answer)
 assert version.rcode()==dns.rcode.BADVERS and version.edns==0
 full=dns.query.tcp(dns.message.make_query('large.lab.test.','TXT'),'127.0.0.1',port=1053,timeout=2)
 assert len(values(full,'TXT'))==8 and len(full.to_wire())>1232
 print(json.dumps({'without_EDNS':description(classic),'EDNS_1232':description(extended),'client_4096_server_1232':description(capped),
 'DO_on_unsigned_zone':description(do),'unsupported_version':description(version),'TCP_full':description(full)},sort_keys=True))
"""
name='xbcatm-dns-'+uuid.uuid4().hex[:10]
def docker(*args,**kwargs):return subprocess.run(['docker',*args],capture_output=True,text=True,timeout=300,**kwargs)
try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'Dockerfile').write_text(recipe)
 built=docker('build','-t',name,str(root));assert built.returncode==0,built.stderr
 result=docker('run','--rm','-i','--name',name,'--network','none','--memory','384m','--pids-limit','128',name,
 '/usr/bin/python3','-',input=program)
 assert result.returncode==0,result.stderr
 print(json.dumps(json.loads(result.stdout),sort_keys=True))
finally:
 docker('rm','-f',name);docker('image','rm',name)

Without EDNS, the medium query returns a 463-byte truncated response containing two TXT records. With an advertised payload of 1232, it returns all four records in 904 bytes with TC clear. The script checks membership count, reserialized message length, and EDNS presence. Its wire_bytes fields use len(response.to_wire()), not a captured datagram or IP-packet length; reserialization is not a byte-for-byte capture guarantee. This shows a concrete answer moving past the traditional limit while staying inside the selected UDP response budget.

For the larger set, the client advertises 4096 but the server remains capped at 1232. The recorded response is only 43 bytes with TC set and no answer records. A subsequent TCP query retrieves all eight records in 1752 bytes. A truncated reply need not fill the entire allowed buffer; record grouping and response construction can produce a much smaller signal that more data is needed.

The program also sets DO on a query to this unsigned zone. The response preserves DO, but has no RRSIG and no authenticated-data AD flag. That combination is expected: requesting DNSSEC-related data cannot manufacture signatures or a chain of trust. The example records the flags separately rather than labeling a DO-bearing exchange as a successful DNSSEC validation.

Finally, a query advertises EDNS version 1. This server returns BADVERS and indicates supported version 0. The library decodes the extended response code from the complete message. This negative test distinguishes unsupported EDNS version from an absent name or an ordinary successful empty answer, and demonstrates why diagnostics must retain OPT information when interpreting extension-related failures.

Measure the configured exchange rather than assuming a default

Record the actual client advertisement, server configuration, response size, flags, and transport. Different software versions and operators choose different payload values; neither 1232 nor 4096 is a universal default guaranteed by the standard. The fixture’s 1232 value is an explicit test configuration. Its loopback results do not measure fragmentation, a remote path’s MTU, or public resolver performance.

When diagnosing large-answer failures, compare a small query, a larger EDNS query, and a full TCP query through the same intended endpoint. Keep error categories separate: TC requests completion through another exchange, BADVERS reports extension-version disagreement, and a timeout does not identify which network component lost a packet. Preserve enough of the reply to avoid collapsing those distinct failures into “DNS is down.”

Use the deployment’s supported sizing and retry policy, then validate from the networks that matter to clients. Retain TCP access even when most observed answers fit in UDP. EDNS improves extensibility and can reduce some truncation retries, but complete delivery still depends on transport behavior, server limits, and client handling of the answer actually received.

Sources

dnspython Message API, to_wire serializes DNS messages; the unchanged fixture measures its returned bytes, checked 2026-10-18.
RFC 6891 extension mechanisms for DNS, OPT, payload advertisement, extended errors, and EDNS version handling, checked 2026-09-12.
RFC 7766 DNS over TCP, TCP remains a required DNS transport, checked 2026-09-12.
BIND 9.18.49 reference, edns-udp-size and max-udp-size configuration, checked 2026-09-12.

HTTP/2 removes HTTP response-order blocking

Tue, 15 Sep 2026 00:00:00 +0000

Here, removing head-of-line blocking means removing HTTP response-order constraints on a shared connection. HTTP/2 does not remove TCP head-of-line blocking. A later response can finish while an earlier handler waits, but shared transport capacity, connection-level flow control, and application bottlenecks can still couple the streams.

Separate the two queues

HTTP/1.1 already supports persistent connections, so it is not one connection per request. Requests can reuse a socket sequentially. Pipelining permits multiple outstanding requests, and a server may process safe methods concurrently, but responses on that connection must remain in request order. A slow first response therefore delays delivery of later responses. Clients often use multiple connections instead.

HTTP/2 assigns stream identifiers to frames and interleaves messages on the connection. The server can send a ready response without first finishing an unrelated response. This is independent progress at the HTTP layer, not a promise that handlers receive separate CPU cores, databases, bandwidth, or unlimited scheduling opportunities.

TCP still presents one ordered byte stream. A missing byte range can prevent later received bytes, including other streams’ HTTP/2 frames, from reaching the application until recovery fills the gap. Previously delivered bytes are not withdrawn. HTTP/3 uses QUIC: each stream’s bytes remain ordered without a cross-stream transport delivery order. Shared congestion and flow-control limits, application dependencies, and QPACK header dependencies still matter; this is separate protocol context, not an HTTP/3 experiment.

Concurrency still has budgets

The peer’s SETTINGS_MAX_CONCURRENT_STREAMS limits streams initiated toward that peer. Nonempty DATA payloads consume both stream-level and connection-level credit, including padding but excluding the frame header. Exhausted connection credit can block other streams’ DATA. An empty DATA frame carrying END_STREAM may still be sent at zero credit; control frames do not consume these windows. Receiver consumption, window updates, worker pools, locks, and upstream connections still matter.

Inspect the negotiated protocol on the hop being measured. Browser-to-proxy HTTP/2 does not establish proxy-to-origin HTTP/2. A successful TLS connection alone is also insufficient evidence. In Go, a custom TLS configuration can require ForceAttemptHTTP2 to attempt HTTP/2; the response protocol and negotiated h2 value establish the actual result.

When it breaks

A fast request completing first proves little if the client opened another TCP connection. Conversely, a single-connection HTTP/1.1 client can queue requests before the server ever sees them. Count accepted connections and inspect client acquisition events before attributing order to server scheduling or network loss.

Increasing stream concurrency cannot repair an overloaded database or a connection window starved by unread data. Track response-body completion, active streams, shared resource waits, and per-hop protocol. Do not label every long request TCP head-of-line blocking, and do not assume a protocol upgrade alone supplies a measured page-speed improvement.

HTTP/2 streams; ordered TCP bytes: conceptual mechanisms and expected checks, not new runtime evidence.

Check it quickly

Save this as queue_probe.go and run go run queue_probe.go with Go 1.23 or newer. It creates separate local HTTP/1.1 and HTTP/2 fixtures over verified TLS 1.3. Each transport permits one TCP connection. A channel holds the slow handler, while GetConn confirms the fast request has begun trying to acquire a connection.

package main

import (
 "context"
 "crypto/tls"
 "fmt"
 "io"
 "net"
 "net/http"
 "net/http/httptest"
 "net/http/httptrace"
 "runtime"
 "sync"
 "sync/atomic"
 "time"
)

type connectionKey struct{}

func must(err error) {
 if err != nil {
 panic(err)
 }
}

func check(ok bool, message string) {
 if !ok {
 panic(message)
 }
}

func exercise(h2 bool) {
 slowEntered, releaseSlow, stop := make(chan struct{}), make(chan struct{}), make(chan struct{})
 var connections atomic.Int64
 var slowPassed atomic.Bool
 var orderMu sync.Mutex
 var order []string
 server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 if r.URL.Path == "/slow" {
 close(slowEntered)
 select {
 case <-releaseSlow:
 slowPassed.Store(true)
 case <-r.Context().Done():
 return
 case <-stop:
 return
 }
 }
 orderMu.Lock()
 order = append(order, r.URL.Path)
 orderMu.Unlock()
 w.Header().Set("X-Connection-ID", fmt.Sprint(r.Context().Value(connectionKey{})))
 fmt.Fprint(w, r.URL.Path)
 }))
 server.Config.ConnContext = func(ctx context.Context, _ net.Conn) context.Context {
 return context.WithValue(ctx, connectionKey{}, connections.Add(1))
 }
 server.EnableHTTP2 = h2
 server.TLS = &tls.Config{MinVersion: tls.VersionTLS13, MaxVersion: tls.VersionTLS13}
 server.StartTLS()
 defer server.Close()
 defer close(stop)
 config := server.Client().Transport.(*http.Transport).TLSClientConfig.Clone()
 check(!config.InsecureSkipVerify, "certificate verification must stay enabled")
 transport := &http.Transport{TLSClientConfig: config, ForceAttemptHTTP2: h2, MaxConnsPerHost: 1}
 defer transport.CloseIdleConnections()
 client := &http.Client{Transport: transport, Timeout: 2 * time.Second}
 protocol := "HTTP/1.1"
 if h2 {
 protocol = "HTTP/2.0"
 }
 call := func(route string, trace *httptrace.ClientTrace) string {
 request, err := http.NewRequest(http.MethodGet, server.URL+route, nil)
 must(err)
 if trace != nil {
 request = request.WithContext(httptrace.WithClientTrace(request.Context(), trace))
 }
 response, err := client.Do(request)
 must(err)
 body, readError := io.ReadAll(response.Body)
 closeError := response.Body.Close()
 must(readError)
 must(closeError)
 check(response.StatusCode == 200 && response.Proto == protocol, "wrong HTTP response or protocol")
 check(response.Header.Get("X-Connection-ID") == "1", "requests must share the first TCP connection")
 check(response.TLS.Version == tls.VersionTLS13 && len(response.TLS.VerifiedChains) > 0, "verified TLS 1.3 required")
 if h2 {
 check(response.TLS.NegotiatedProtocol == "h2", "HTTP/2 must be negotiated")
 }
 return string(body)
 }
 slowDone, fastDone := make(chan string, 1), make(chan string, 1)
 go func() { slowDone <- call("/slow", nil) }()
 <-slowEntered
 fastAttempted := make(chan struct{}, 1)
 trace := &httptrace.ClientTrace{GetConn: func(string) {
 select {
 case fastAttempted <- struct{}{}:
 default:
 }
 }}
 go func() { fastDone <- call("/fast", trace) }()
 <-fastAttempted
 wantFirst := "/slow"
 if h2 {
 check(<-fastDone == "/fast" && !slowPassed.Load(), "fast stream must finish while slow is gated")
 close(releaseSlow)
 check(<-slowDone == "/slow", "wrong slow response")
 wantFirst = "/fast"
 } else {
 close(releaseSlow)
 check(<-slowDone == "/slow" && <-fastDone == "/fast", "wrong sequential responses")
 }
 orderMu.Lock()
 defer orderMu.Unlock()
 check(len(order) == 2 && order[0] == wantFirst && order[1] != wantFirst, "wrong handler response order")
 check(connections.Load() == 1, "fixture must use exactly one TCP connection")
 fmt.Println(protocol, order, "one TCP connection")
}

func main() {
 exercise(false)
 exercise(true)
 fmt.Println("gated ordering test; no packet-loss or speedup measurement")
 fmt.Println(runtime.Version())
}

The HTTP/1.1 run prints [/slow /fast]; the HTTP/2 run prints [/fast /slow]. Both assert exactly one accepted TCP connection, complete response bodies, the expected HTTP version, and validated TLS. HTTP/2 additionally asserts h2 negotiation and that the fast body finishes before the slow handler is released.

The control is specifically Go’s non-pipelining HTTP/1.1 transport under a one-connection cap. It does not demonstrate that all HTTP/1.1 servers must execute pipelined requests serially. The gate is released at different points: after the fast acquisition attempt for HTTP/1.1, but only after fast completion for HTTP/2. Consequently, this is a controlled ordering experiment, not a fair latency benchmark.

Earlier article text reported success on macOS with Go 1.27.0; the retained exact-source review establishes compilation only, not that historical runtime. This review did not execute the fixture. Requests have two-second client timeouts, but that is not an externally enforced process deadline; a panic in a request goroutine can bypass another goroutine’s deferred cleanup. No packet loss, exhausted window, browser waterfall, or throughput was measured. TCP and QUIC distinctions come from standards.

Sources

RFC 9112: HTTP/1.1, persistence and pipelined response ordering, checked 2026-09-09.
RFC 9113: HTTP/2, multiplexing, TCP limitations, stream concurrency, and two-level flow control, checked 2026-09-09.
RFC 9000: QUIC, ordered stream delivery and shared transport limits, checked 2026-09-09.
Go net/http, transport connection limits and HTTP/2 configuration, checked 2026-09-09.
Go net/http/httptrace, acquisition events and their scope, checked 2026-09-09.
Go net/http/httptest, local TLS fixtures and HTTP/2 setup, checked 2026-09-09.
RFC 9114: HTTP/3, QUIC mapping and QPACK dependency boundaries, checked 2026-10-18.
Go specification: handling panics, defer scope and termination of an unrecovered goroutine panic, checked 2026-10-18.

Protocols · XB Field Notes

Gossip converges under explicit assumptions

Separate dissemination from reconciliation

When it breaks

Check it quickly

Further reading

Sources

The TCP handshake establishes sequence spaces, not synchronized clocks

When it breaks

Check it quickly

Diagnose setup separately from useful service

Sources

Well-known endpoints provide a discovery convention

Follow the suffix’s specification

When it breaks

Check it quickly

Sources

DNS needs a working TCP path when a UDP answer is truncated

When it breaks

Check it quickly

Diagnose truncation and connection failure separately

Sources

IPv6 resolves neighbors with ICMPv6 before the TCP connection can work

When it breaks

Check it quickly

Keep discovery, addressing, and authentication separate

Sources

UDP preserves datagrams; recovery and timing belong to the application

When it breaks

Check it quickly

Choose the missing semantics deliberately

Sources

EDNS advertises UDP capacity without guaranteeing path delivery

When it breaks

Check it quickly

Measure the configured exchange rather than assuming a default

Sources

HTTP/2 removes HTTP response-order blocking

Separate the two queues

Concurrency still has budgets

When it breaks

Check it quickly

Further reading

Sources