Filesystem · XB Field Notes

Extended attributes belong to the inode and to a namespace

Tue, 14 Dec 2027 00:00:00 +0000

Extended attributes attach named byte values to filesystem objects outside the ordinary file-content stream. Applications can use the user namespace for small metadata, while other namespaces have kernel, security-module, or privileged semantics. Their usefulness depends on the filesystem, access rules, and transfer tools. A metadata tag is reliable only when the workflow preserves and interprets it according to an explicit contract.

Names can share an inode and its attributes. A copy creates a separate object whose metadata depends on the transfer tool, destination and permissions; content equality alone does not verify the required attributes.

When it breaks

An attribute value is not necessarily text. It can contain NUL bytes or arbitrary binary data, so treating it as a terminated string or assuming a particular encoding can corrupt the value. An empty attribute and a missing attribute are also different states. Returning an empty string for both can hide whether the metadata was deliberately set or lost during transfer.

Same-filesystem rename retains the object and its attributes, but copying content usually creates another inode. The copy tool decides which metadata it attempts to preserve. Equal file hashes therefore do not prove equal extended attributes. A pipeline can successfully deliver every content byte while omitting a classification, origin, or application version tag that downstream processing expects.

Namespace names do not create application authorization rules. Setting user.classification=private does not make a readable file confidential. User attributes follow applicable file-access permissions; security attributes have their own documented semantics and policy. Confusing an application label with enforced access control can expose content while the label appears reassuringly intact.

Check it quickly

Save this as xattrs.py and run python3 xattrs.py as root in a disposable Linux container whose temporary directory supports user extended attributes. The recorded run used Linux 6.12.76, Python 3.11.2, and an ext4 volume. Root is needed only for the second-identity permission probe. Use an isolated disposable machine or container, never add privileges to a shared host for this demonstration; no account is created:

from pathlib import Path
from tempfile import TemporaryDirectory
import errno,os,shutil,subprocess,sys

def fails(code,operation):
 try:operation()
 except OSError as error:assert error.errno==code,error
 else:raise AssertionError('expected attribute error')
with TemporaryDirectory(prefix='xbcatm-xattr-') as directory:
 root=Path(directory);root.chmod(0o755);path=root/'data';path.write_bytes(b'public fixture');path.chmod(0o644)
 os.setxattr(path,'user.review',b'binary\0value\xff',os.XATTR_CREATE)
 assert os.getxattr(path,'user.review')==b'binary\0value\xff'
 fails(errno.EEXIST,lambda:os.setxattr(path,'user.review',b'other',os.XATTR_CREATE))
 fails(errno.ENODATA,lambda:os.setxattr(path,'user.missing',b'value',os.XATTR_REPLACE))
 os.setxattr(path,'user.empty',b'');assert os.getxattr(path,'user.empty')==b''
 os.removexattr(path,'user.empty');fails(errno.ENODATA,lambda:os.getxattr(path,'user.empty'))
 print('attribute values are bytes; empty differs from missing; CREATE and REPLACE enforce their preconditions')
 alias=root/'alias';os.link(path,alias)
 os.setxattr(alias,'user.review',b'shared',os.XATTR_REPLACE)
 assert os.getxattr(path,'user.review')==b'shared'
 moved=root/'moved';path.rename(moved);assert os.getxattr(moved,'user.review')==b'shared'
 plain=root/'plain';preserved=root/'preserved';shutil.copyfile(moved,plain);shutil.copy2(moved,preserved)
 fails(errno.ENODATA,lambda:os.getxattr(plain,'user.review'))
 assert os.getxattr(preserved,'user.review')==b'shared'
 assert plain.read_bytes()==preserved.read_bytes()==moved.read_bytes()
 print('hard links and same-filesystem rename retain inode attributes; copyfile omits them, copy2 preserves this supported user attribute')
 os.setxattr(moved,'user.classification',b'private')
 probe=r'''
from pathlib import Path
import errno,os,sys
path=Path(sys.argv[1]);assert path.read_bytes()==b'public fixture'
assert os.getxattr(path,'user.classification')==b'private'
try:os.setxattr(path,'user.classification',b'changed')
except OSError as error:assert error.errno==errno.EACCES
else:raise AssertionError('another identity changed a read-only user attribute')
'''
 result=subprocess.run([sys.executable,'-c',probe,str(moved)],user=61001,group=61001,extra_groups=[],capture_output=True,text=True,timeout=5)
 assert result.returncode==0,result.stderr
 fails(errno.E2BIG,lambda:os.setxattr(moved,'user.too_large',b'x'*65537))
 print('a user.classification label does not enforce confidentiality; file permissions block the other identity from changing it; oversized value returns E2BIG')

The fixture stores a value containing both NUL and a non-UTF-8 byte and retrieves the exact bytes. Creating the same key again with XATTR_CREATE fails with EEXIST, while replacing a missing key with XATTR_REPLACE fails with ENODATA. These flags express preconditions instead of silently turning every write into an unconditional update.

An explicitly empty value reads back as empty bytes. After removal, reading that key returns ENODATA. This provides a direct distinction between absence and an intentionally empty attribute. A caller using metadata to drive behavior needs to decide how each state affects the operation rather than collapsing both into one convenient default.

A hard link shares the same inode, so updating the attribute through that name changes the value seen through the original name. Renaming the original within the filesystem preserves the attribute. The experiment then copies the file twice: copyfile preserves the content but omits the user attribute, while copy2 preserves this supported attribute in the tested environment.

All three content streams compare equal despite the metadata difference. Python documents metadata copying as dependent on platform support and permissions, so this successful copy2 result is not a universal guarantee for every attribute namespace or destination. Verification should inspect the attributes actually needed after the specific transfer, not merely the copy command’s exit status.

The file remains mode 0644 when tagged private. Another ordinary identity reads both its content and the tag, proving that the label did not enforce confidentiality. That identity cannot change the attribute because it lacks the relevant write permission. Finally, a value larger than 65536 bytes is rejected with E2BIG, demonstrating a real size boundary.

Preserve meaning across the workflow

Choose application-specific key names to avoid collisions within the shared user namespace, and define the value encoding and version. Keep values small enough for the target filesystems; their practical limits can be tighter than the Linux interface maximum and can depend on other metadata already attached to the inode.

Test backup, archive, restore, and cross-filesystem copy paths using both content and attribute assertions. Some destinations do not support the required namespaces, and some tools preserve them only with explicit options. Decide whether loss should fail the workflow or be represented through a documented alternative, based on the actual metadata contract.

Updating file bytes and updating a separate attribute are not automatically one application transaction. A checksum tag can become stale after a later write, and a writer able to change both data and metadata is not an independent integrity authority. Validate the relationship when consuming it, and use a stronger storage or signing contract when that relationship must survive concurrent changes or untrusted writers.

Sources

xattr(7), attribute namespaces, permission rules, and storage limits, checked 2026-09-12.
setxattr(2), CREATE/REPLACE behavior and attribute error conditions, checked 2026-09-12.
getxattr(2), binary values and missing-attribute errors, checked 2026-09-12.
Python 3.11 shutil documentation, copyfile and best-effort metadata preservation by copy2, checked 2026-09-12.

Btrfs snapshots are copy-on-write, not copies

Sun, 14 Nov 2027 00:00:00 +0000

A Btrfs snapshot is a new subvolume with the source’s initial contents, sharing existing file extents rather than duplicating every data block. Creation copies root metadata and may flush dirty source data, so it is not guaranteed to finish in milliseconds. More than two subvolumes or reflinks can reference the same extent; there is no fixed two-reference limit.

Snapshots make local recovery points economical, but their cost includes metadata and blocks retained by later divergence. They share the filesystem’s storage pool with the source, which may span several devices. Keeping many snapshots can retain deleted or overwritten data. A snapshot within that same filesystem is not an independent backup, even when it is read-only.

Shared extents, not copied blocks

Snapshots are subvolumes with their own roots. The example btrfs subvolume snapshot -r "$src" "$snap" requests a read-only snapshot; ordinary snapshots are writable. Read-only status prevents routine changes through that subvolume, not administrator deletion or underlying corruption. Snapshotting is not recursive: nested subvolumes appear as empty stubs, so their contents require separate handling.

When a shared extent changes, copy-on-write preserves the older contents for remaining references. Allocation depends on extent layout, compression, metadata and later snapshots, not simply application bytes written. The earlier APFS clone figures are not Btrfs measurements and have no recovered trace here. They cannot establish zero-cost creation or zero allocation on a repeated Btrfs overwrite.

Where the space actually goes

btrfs filesystem du reports file-extent Total, Exclusive and Set shared values using FIEMAP. Each argument defines its own recursively scanned file set; Set shared counts shared extents referenced by that set, accounting for overlaps, rather than only sharing with other arguments. compsize reports compression-oriented Disk Usage, Uncompressed and Referenced columns, not the same split. Qgroup accounting additionally requires enabled, consistent quotas.

Deleting the source does not release extents still referenced by snapshots or reflinks. Those extents become exclusive only if no other references remain. Subvolume deletion removes the directory first and cleans extents asynchronously; space need not return immediately. df gives filesystem-level capacity, not an attribution of every retained byte to one snapshot.

A restore point is not a replica

A local snapshot can preserve a useful filesystem view without providing application-consistent recovery. A database may need coordinated backup procedures, its logs and recovery after restoration; unsent application buffers are outside the snapshot. Source and snapshot also share underlying failure risks. Local rollback is neither a fixed-time guarantee nor a substitute for testing that the application can restart correctly.

btrfs send generates a stream for btrfs receive on another filesystem. Incremental -p operation relies on a suitable unchanged parent present at both ends; all participating snapshots must be read-only. Separate storage helps only against failures it does not share. Retention, successful receipt and content-verified restoration still matter; more frequent snapshots do not guarantee a fixed reduction in transferred bytes.

When it breaks

When deleting a large directory frees little space, inspect snapshot and reflink references before deleting more data. btrfs filesystem du -s and qgroups provide different accounting views; one Exclusive value need not explain the whole discrepancy. Only remove recovery points after checking retention and send dependencies. Several snapshots can collectively retain blocks that none owns exclusively by itself.

Copy-on-write needs allocation headroom for data and metadata. Review both with btrfs filesystem usage; unallocated device space and free space inside allocated block groups are different quantities. Neither a universal five-percent reserve nor a fixed order of exhaustion follows from these counters. An allocation failure alone also does not establish that the filesystem became read-only.

Btrfs shared extents — mechanism diagram.

Check it quickly

This limited inspection remains unexecuted. It requires authorized disposable source /mnt/btrfs-lab/data, an existing snapshots parent on the same Btrfs filesystem, tools and permissions. Keep all paths symlink-free and exclusively controlled throughout. An existing destination directory changes where Btrfs creates the snapshot; the guard rejects existing paths, including dangling symlinks. It is not a race-proof lock; stop if exclusive control is unavailable.

(
 set -eu
 src='/mnt/btrfs-lab/data'
 snap='/mnt/btrfs-lab/snapshots/data-2026-09-16'
 if [ -e "$snap" ] || [ -L "$snap" ]; then
 printf '%s\n' 'Refusing an existing snapshot destination.' >&2
 exit 1
 fi
 btrfs subvolume snapshot -r "$src" "$snap"
 btrfs filesystem du -s "$src" "$snap"
 compsize "$snap"
 btrfs filesystem usage "$src"
)

Snapshot inspection now uses the bound destination; reruns refuse it rather than removing it. File-extent counters do not measure total snapshot metadata. Exclusive equalling Total means currently unshared extents, not absence of snapshot history. This is not a divergence or restore test: separately record known fixture bytes, verify both views initially, change only the source, then verify old snapshot bytes and new source bytes. Those storage checks and application recovery remain held.

Sources

btrfs subvolume documentation, subvolume and snapshot semantics including read-only snapshots, checked 2026-09-16.
btrfs filesystem documentation, defines the du Exclusive and Set shared columns used above, checked 2026-09-16.
btrfs qgroup documentation, exclusive and referenced byte accounting per qgroup, checked 2026-09-16.
btrfs send documentation, send-stream generation and read-only incremental-base requirements, checked 2026-09-16.
compsize upstream manual, compression and referenced-size accounting differs from filesystem du, checked 2026-10-18.
PostgreSQL filesystem backup, consistent snapshot scope, WAL and multi-volume recovery requirements, checked 2026-10-18.

Inotify reports changes, but it is not a complete history

Wed, 13 Oct 2027 00:00:00 +0000

Inotify delivers Linux filesystem notifications through an instance with registered watches and a readable event queue. Each record identifies a watch, event mask, and possibly a name and move cookie. This can avoid repeatedly checking unchanged paths, but the stream is not an unlimited audit log. Correct consumers interpret notifications as evidence that relevant state may need to be inspected again.

Notifications provide bounded evidence of change. Watch coverage, object replacement and queue loss all require checking current filesystem state; a rescan cannot recover missing event history.

When it breaks

Watching a directory does not recursively watch its descendants. It reports relevant activity on immediate entries, so a new subdirectory needs its own watch if deeper changes matter. A tree watcher must handle initial enumeration and directories arriving later. Assuming one root watch covers the entire tree silently misses changes below that root.

A file watch follows the watched object, not a pathname forever. An editor can finish writing a temporary file and rename it over the target. The parent directory reports the replacement, while the old file watch does not transfer to the new inode. Waiting for a later close-write event on the destination name can fail because the replacement was already closed before publication.

Successive identical unread events can coalesce, and a full queue drops events while reporting IN_Q_OVERFLOW. Event counts therefore do not reliably count writes. After overflow, a consumer cannot reconstruct every missing operation from the remaining records. It must reconcile the current watched state instead of continuing with an assumption that its event-derived model is complete.

Check it quickly

Save the script as inotify-events.py and run python3 inotify-events.py in a disposable Linux environment. The earlier record used Linux 6.12.76 and ext4; the 2026-10-15 review recorded both translations passing on Linux 6.18.44 with Python 3.12.14. That is historical execution, not a rerun here. It creates ASCII-named temporary entries and intentionally fills its own queue. The configured queue limit must be at most 32768; the script reads that limit without changing any sysctl:

from pathlib import Path
from tempfile import TemporaryDirectory
import ctypes,os,struct

MODIFY=0x2;ATTRIB=0x4;CLOSE_WRITE=0x8;MOVED_FROM=0x40;MOVED_TO=0x80
CREATE=0x100;DELETE=0x200;DELETE_SELF=0x400;MOVE_SELF=0x800;OVERFLOW=0x4000;IGNORED=0x8000
libc=ctypes.CDLL(None,use_errno=True)
libc.inotify_init1.argtypes=[ctypes.c_int];libc.inotify_init1.restype=ctypes.c_int
libc.inotify_add_watch.argtypes=[ctypes.c_int,ctypes.c_char_p,ctypes.c_uint32];libc.inotify_add_watch.restype=ctypes.c_int
header=struct.Struct('iIII')
with TemporaryDirectory(prefix='xbcatm-inotify-') as directory:
 root=Path(directory);fd=libc.inotify_init1(os.O_NONBLOCK|os.O_CLOEXEC);assert fd>=0,os.strerror(ctypes.get_errno())
 def watch(path,mask):
 wd=libc.inotify_add_watch(fd,os.fsencode(path),mask);assert wd>=0,os.strerror(ctypes.get_errno());return wd
 def drain():
 events=[]
 while True:
 try:data=os.read(fd,65536)
 except BlockingIOError:return events
 offset=0
 while offset<len(data):
 wd,mask,cookie,length=header.unpack_from(data,offset);offset+=header.size
 name=data[offset:offset+length].split(b'\0',1)[0].decode();offset+=length
 events.append((wd,mask,cookie,name))
 try:
 parent=watch(root,MODIFY|CLOSE_WRITE|CREATE|DELETE|MOVED_FROM|MOVED_TO)
 sub=root/'sub';sub.mkdir();drain();(sub/'unwatched').touch();assert drain()==[]
 nested=watch(sub,CREATE);(sub/'watched').touch();assert any(wd==nested and mask&CREATE and name=='watched' for wd,mask,_,name in drain())
 print('a directory watch is not recursive; adding a child-directory watch observes its new entries')
 counter=root/'counter';stream=os.open(counter,os.O_CREAT|os.O_WRONLY,0o600);drain()
 try:
 for _ in range(3):os.write(stream,b'x')
 finally:os.close(stream)
 events=drain();assert sum(wd==parent and bool(mask&MODIFY) and name=='counter' for wd,mask,_,name in events)==1
 moved=root/'moved';counter.rename(moved);events=drain()
 before=next(e for e in events if e[1]&MOVED_FROM and e[3]=='counter');after=next(e for e in events if e[1]&MOVED_TO and e[3]=='moved')
 assert before[2]==after[2]!=0
 print('three unread identical modifications coalesce; rename FROM/TO events share a nonzero cookie')
 target=root/'target';target.write_bytes(b'old');file_watch=watch(target,MODIFY|ATTRIB|DELETE_SELF|MOVE_SELF);drain()
 replacement=root/'replacement';replacement.write_bytes(b'new');drain();replacement.replace(target);events=drain()
 assert any(wd==file_watch and mask&IGNORED for wd,mask,_,_ in events)
 assert any(wd==parent and mask&MOVED_TO and name=='target' for wd,mask,_,name in events)
 assert not any(wd==parent and mask&CLOSE_WRITE and name=='target' for wd,mask,_,name in events)
 target.write_bytes(b'newer');events=drain()
 assert any(wd==parent and mask&MODIFY and name=='target' for wd,mask,_,name in events)
 assert all(wd!=file_watch for wd,_,_,_ in events)
 print('replacement invalidates the old file watch; the parent-directory watch continues to observe the pathname')
 burst=root/'burst';burst.mkdir();drain();watch(burst,CREATE)
 limit=int(Path('/proc/sys/fs/inotify/max_queued_events').read_text());assert 0<limit<=32768,'use a disposable environment with a bounded queue limit'
 for number in range(limit+8):(burst/str(number)).touch()
 events=drain();assert any(wd==-1 and mask&OVERFLOW for wd,mask,_,_ in events)
 current=len(list(burst.iterdir()));assert current==limit+8
 print('actual queue overflow:',len(events),'events retained for',current,'creates; rescan sees the current entries')
 finally:os.close(fd)

The first case watches the parent directory, creates a subdirectory, and drains the parent event. Creating a file inside that subdirectory produces no event for the parent watch. Adding an explicit subdirectory watch makes the next creation visible. This distinguishes direct-entry coverage from recursive monitoring using the same inotify instance.

Three writes to one file occur before the queue is read. The fixture observes one matching modify event, demonstrating coalescing rather than one notification per write. Renaming the file then produces FROM and TO events with the same nonzero cookie. The cookie associates these two records; it is not a durable identifier for the file’s entire future lifetime.

Next, the script watches a file and replaces its pathname with a previously written and closed file. The old watch receives an ignored event when its object is deleted, while the parent watch reports MOVED_TO for the target name. No destination close-write event accompanies that publication. A subsequent write to the replacement is visible through the parent watch, not the removed file watch.

For overflow, the fixture creates 16392 distinct entries with a configured limit of 16384 queued events. It receives 16385 records, including the overflow notification with watch descriptor minus one. A directory rescan finds all 16392 current entries. Those results show both missing event history and a concrete way to recover present-state knowledge.

All watches share one owned instance, which closes before temporary-tree cleanup. The experiment does not induce a system-wide watch shortage or modify other applications’ queues. Its deliberately bounded burst validates overflow behavior without requiring an application to remain permanently overloaded or relying on an assumed loss signal that was never observed.

Reconcile names and object lifetimes

Pair move cookies when both sides are visible, but allow for moves into or out of the observed set and records separated by other events. Recheck the relevant paths before acting. A name in an event describes the reported change; it does not guarantee that the same object will still occupy that name when the consumer later opens it.

For trees, establish watches and reconcile directory contents while accounting for activity during enumeration. On overflow, rebuild the affected state from the filesystem and restore the necessary watch coverage. An overflow record does not identify one missing filename, so handling it as an ordinary per-watch event is insufficient.

Use notifications to trigger work that tolerates repeated or combined changes, such as reloading a complete configuration after validating its current contents. If every transition must be preserved, use an application journal or another interface with the required history contract. Inotify is useful precisely when its coverage and loss rules match the consumer’s actual needs.

Sources

inotify(7), event formats, coalescing, overflow, renames, and nonrecursive watches, checked 2026-10-15.
inotify_add_watch(2), watch registration and object identity, checked 2026-10-15.
Linux 6.12 inotify UAPI, event structure and flag values used by the executable fixture, checked 2026-10-15.

A sparse file separates logical length from allocated blocks

Mon, 11 Oct 2027 00:00:00 +0000

A sparse regular file can have a large logical length while allocating storage for only part of its address range. Reads from holes within that length return zero bytes. File length, byte content, and filesystem allocation are therefore different measurements. Understanding their relationship matters when budgeting virtual disks, copying backups, or deciding whether a zero-filled region actually reserves space for future writes.

Measure length, bytes and allocation separately, then verify the actual copy or reservation contract.

When it breaks

Seeking beyond the end of a file changes the descriptor’s offset, not the file’s length. A later write or an explicit extension can create the intervening zero-reading range. Calling seek alone and reporting that a large sparse file now exists is incorrect. Likewise, a large logical size does not mean the filesystem has reserved enough blocks to fill that file later.

A copy that reads every byte and writes it normally can turn holes into allocated zero-filled ranges. The resulting checksum can match perfectly while its allocation is much larger. Transfer and restore workflows that budget only the source’s current allocated space can therefore run out of capacity despite preserving the logical content correctly.

Hole punching is an intentional data operation. Punching an allocated range makes the corresponding bytes read as zeros while preserving length when requested with KEEP_SIZE. It is not a harmless command to tidy away arbitrary file regions. Preallocation solves a different problem: it reserves filesystem allocation, even though an unwritten allocated range can also read as zeros.

Check it quickly

Save the script as sparse.py and run python3 sparse.py using Python 3.11 or later on a disposable Linux system with an ext4 temporary directory. The recorded run used Linux 6.12.76 and Python 3.11.2. It creates three temporary files and uses about 32 MiB of fully allocated data plus the small sparse allocation:

from pathlib import Path
from tempfile import TemporaryDirectory
import ctypes,hashlib,os

size=16*1024*1024;first=4*1024*1024;second=12*1024*1024
libc=ctypes.CDLL(None,use_errno=True);libc.fallocate.argtypes=[ctypes.c_int,ctypes.c_int,ctypes.c_longlong,ctypes.c_longlong];libc.fallocate.restype=ctypes.c_int
def digest(path):
 with path.open('rb') as handle:return hashlib.file_digest(handle,'sha256').hexdigest()
with TemporaryDirectory(prefix='xbcatm-sparse-') as directory:
 root=Path(directory);path=root/'sparse';fd=os.open(path,os.O_RDWR|os.O_CREAT|os.O_EXCL,0o600)
 try:
 os.lseek(fd,size,os.SEEK_SET);assert os.fstat(fd).st_size==0
 os.ftruncate(fd,size);assert os.fstat(fd).st_size==size and os.fstat(fd).st_blocks==0
 assert os.pread(fd,16,0)==b'\0'*16
 os.pwrite(fd,b'A',first);os.pwrite(fd,b'B',second);os.fsync(fd)
 allocated=os.fstat(fd).st_blocks*512;assert allocated<size
 data_start=os.lseek(fd,0,os.SEEK_DATA);hole_start=os.lseek(fd,data_start,os.SEEK_HOLE)
 assert data_start==first and first<hole_start<=second
 print('seek alone leaves size zero; truncate creates zero-reading holes; sparse size',size,'allocated',allocated)
 print('first SEEK_DATA',data_start,'following SEEK_HOLE',hole_start)
 dense=root/'dense'
 with path.open('rb') as source,dense.open('wb') as target:
 while chunk:=source.read(65536):target.write(chunk)
 target.flush();os.fsync(target.fileno())
 assert digest(path)==digest(dense)
 assert dense.stat().st_blocks*512>=size
 print('explicitly writing all copied bytes preserves content but allocates the zero-filled ranges')
 result=libc.fallocate(fd,0x01|0x02,first,4096) # KEEP_SIZE | PUNCH_HOLE
 assert result==0,os.strerror(ctypes.get_errno())
 os.fsync(fd)
 assert os.fstat(fd).st_size==size and os.pread(fd,1,first)==b'\0'
 assert os.pread(fd,1,second)==b'B' and os.fstat(fd).st_blocks*512<allocated
 print('punching the aligned data range releases its blocks and makes it read zero without changing file length')
 reserved=root/'reserved'
 with reserved.open('w+b') as handle:
 os.posix_fallocate(handle.fileno(),0,size);os.fsync(handle.fileno())
 info=os.fstat(handle.fileno());assert info.st_size==size and info.st_blocks*512>=size
 assert handle.read(16)==b'\0'*16
 print('preallocation reserves blocks even while the newly allocated range reads as zeros')
 finally:os.close(fd)

The fixture first seeks to sixteen MiB in an empty file and verifies that its size remains zero. Truncating to that length then creates a file with sixteen MiB of logical content and no allocated data blocks in this ext4 case. Reading its beginning returns zeros, establishing the difference between moving an offset and extending the file.

Writing one byte at four MiB and another at twelve MiB allocates two small regions. After synchronization, the recorded file length is 16777216 bytes and its reported allocation is 8192 bytes. Linux’s st_blocks count is converted using 512-byte units; treating it as a count of the filesystem’s larger block size would miscalculate the result.

The first SEEK_DATA result is 4194304, and the following SEEK_HOLE result is 4198400 in the recorded run. These describe the ext4 region around the first written byte, not only that byte’s nonzero position. A data extent may contain zeros too, so extent reporting is not a search for nonzero content.

The script explicitly copies all logical bytes through ordinary writes into a second file. Its digest matches the sparse source, but its allocated space covers the full logical length. This controlled dense copy demonstrates the expansion risk without claiming that every modern copy tool or filesystem necessarily materializes holes in the same way.

Next, an aligned punch removes the first allocated data region. The original A byte now reads as zero, the distant B byte remains, length stays unchanged, and allocated blocks decrease. Finally, preallocating the third file reserves space across sixteen MiB while its initial bytes read as zeros. Zero content alone cannot distinguish a hole from reserved allocation.

Budget and preserve the intended representation

Check both logical size and allocated blocks after the actual archive, transfer, and restore path. A sparse-aware option may preserve holes, while compression, reflinks, and filesystem-specific copying can produce other storage relationships. Per-file allocation also should not automatically be interpreted as exclusive physical-device usage when the filesystem shares or compresses extents.

Use the target filesystem’s documented preallocation contract when future allocation must be reserved. A merely truncated virtual image can grow its storage consumption later and fail when the shared pool is full. Capacity planning needs the expected populated size and competing allocations, not just an attractive small allocation measured at creation time.

Treat SEEK_DATA and SEEK_HOLE as filesystem-provided extent information. Implementations are allowed to report holes conservatively, including representing a broader range as data. Verify support and behavior before depending on exact offsets. When punching or copying ranges, preserve the application’s intended bytes first; space savings are useful only when the resulting data and allocation match the operation’s actual purpose.

Sources

lseek(2), seek position, SEEK_DATA/SEEK_HOLE, and filesystem reporting limits, checked 2026-09-12.
truncate(2), extending logical length with zero-reading regions, checked 2026-09-12.
fallocate(2), preallocation and hole punching without changing logical size, checked 2026-09-12.
Linux 6.12 fallocate UAPI, KEEP_SIZE and PUNCH_HOLE flags used by the fixture, checked 2026-09-12.

Hard links share an inode until a name is replaced

Sun, 10 Oct 2027 00:00:00 +0000

A hard link adds another directory entry referring to an existing file. Neither name is intrinsically the original: both identify the same inode within its filesystem. File content, ownership, permissions, and link count belong to that shared object. A filename is a way to reach it, so changes to directory entries must be distinguished from changes to the file itself.

A name and an open reference have different lifetimes. Track the inode and the actual update method.

When it breaks

Treating a hard link as an independent backup creates a false recovery boundary. Writing or truncating the shared inode through either name changes what the other name reads. Changing its mode likewise changes the same object’s permissions. A second pathname can survive removal of the first, but it does not preserve an independent historical version against ordinary in-place modifications.

Many editors and deployment tools update differently: they create a replacement file and rename it over one pathname. That directory entry now reaches a new inode, while another hard link still reaches the old one. Whether both names show the update depends on the update method. The mere fact that they were linked yesterday does not establish their identity today.

Hard links also stay within the applicable filesystem mount boundary. link cannot join arbitrary files from different mounts, even when directories appear next to each other in one pathname tree. Ordinary directory hard links are prohibited on Linux. These restrictions prevent using hard links as a general replacement for symbolic links across a directory hierarchy.

Check it quickly

Save the script as hardlinks.py and run python3 hardlinks.py on Linux with permission to create temporary directories. It requires writable /tmp and /run on different filesystems; set CROSS_ROOT to another writable directory on a second filesystem when needed. No root privilege or mount creation is required. The recorded environment used separate tmpfs mounts, Linux 6.12.76, and Python 3.11.2. All created entries are confined to temporary directories:

from pathlib import Path
from tempfile import TemporaryDirectory
import errno,os

with TemporaryDirectory(prefix='xbcatm-hardlinks-',dir='/tmp') as directory, TemporaryDirectory(prefix='xbcatm-crosslink-',dir=os.environ.get('CROSS_ROOT','/run')) as other:
 root=Path(directory);source=root/'source';alias=root/'alias'
 source.write_bytes(b'original');os.link(source,alias)
 before=source.stat();linked=alias.stat()
 assert (before.st_dev,before.st_ino)==(linked.st_dev,linked.st_ino)
 assert before.st_nlink==linked.st_nlink==2 and before.st_size==linked.st_size==8
 alias.write_bytes(b'changed')
 assert source.read_bytes()==b'changed'
 print('two names share device/inode, size, and link count; in-place write changes both views')
 replacement=root/'replacement';replacement.write_bytes(b'new file');replacement.replace(source)
 assert source.stat().st_ino!=alias.stat().st_ino
 assert source.read_bytes()==b'new file' and alias.read_bytes()==b'changed'
 assert source.stat().st_nlink==alias.stat().st_nlink==1
 print('atomic replacement changes one name to a new inode; the other name retains the old file')
 with alias.open('rb') as opened:
 alias.unlink()
 assert os.fstat(opened.fileno()).st_nlink==0 and opened.read()==b'changed'
 print('last unlink removes the final name but an existing open descriptor still reads the old data')
 assert root.stat().st_dev!=Path(other).stat().st_dev,'exercise requires two filesystem mounts'
 try:os.link(source,Path(other)/'cross-filesystem')
 except OSError as error:assert error.errno==errno.EXDEV,error
 else:raise AssertionError('cross-filesystem hard link unexpectedly succeeded')
 child=root/'directory';child.mkdir()
 try:os.link(child,root/'directory-link')
 except OSError as error:assert error.errno==errno.EPERM,error
 else:raise AssertionError('directory hard link unexpectedly succeeded')
 print('cross-filesystem link fails with EXDEV; directory hard link fails with EPERM on Linux')

The fixture creates source, writes eight bytes, and adds alias with os.link. Both names report identical device/inode pairs, size eight, and link count two. Comparing the device as well as the inode matters because inode numbers are not globally unique across filesystems. Matching file contents alone would also be insufficient evidence of shared identity.

Writing changed through alias updates the existing inode, so reading source returns the same changed bytes. The operation uses an ordinary truncating write, not replacement. This establishes why the second name cannot serve as a separate copy protecting the prior contents from a writer with access to the shared file.

Next, a separate file containing new file replaces source by rename. The two names now have different inode numbers and each has link count one. source reads the replacement, while alias retains changed. Replacement affects the selected directory entry; it neither searches for other names nor rewrites every hard link to the old inode.

The fixture opens alias and removes its final directory entry. File status through the open descriptor reports link count zero, yet that descriptor still reads changed. The open reference preserves access until it is closed. A zero link count means no hard-link names remain, not that every process has immediately lost its existing reference.

Finally, linking into the second filesystem fails with EXDEV, and attempting to hard-link a directory fails with EPERM. The script checks the specific errors. If its mount precondition is absent, it fails explicitly rather than reporting that a cross-filesystem operation was tested when both directories actually used the same filesystem.

Inspect identity and update behavior

Before relying on shared storage, inspect the device/inode pair and link count of the actual paths. Link count tells how many hard-link directory entries refer to the inode; it does not list where those names are. Traversing directories to find matching inodes is a separate operation, affected by permissions and changes happening during the search.

For snapshot-like directory trees built with hard links, ensure updates replace files instead of modifying shared inodes, and control all writers that can reach them. Filesystem snapshots and independent copies have different contracts. A space-saving tree of links should be described by its actual update and retention behavior rather than simply being called a backup.

Unlinking and renaming concern namespace visibility. This experiment verifies identity and open-reference behavior, not persistence across power loss. If durable publication matters, separately verify the filesystem’s synchronization requirements and the application’s recovery procedure. Keeping another name or observing an atomic rename does not by itself prove the intended data will survive a crash.

Sources

link(2), shared inode identity, cross-filesystem restrictions, and directory-link errors, checked 2026-09-12.
unlink(2), link-count reduction and lifetime of an unlinked open file, checked 2026-09-12.
rename(2), replacement of one pathname without changing other hard links, checked 2026-09-12.

A symbolic link stores a path, not a permanent target identity

Tue, 03 Aug 2027 00:00:00 +0000

A symbolic link is a filesystem object containing a target pathname. Following the link resolves that pathname at the time of the operation; it does not bind the link permanently to the target’s inode. This distinction explains why links can cross filesystem boundaries, become dangling, or start reaching a different object even though their stored target text has not changed.

A symlink stores a route. State the resolution base, observed object and traversal policy.

When it breaks

A relative target is interpreted from the directory containing the symbolic link, not from the calling process’s working directory. Moving the link can therefore change its meaning. Moving or renaming the target can instead leave the link dangling. A deployment that rearranges directories without accounting for these bases can silently change which file a familiar link reaches.

Existence checks answer different questions. An operation that follows the link can report a missing target while an operation inspecting the directory entry still finds a symbolic link. Likewise, a permissive pathname-resolution function may produce an absolute string even when its final target is missing. A normalized string is not proof that a usable file currently exists.

Checking a path and later opening it creates a time-of-check/time-of-use gap when another actor can change the relevant directory entries. Quoting the pathname protects shell argument parsing, but does not freeze filesystem resolution. O_NOFOLLOW on ordinary open rejects a symbolic link in the final component; it does not automatically prohibit links in earlier directory components.

Check it quickly

Save this script as symlinks.py and run python3 symlinks.py in a disposable Linux environment. The recorded run used Linux 6.12.76 and Python 3.11.2. It creates harmless temporary files and changes its own links deliberately, so the observations are deterministic and require no concurrent attacker or access to unrelated paths:

from pathlib import Path
from tempfile import TemporaryDirectory
import errno,os

def read_no_follow(path):
 fd=os.open(path,os.O_RDONLY|os.O_NOFOLLOW)
 try:return os.read(fd,100)
 finally:os.close(fd)
with TemporaryDirectory(prefix='xbcatm-symlinks-') as directory:
 root=Path(directory);first=root/'first';second=root/'second';first.mkdir();second.mkdir()
 (first/'data').write_text('first');(second/'data').write_text('second')
 link=first/'link';link.symlink_to('data')
 assert os.readlink(link)=='data' and link.read_text()=='first'
 assert link.lstat().st_ino!=link.stat().st_ino
 moved=second/'link';link.rename(moved)
 assert os.readlink(moved)=='data' and moved.read_text()=='second'
 print('moving a relative symlink changes its target because resolution starts at its containing directory')
 (second/'data').rename(second/'renamed')
 assert moved.is_symlink() and os.path.lexists(moved) and not moved.exists()
 assert moved.resolve(strict=False)==second/'data'
 try:moved.resolve(strict=True)
 except FileNotFoundError:pass
 else:raise AssertionError('strict resolution unexpectedly found missing target')
 moved.unlink();assert (second/'renamed').read_text()=='second'
 print('dangling link exists as a link; permissive resolution can name a missing target; unlink removes only the link')
 safe=root/'safe';outside=root/'outside';safe.write_bytes(b'approved');outside.write_bytes(b'replaced')
 current=root/'current';current.symlink_to('safe');checked=current.resolve(strict=True)
 assert checked==safe
 opened=os.open(current,os.O_RDONLY)
 try:
 replacement=root/'next';replacement.symlink_to('outside');replacement.replace(current)
 assert current.read_bytes()==b'replaced' and os.read(opened,100)==b'approved'
 finally:os.close(opened)
 print('a checked path string can resolve differently later; an already opened descriptor keeps the prior object')
 try:read_no_follow(current)
 except OSError as error:assert error.errno==errno.ELOOP,error
 else:raise AssertionError('O_NOFOLLOW accepted a final-component symlink')
 alias=root/'parent-alias';alias.symlink_to(first.name,target_is_directory=True)
 assert read_no_follow(alias/'data')==b'first'
 print('O_NOFOLLOW rejects the final symlink but still allows symlinks in earlier path components')

The first directory contains a file named data and a link whose stored text is simply data. Reading through the link returns first. lstat inspects the link itself, while stat follows it and reports the target; the fixture confirms these are different inode identities.

After moving the link into the second directory, readlink still returns the same text, but reading through it returns second. The target is now the second directory’s data file. This result demonstrates the resolution base directly, without relying on a changed process working directory to explain the different bytes.

Renaming that target leaves a dangling link. is_symlink and lexists remain true, whereas exists, which follows the link, becomes false. Python’s permissive resolve(strict=False) returns the expected missing target path; strict resolution raises FileNotFoundError. Removing the dangling link leaves the renamed target intact, because unlinking the link does not unlink the target it once referenced.

Next, the script resolves a link to a harmless approved file and opens it. It then replaces the link with one pointing at another fixture file. A new pathname read returns replaced, while the already opened descriptor still returns approved. The earlier check did not constrain the later lookup; the existing descriptor retained the earlier object.

Opening the final link with O_NOFOLLOW fails with ELOOP. However, using that same flag to open parent-alias/data succeeds when parent-alias is a directory symlink. Both cases matter: testing only the final-component rejection could otherwise encourage an incorrect claim that this flag protects the entire path from symbolic-link traversal.

Choose the object and the traversal policy

Use readlink when the question concerns stored link text, lstat when it concerns the link object, and following operations when it concerns the current target. Record which identity a diagnostic reports. A log showing an absolute resolved pathname is helpful context, but is still an observation at a particular time.

For operations within a trusted directory boundary, design traversal around directory descriptors and restrictions appropriate to the threat model. Linux openat2 offers resolution policies such as rejecting symbolic links or constraining traversal beneath a directory. Their exact behavior and kernel support must be checked; merely joining a trusted prefix with untrusted text does not establish containment.

An opened descriptor stabilizes the referenced object against later pathname replacement, but it does not make the object’s contents immutable or establish that opening it was authorized. Validate the selected object and perform the intended operation through the appropriate reference. Keep path-selection guarantees, access permissions, and content consistency as separate properties of the design.

Sources

symlink(7), stored target strings, relative resolution, and dangling links, checked 2026-09-12.
open(2), O_NOFOLLOW and descriptor references after pathname changes, checked 2026-09-12.
path_resolution(7), component-by-component traversal and directory permissions, checked 2026-09-12.
openat2(2), kernel-enforced resolution restrictions for controlled traversal, checked 2026-09-12.

NFS shares a remote directory over the network

Wed, 23 Jun 2027 00:00:00 +0000

An NFS export combines a server-side filesystem path with client and access policy. On Linux, exportfs maintains /var/lib/nfs/etab from /etc/exports, eligible files under /etc/exports.d, or command-line updates. rpc.mountd is a daemon, not a third configuration file. Its MOUNT service matters to older protocol workflows; NFSv4 does not require that separate service for mounting.

Configuration text can differ from effective exports, but that is one possible cause of refusal, not a diagnosis from appearance alone. Check the intended server, protocol, export policy, identity and filesystem permissions. Also separate server durability from client caching: seeing a write succeed in an application is not by itself proof that the server has committed it to stable storage.

The file and the table

An /etc/exports entry associates a directory with clients and options. /srv/data 10.0.0.0/24(rw,sync,no_subtree_check,root_squash) illustrates one network; further client groups can have their own options. Hostnames, network ranges and netgroups have documented matching rules. exportfs -ra reconciles configured exports, but other exportfs actions or service management can also apply changes; it is not the sole activation mechanism.

showmount -e asks the server’s MNT service for advertised exports. Completeness depends on the implementation, and NFSv4-only servers may not expose that service. It is neither a complete effective-policy dump nor a proof that a client can access a share. The old four-versus-three-entry and NOENT story has no recovered raw test here; a missing export does not promise one universal error code.

What the options actually decide

The default export is read-only unless rw enables writes; filesystem permissions and other controls still apply. sync constrains server replies according to stable-storage requirements, while async permits replies before changes are committed and risks acknowledged data after a crash. Neither option removes client caching or makes every application write a synchronous durable commit. nfs-utils changed its default from async to sync after release 1.0.0.

root_squash maps request uid/gid zero to anonymous IDs; no_root_squash disables that protection. all_squash maps all IDs, and anonuid/anongid select the anonymous values. These do not authenticate a client or make an arbitrary public share safe. subtree_check has security as well as rename-reliability consequences; its default changed to no_subtree_check in nfs-utils 1.1.0. Evaluate the actual exported hierarchy and trust boundary.

Mounting, and the id mapping trap

The illustrative mount -t nfs -o vers=4.2 server:/srv/data /mnt/data requires a server supporting that protocol and an applicable export path. NFSv4 paths are relative to its exported namespace; fsid=0 identifies the export root, not a requirement to expose the host’s entire root filesystem. Hard mounts retry timed-out requests indefinitely; soft mounts can return EIO and risk data corruption. Neither choice proves a particular process state.

With AUTH_SYS, numeric request IDs participate in permission checks; lacking a server-side account name does not automatically squash a nonzero ID. NFSv4 owner-name mapping and stronger authentication introduce additional identity boundaries. Compare id, ls -ln, squash policy and mapping configuration. An owner displayed as 65534 or nobody can reflect deliberate squashing or mapping failure, so it is a symptom rather than a unique diagnosis.

When it breaks

If exports are stale, reconcile the intended configuration including /etc/exports.d and command-line overrides, then inspect the result. exportfs -ra changes server policy and may affect existing access; review it before use. A configuration mismatch is distinct from name resolution, reachability, protocol support or permission failures. Do not apply a policy refresh simply because one client operation failed.

An unavailable server can leave operations waiting and cause serious application disruption. It does not follow that every process touching a hard mount is permanently unkillable. The old thirty-five-second showmount failure is uncorroborated here and does not establish the mounted kernel client’s timeout. intr is ignored after Linux 2.6.25; changing to soft requires weighing data-integrity risk, not merely shortening a wait.

NFS access crosses policy — mechanism diagram.

Check it quickly

The original commands remain held. They mix server checks, a localhost MNT query, a policy mutation and client observations without identifying separate hosts. No NFS service, RPC request, export refresh or mount was exercised:

exportfs -v
showmount -e localhost
exportfs -ra && exportfs -v
mount | grep 'type nfs'
id; ls -ln /mnt/data | head

Compare policy, not printed option order. This offline proposal supports one /srv/data IPv4-network export, using nfs-utils defaults since 1.1.0. It expands omitted defaults, rejects conflicting/unknown options and compares exact paths plus canonical networks. Hostnames, netgroups, wildcards, multiple security flavors and additional records are unsupported, never silently equivalent.

import ipaddress
import re

# A single-export, numeric IPv4-network fixture; not a general exports parser.
def export_policy(text):
 match = re.fullmatch(
 r"(/srv/data)[ \t\n]+([0-9./]+)\(([^()\s]+)\)\n", text)
 if match is None or "/" not in match[2]:
 raise ValueError("incomplete or unsupported export record")
 client = ipaddress.IPv4Network(match[2], strict=True)
 mask = match[2].split("/")[1]
 if mask not in (str(client.prefixlen), str(client.netmask)):
 raise ValueError("noncanonical prefix or unsupported hostmask")
 groups = [
 ("ro", "rw"), ("sync", "async"), ("wdelay", "no_wdelay"),
 ("hide", "nohide"), ("no_subtree_check", "subtree_check"),
 ("secure", "insecure"), ("root_squash", "no_root_squash"),
 ("no_all_squash", "all_squash")]
 defaults = {group[0]: group[0] for group in groups}
 defaults.update(sec="sys", anonuid=65534, anongid=65534)
 owner = {token: group[0] for group in groups for token in group}
 seen = set()
 for token in match[3].split(","):
 if token in owner:
 name, value = owner[token], token
 elif token == "sec=sys":
 name, value = "sec", "sys"
 elif re.fullmatch(r"anon(?:uid|gid)=[0-9]+", token):
 name, raw = token.split("=")
 value = int(raw)
 if not 0 <= value < 2**32 - 1:
 raise ValueError("unsupported anonymous ID")
 else:
 raise ValueError(f"unsupported option: {token}")
 if name in seen:
 raise ValueError(f"duplicate or conflicting option: {name}")
 seen.add(name)
 defaults[name] = value
 return match[1], str(client), defaults

def compare_exports(configured, effective, *, complete, returncode, stderr):
 if (complete is not True or type(returncode) is not int or returncode != 0
 or type(stderr) is not str or stderr != ""):
 raise ValueError("failed, partial or unverified capture")
 if export_policy(configured) != export_policy(effective):
 raise ValueError("export path, client network or policy differs")

# Fabricated complete inputs exercise comparison only; no NFS command runs.
configured = "/srv/data 10.0.0.0/24(rw,sync,no_subtree_check,root_squash)\n"
effective = ("/srv/data\n\t10.0.0.0/255.255.255.0"
 "(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,"
 "root_squash,no_all_squash)\n")
compare_exports(configured, effective, complete=True, returncode=0, stderr="")
print("synthetic policy comparison passed; live NFS remains untested")

Expect only a synthetic comparison message. For live evidence, require complete successful captures from the same identified server, including all configuration inputs and intended overrides; never manufacture completeness from filtered output. Unsupported configurations require separate review. No capture, RPC, mount or access test ran. Localhost discovery and ownership listings prove no access; the complete authorized server/client exercise and any policy change remain held.

Sources

exports(5), export syntax, squash options, sync/async and the subtree_check defaults, checked 2026-09-16.
exportfs(8), the master export table in /var/lib/nfs/etab and how exportfs maintains it, checked 2026-09-16.
nfs(5), mount options including hard/soft, timeo and the obsolescence of intr, checked 2026-09-16.
showmount(8), querying the server’s export list from a client, checked 2026-09-16.

A read-only root restricts filesystem mutation within its scope

Wed, 23 Dec 2026 00:00:00 +0000

A read-only root filesystem prevents ordinary writes to the filesystem layer containing a service’s packaged code and configuration. Explicit mounts can still provide writable locations for runtime data. Its boundary is filesystem mutation. It does not by itself remove process privileges, prevent reading available data, or block communication with other services.

When it breaks

Applications often write outside their main data directory: temporary files, caches, lock files, generated configuration, or runtime state can appear during startup and uncommon operations. Making the root read-only can expose those assumptions as EROFS failures. Testing only a health endpoint may miss the path that generates a report, refreshes a cache, or performs shutdown cleanup.

A writable mount remains writable even when the container root layer is read-only. It can contain state that a compromised or faulty process can corrupt. Broadly mounting a writable directory over packaged paths can also defeat the intended protection for those paths. Inspect the effective mount layout rather than inferring the entire pathname tree’s behavior from one root-level option.

Read-only and confidential are different properties. A process may still read packaged files, execute existing programs, and send data through allowed network paths. The setting also does not guarantee protection after a container escape or against authority capable of changing the mount arrangement. Its contribution must be understood alongside credentials, capabilities, isolation, and application behavior.

A read-only root restricts its own layer; writable mounts, reading, execution and local communication remain separate.

Check it quickly

Use root inside a disposable Linux container with a read-only root, writable tmpfs at /tmp, and a separate writable volume at /work. Python 3.7 or later and /etc/os-release must be present; assertions must remain enabled. Earlier records report Linux 6.12.76 and Python 3.11.2, without host bind mounts, privileged mode or external networking. That run has not been reproduced here; these launch conditions are prerequisites, not all checked by the script:

from pathlib import Path
from tempfile import mkdtemp
import errno,os,socket,subprocess,sys,time,uuid

def cleanup_steps(steps, primary):
 failures=[]
 for label, action in steps:
 try:action()
 except BaseException as error:
 failures.append(label+': '+type(error).__name__+': '+str(error))
 if failures:
 message='cleanup incomplete or uncertain: '+'; '.join(failures)
 try:print(message,file=sys.stderr)
 except (OSError,ValueError):pass
 if primary is None:raise RuntimeError(message)
 return failures

def remove_file(path):
 try:path.unlink()
 except FileNotFoundError:pass

def require_empty(value):
 if value:raise RuntimeError('residual resources: '+str(value))

assert os.geteuid()==0,'run inside a disposable read-only-root container as root'
assert os.statvfs('/').f_flag&os.ST_RDONLY
name='xbcatm-readonly-'+uuid.uuid4().hex
print('fixture root-tree probe:',name,flush=True)
for base in [Path('/'),Path('/usr'),Path('/etc')]:
 path=base/name
 fd=None
 try:
 try:fd=os.open(path,os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600)
 except OSError as error:assert error.errno==errno.EROFS,error
 else:raise AssertionError('root filesystem accepted a new file')
 finally:
 primary=sys.exc_info()[1]
 if fd is not None:
 cleanup_steps([('probe descriptor',lambda:os.close(fd)),('unexpected root-tree file '+str(path),lambda:remove_file(path))],primary)
print('even this root process receives EROFS on paths in the read-only root, usr and etc tree')
for base in ['/tmp','/work']:
 assert not os.statvfs(base).f_flag&os.ST_RDONLY
 directory=None
 try:
 directory=Path(mkdtemp(prefix='xbcatm-write-',dir=base))
 path=directory/'data';path.write_bytes(b'allowed');assert path.read_bytes()==b'allowed'
 path.write_bytes(b'changed');assert path.read_bytes()==b'changed'
 finally:
 primary=sys.exc_info()[1]
 if directory is not None:
 cleanup_steps([('temporary data',lambda:remove_file(directory/'data')),('temporary directory',directory.rmdir)],primary)
print('explicit tmpfs and data-volume mounts remain writable, including overwriting their contents')
assert Path('/etc/os-release').read_text()
result=subprocess.run([sys.executable,'-c','print("executed")'],capture_output=True,text=True,check=True,timeout=5)
assert result.stdout.strip()=='executed'
def remaining(deadline):
 seconds=deadline-time.monotonic()
 if seconds<=0:raise TimeoutError('loopback exchange deadline expired')
 return seconds

def receive_exact(connection,count,deadline):
 chunks=[];received=0
 while received<count:
 connection.settimeout(remaining(deadline))
 chunk=connection.recv(count-received)
 if not chunk:raise EOFError('loopback peer closed before the complete payload')
 chunks.append(chunk);received+=len(chunk)
 return b''.join(chunks)

listener=client=connection=None
try:
 deadline=time.monotonic()+5
 listener=socket.socket();listener.settimeout(remaining(deadline))
 listener.bind(('127.0.0.1',0));listener.listen(1)
 client=socket.socket();client.settimeout(remaining(deadline));client.connect(listener.getsockname())
 listener.settimeout(remaining(deadline));connection,_=listener.accept()
 client.settimeout(remaining(deadline));client.sendall(b'local')
 assert receive_exact(connection,5,deadline)==b'local'
finally:
 primary=sys.exc_info()[1]
 cleanup_steps([(label,sock.close) for label,sock in [('connection',connection),('client',client),('listener',listener)] if sock is not None],primary)
print('reading packaged files, executing an existing program, and loopback network I/O still work')

The script first checks the root filesystem’s reported read-only flag. It then attempts exclusive creation of uniquely named files under the root, /usr, and /etc. Each fails with EROFS, even for this root identity. Exclusive creation and unique names ensure the probes cannot overwrite a preexisting configuration file.

The fixture next checks that /tmp and /work do not report that flag. Temporary files can be created, read, and overwritten in both mounts. This confirms that writable exceptions remain operational and also establishes their exposure: the root setting does not protect the contents of those deliberately writable locations against the same process.

It reads /etc/os-release and executes an existing Python interpreter successfully. This distinguishes read-only storage from blocked reads or execution. Disallowing modification of packaged files can be useful, but it is not a general restriction on what existing code may do with its remaining authority.

Finally, a listener and client exchange a harmless payload over loopback. The container has no external networking, but local socket operations still work. The result shows that the filesystem option does not itself disable networking. It does not establish an external connectivity policy or test communication with any production service.

This revised fixture is unrun. Receive accumulates short reads, rejects early EOF and shares a five-second monotonic deadline with connect, accept and send. The interpreter timeout is also five seconds. Guards precede creation; independent socket/file cleanup preserves the original exception and fails on cleanup-only errors. Root identity, container markers and read-only flags do not prove isolation; verify launch conditions independently. The launcher owns container/volume teardown. Inspect reported leftovers after failure. Process creation, stalled filesystems and abrupt termination remain outside these bounds. These probes cover selected operations only; failure of root writes neither covers every application path nor restricts future writable mounts.

Separate packaged files, scratch space, and persistent state

Choose writable paths from observed application needs and configure the application to use them. Keep the exceptions narrow enough to explain their purpose. A failure should lead to identifying the required path and lifetime, rather than reopening the whole root layer to accommodate one unexpected cache or temporary file.

Use tmpfs for suitable temporary data and account for its memory use and lifecycle. Docker notes that tmpfs pages can be written to swap; calling this mount memory-backed is not a guarantee that sensitive bytes never reach persistent storage. Use a separately managed volume when state needs a lifetime independent of the container’s writable layer, with appropriate backup and recovery. A writable volume supplies a storage location; it does not automatically establish crash durability, confidentiality, or a tested restore procedure.

For systemd-managed services, mechanisms such as ProtectSystem and designated state or runtime directories express related filesystem restrictions with their own documented semantics. Validate the service’s actual view and identity rather than translating an option name mechanically between platforms. Exercise startup, ordinary work, exceptional operations, and cleanup, and confirm both prohibited writes and required writes behave as intended.

Sources

Docker container run reference, read-only root filesystems and separately mounted writable paths, checked 2026-09-12.
Docker tmpfs mounts, temporary mount storage and lifecycle, checked 2026-09-12.
Docker volumes, volume lifetime independent of a container root layer, checked 2026-09-12.
systemd 252 execution environment, ProtectSystem and explicitly writable service directories, checked 2026-09-12.

Give durable state and scratch space different lifecycles

Thu, 03 Dec 2026 00:00:00 +0000

Durable application state and disposable scratch files need different retention, capacity, and recovery policies. Separating their paths makes those policies easier to express, but directories alone do not create persistence or isolate disk capacity. The storage backing each path determines what survives process restart, container replacement, host failure, or cleanup. A reliable layout connects each class of data to an explicit lifecycle.

When it breaks

A cleanup job that treats one mixed directory as disposable can remove committed records alongside temporary downloads. The reverse problem is backing up large rebuildable caches while missing the small database that actually matters. File extensions and age are weak substitutes for knowing which application owns a path and whether its contents can be reconstructed from an authoritative source.

Container restart and replacement also differ. Stopping and starting the same container normally retains its writable filesystem layer. Removing that container discards the layer, while an explicitly retained named volume can survive. A tmpfs mount has another lifecycle and loses its contents when the container stops. Calling all three locations temporary hides distinctions that affect real recovery procedures.

Two separate directories on the same full filesystem still compete for that filesystem’s free space. Moving scratch from /data/tmp to /scratch without changing storage policy does not automatically protect database writes. Capacity isolation requires an appropriate mount, quota, or other enforceable budget, together with monitoring that distinguishes scratch growth from durable-state growth.

Storage lifecycles differ across stop/start and replacement. The source also checks a bounded scratch mount and a reopened same-volume database copy; neither is independent disaster recovery.

Check it quickly

Run this script using host Python and Docker after obtaining the pinned image. The recorded environment used Docker 29.3.1 and Linux arm64 Python 3.11.16 with its SQLite module. It creates one private named volume and an eight-mebibyte tmpfs scratch mount. Only its own containers and volume are removed; the small scratch exhaustion stays inside the experiment:

import subprocess,uuid

BASE='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],text=True,capture_output=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def execute(container,code):return docker('exec',container,'python','-c',code).stdout.strip()

name='xbcatm-state-'+uuid.uuid4().hex[:10];volume=name+'-data';containers=[name+'-one',name+'-replacement']
try:
 docker('volume','create',volume)
 def start(container):
 docker('run','-d','--name',container,'--network','none','--mount','type=volume,src='+volume+',dst=/state','--tmpfs','/scratch:rw,size=8m',BASE,'python','-c','import time;time.sleep(180)')
 start(containers[0])
 execute(containers[0],'from pathlib import Path;import sqlite3;c=sqlite3.connect("/state/data.sqlite");c.execute("create table records(value text)");c.execute("insert into records values (?)",("committed",));c.commit();c.close();Path("/ephemeral").write_text("container-layer");Path("/scratch/crumb").write_text("scratch")')
 docker('stop','--time','1',containers[0]);docker('start',containers[0])
 execute(containers[0],'from pathlib import Path;import sqlite3;assert Path("/ephemeral").exists();assert not Path("/scratch/crumb").exists();c=sqlite3.connect("/state/data.sqlite");assert c.execute("select value from records").fetchall()==[("committed",)];c.close()')
 print('stop/start retains the container writable layer and volume data, while the tmpfs scratch mount loses its file')
 docker('rm','-f',containers[0]);start(containers[1])
 execute(containers[1],'from pathlib import Path;import sqlite3;assert not Path("/ephemeral").exists();c=sqlite3.connect("/state/data.sqlite");assert c.execute("select value from records").fetchall()==[("committed",)];c.close()')
 print('replacement loses the old writable layer but retains the named-volume database')
 execute(containers[1],'''import errno,os,sqlite3
fd=os.open('/scratch/fill',os.O_CREAT|os.O_WRONLY,0o600)
try:
 try:
 while True:os.write(fd,b'x'*1048576)
 except OSError as error:assert error.errno==errno.ENOSPC
finally:os.close(fd)
source=sqlite3.connect('/state/data.sqlite');source.execute('insert into records values (?)',('after-scratch-full',));source.commit()
backup=sqlite3.connect('/state/check-copy.sqlite');source.backup(backup);backup.close();source.close()
restored=sqlite3.connect('/state/check-copy.sqlite');assert restored.execute('select value from records order by rowid').fetchall()==[('committed',),('after-scratch-full',)];restored.close()
''')
 print('bounded scratch reaches ENOSPC while state still commits; a SQLite backup copy reopens with both committed records')
finally:
 docker('rm','-f',*containers,check=False);docker('volume','rm',volume,check=False)

The first container creates a SQLite database under /state, commits one record, writes /ephemeral in its ordinary writable layer, and puts a crumb under /scratch. The three writes have distinct storage locations even though the process accesses all of them through ordinary filesystem paths. SQLite commit is used for the durable application record rather than relying on an unclosed output buffer.

After stopping and starting that same container, the database record and /ephemeral still exist, but the scratch crumb is absent. The container command does not recreate these files during startup, so the observation is about storage lifetime rather than initialization hiding the loss. This distinguishes a process restart within a retained container from destruction of its writable layer.

The original container is then removed, and a replacement mounts the same named volume. Its database still contains committed, while /ephemeral is absent. The replacement gets a new writable layer but reconnects to the retained volume. This demonstrates the volume’s independence from one container’s lifecycle, without claiming protection against losing the Docker host or deleting the volume itself.

Next, the replacement fills its bounded scratch mount until a write reports ENOSPC. It can still insert and commit after-scratch-full into the database on the volume. The successful commit demonstrates the particular capacity boundary configured here. It does not imply unlimited durable storage or protection from exhausting some other shared resource, such as total host memory.

Finally, SQLite’s backup API creates a database copy that is closed, reopened, and checked for both committed records in order. This uses the database’s copying interface instead of assuming that copying one live database filename captures every required file consistently. The copy remains on the same experimental volume, so it validates copy readability and content, not an independent disaster-recovery backup.

Make policy follow the data contract

Use a stable state directory for authoritative application data and a separately managed scratch location for reconstructable work. The Filesystem Hierarchy Standard distinguishes application state under /var/lib; container paths can differ while preserving that semantic distinction. Name the directories in application configuration and document which component owns creation, permissions, cleanup, and migration.

Set scratch budgets deliberately and handle exhaustion without deleting unrelated state. A cache is disposable only if its source and reconstruction procedure are actually available. Some apparently temporary files are required for an in-progress transaction or recovery, so cleanup needs the application’s contract rather than an indiscriminate age rule.

Keep independent backups with retention appropriate to the data, and test restoring into a fresh environment. A volume surviving replacement is persistence evidence, not backup evidence; a same-volume copy cannot survive loss of that volume. Monitor the actual backing stores, verify restore contents, and make destructive volume removal an explicit lifecycle decision.

Sources

Docker volumes, data lifecycle independent of container writable layers, checked 2026-09-12.
Docker tmpfs mounts, temporary in-memory mounts and size limits, checked 2026-09-12.
Filesystem Hierarchy Standard: /var/lib, variable state information retained for applications, checked 2026-09-12.
SQLite online backup API, consistent database copying through the database API, checked 2026-09-12.

Filesystem · XB Field Notes

Extended attributes belong to the inode and to a namespace

When it breaks

Check it quickly

Preserve meaning across the workflow

Sources

Btrfs snapshots are copy-on-write, not copies

Shared extents, not copied blocks

Where the space actually goes

A restore point is not a replica

When it breaks

Check it quickly

Sources

Inotify reports changes, but it is not a complete history

When it breaks

Check it quickly

Reconcile names and object lifetimes

Sources

A sparse file separates logical length from allocated blocks

When it breaks

Check it quickly

Budget and preserve the intended representation

Sources

Hard links share an inode until a name is replaced

When it breaks

Check it quickly

Inspect identity and update behavior

Sources

A symbolic link stores a path, not a permanent target identity

When it breaks

Check it quickly

Choose the object and the traversal policy

Sources

NFS shares a remote directory over the network

The file and the table

What the options actually decide

Mounting, and the id mapping trap

When it breaks

Check it quickly

Sources

A read-only root restricts filesystem mutation within its scope

When it breaks

Check it quickly

Separate packaged files, scratch space, and persistent state

Further reading

Sources

Give durable state and scratch space different lifecycles

When it breaks

Check it quickly

Make policy follow the data contract

Further reading

Sources