# XB Field Notes — offline reading bundle Generated from the site's note Markdown sources for offline reading. Each section keeps its title and Sources section for traceability. --- ======================================================================== ## atomic-static-site-switch.md --- title: Publish a static site by switching complete directories date: 2026-06-01 description: Stage a complete artifact beside the live tree and replace the directory name only after verification. summary: Staging avoids file-by-file overwrites, but this two-step switch briefly removes the live path and still requires compatible assets for older pages. tags: - release - static-web - deployment series: - Release and recovery ShowToc: true cover: image: /visuals/atomic-switch.svg alt: "A chronological sequence shows live holding the old tree, live absent between two moves, and live holding the new tree. Move one renames live to rollback; move two renames stage to live." series_weight: 4 --- Copying new files directly over a live static directory creates a period where visitors can receive HTML from one release and assets from another. A safer pattern stages one complete tree beside the live tree, verifies it, then changes directory names. Diagram description: After same-filesystem and stage verification with one deployer, live contains old and stage contains new. Move one sends live to rollback, leaving live absent while stage and rollback remain. Move two sends stage to live. On success live contains new and rollback old; on failure live stays absent and an explicit restore is attempted. A failed restore requires inspection. Published results require public checks. These two moves leave live absent between them; crashes can prolong the gap. Restore must be attempted and checked. Old asset URLs must remain reachable across separate browser requests. ## Keep all release directories on one filesystem Keep the live tree, staged tree, and rollback tree on the same filesystem. These fragments assume one Bash session, GNU Coreutils, and a single deployer with exclusive access to these names. Stop on a failed command or either side of the copy pipeline: ~~~sh set -euo pipefail df -P /srv/www ~~~ Create a uniquely named stage directory under that parent. Do not reuse an unresolved variable or broad wildcard as a deletion target. ~~~sh stage=$(mktemp -d /srv/www/site.new.XXXXXX) ~~~ ## Populate and verify the stage Copy the generated artifact into the empty directory, then check representative files. The final command displays a count; compare it with the verified artifact’s expected count or manifest rather than treating any number as a pass: ~~~sh tar -C public -cf - . | tar -C "$stage" -xf - test -s "$stage/index.html" test -s "$stage/404.html" find "$stage" -type f | wc -l ~~~ Run link and checksum checks against the staged path when the tooling supports a directory argument. Also verify that the serving user can traverse the directories and read the files: `mktemp -d` starts with restrictive permissions, and the extracted artifact must supply the intended modes. A stage that has not passed verification must never acquire the live name. ## Switch names with an explicit rollback path Resolve every path before the first move: ~~~sh live=/srv/www/site stamp=$(date -u +%Y%m%dT%H%M%SZ) rollback=/srv/www/site.old-$stamp test -d "$live" && test ! -L "$live" || exit 1 test -d "$stage" && test ! -L "$stage" || exit 1 test ! -e "$rollback" && test ! -L "$rollback" || exit 1 df -P "$live" "$stage" live_device=$(stat -c %d -- "$live") || exit 1 stage_device=$(stat -c %d -- "$stage") || exit 1 test "$live_device" = "$stage_device" || exit 1 mv -T -- "$live" "$rollback" || exit 1 if mv -T -- "$stage" "$live"; then test -s "$live/index.html" else mv -T -- "$rollback" "$live" || { printf 'Restore failed; inspect %s and %s before retrying.\n' "$rollback" "$live" >&2 exit 1 } exit 1 fi ~~~ This is a two-step switch, not one atomic replacement: the live path is absent between moves, and a crash can leave it absent until recovery. GNU `mv -T` prevents an existing directory from silently becoming a container for the moved tree; it does not serialize competing deployers. Do not proceed if the filesystem check differs. Platform-specific atomic exchange or symlink replacement can remove the missing-name interval, but must be tested with the actual server. ## Verify the public edge After the switch, request the homepage, one deep route, one fingerprinted asset, and a missing path. Compare public bytes with the staged artifact. Keep the rollback directory until that check and the observation window complete. Even an atomic name switch does not make a browser’s HTML and asset requests one transaction. Keep older fingerprinted assets reachable for cached or in-flight HTML; merely retaining an unserved rollback directory does not do that. A filesystem rename also makes no promise of crash durability without the platform-specific persistence steps. ## Sources - [POSIX `rename()`](https://pubs.opengroup.org/onlinepubs/9799919799/functions/rename.html), name replacement semantics, checked 2026-06-01. - [Linux `rename(2)`](https://man7.org/linux/man-pages/man2/rename.2.html), filesystem and replacement behavior, checked 2026-10-15. - [GNU tar manual](https://www.gnu.org/software/tar/manual/tar.html), archive creation and extraction, checked 2026-06-01. - [GNU Coreutils: Target directory](https://www.gnu.org/software/coreutils/manual/html_node/Target-directory.html), `-T` and directory nesting hazards, checked 2026-10-15. - [GNU Coreutils: stat](https://www.gnu.org/software/coreutils/manual/html_node/stat-invocation.html), filesystem device-number checks, checked 2026-10-15. ======================================================================== ## backup-restore-rehearsal.md --- title: Backups are only useful after a restore rehearsal date: 2026-06-03 description: A backup plan should prove that the important files can be found, verified, and restored. summary: Choose a small recovery unit, verify it, and practice restoring it somewhere disposable. tags: - backups - recovery - operations series: - Release and recovery ShowToc: true cover: image: /visuals/backup-restore.svg alt: "Retrieve an archive and checksum from storage, verify the retrieved copy before use, and restore into an isolated disposable test target. Check files, permissions and elapsed restore time. A stored file alone is not proof of recovery." series_weight: 11 --- “We have backups” is not yet a recovery capability. A useful backup has a known scope, a known freshness, a verification step, and a restore procedure that someone has actually run. Diagram description: The source repository supplies authoritative rebuild inputs; the published artifact supplies last-published restore bytes. Retrieve archive and checksum, verify before extraction, and restore a trusted archive as an unprivileged user into an empty isolated target. Record checksums, permissions, dependencies, recovery targets and elapsed time. A checksum is not authenticity, and two nonempty pages are not full acceptance. A recovery rehearsal starts with identifiable inputs and a retrieved, verified stored copy. Restore into a disposable isolated target, then record files, permissions, dependencies and elapsed time. A checksum or existing archive alone does not demonstrate recoverability. ## Define the recovery unit For a static site, the source repository and the generated artifact have different purposes. The repository is the authoritative input; the artifact is the quickest way to restore the last published version. Keep both distinctions clear. At minimum, record: - the source commit or archive identifier; - the generator and theme versions; - the list of files included in the artifact; - the checksum of important downloads; - the retention period and storage location. Do not include credentials, private keys, cookies, or access tokens in a website backup. Those belong in a separate secret-management process. ## Verify before storing An archive that cannot be opened is not a backup. A minimal verification loop looks like this: ~~~sh set -eu tar -czf site-2026-06-03.tar.gz public/ tar -tzf site-2026-06-03.tar.gz >/dev/null sha256sum site-2026-06-03.tar.gz > site-2026-06-03.tar.gz.sha256 ~~~ Creating a checksum records a baseline; it does not read back the stored copy. Retrieve the archive and checksum from backup storage, then verify them before extraction. A matching checksum detects accidental change but does not authenticate an archive if an attacker can replace both files. Listing members also does not prove that the right files were backed up. ## Restore somewhere disposable Use a trusted archive, extract as an unprivileged user into an empty temporary directory or isolated host, then check entry points and a representative resource. An untrusted archive needs stronger isolation because paths and links can affect extraction. Do not test by overwriting the only live copy. ~~~sh set -eu sha256sum -c site-2026-06-03.tar.gz.sha256 restore_dir=$(mktemp -d) tar -xzf site-2026-06-03.tar.gz -C "$restore_dir" test -s "$restore_dir/public/index.html" test -s "$restore_dir/public/404.html" ~~~ The command is intentionally boring. A rehearsal should reveal missing files, wrong permissions, or undocumented dependencies before an incident does. ## Measure freshness honestly A backup schedule is meaningful only relative to the amount of work that can be lost. If the source changes once a month, daily snapshots may be unnecessary. If a resource is updated every hour, a monthly archive is not enough. Write down the target recovery point and recovery time, then choose storage and checks that can meet those targets. Avoid claiming “continuous protection” when the last verified restore is old. Keep at least one recoverable copy outside the live host’s failure and deletion boundary. Record the elapsed restore time, restored checksums, permissions, and any manual dependencies; two nonempty HTML files are a smoke test, not a complete recovery acceptance test. ## Sources - [GNU tar manual](https://www.gnu.org/software/tar/manual/tar.html), archive creation and verification behavior, checked 2026-06-03. - [Static-site release checklist](/resources/), local companion resource. - [GNU Coreutils: SHA-2 utilities](https://www.gnu.org/software/coreutils/manual/html_node/sha2-utilities.html), checksum creation and `--check` verification, checked 2026-10-15. ======================================================================== ## caddy-static-files.md --- title: A static server should have one obvious root date: 2026-06-04 description: Keep the Caddy file server, generated artifact, and rollback path aligned. summary: One root, one file_server, and a small header policy make a static deployment easy to inspect. tags: - caddy - static-web - deployment series: - Release and recovery ShowToc: true cover: image: /visuals/caddy-root.svg alt: "An explicit publication root containing files leads to a public response. A dashed arrow warns that a symlink can reach outside the root; the root is not a sandbox." series_weight: 2 --- The most useful property of a static deployment is that an operator can answer “which directory served this response?” without reading application code. Caddy's `root` and `file_server` directives provide that boundary directly. Diagram description: A complete directory is verified before becoming the Caddy root, while the prior known-good directory is retained through smoke tests. A dashed publication tree contains HTML, text and assets. An outward symlink can expose an outside target readable by the service user. Private files remain outside the tree; separate edge checks cover status, content type, redirects, certificate names and missing-route behavior. Make the serving directory explicit. Audit symlinks and read/traverse permissions, then verify the actual response at the edge. ## Make the root explicit A minimal site block can be kept close to the artifact layout: ~~~text www.example.test { root * /srv/example-site encode zstd gzip header { -Server X-Content-Type-Options nosniff X-Frame-Options DENY Referrer-Policy strict-origin-when-cross-origin } file_server } ~~~ The hostname and path are placeholders; `www.example.test` is not a publicly issuable production name. Use an owned, resolvable hostname for production and plan local TLS trust separately for a test environment. The release command should publish a complete directory, verify it, and only then make that directory the configured root. Keep the prior known-good directory until the new one has passed smoke tests. ## Check the edge, not only the file tree Request an HTML page, a downloadable text file, and a missing path. Compare content types and status codes with the local artifact. A 404 page that returns `200` is a deployment bug even if the page looks polished. When Caddy manages HTTPS, certificate provisioning and HTTP-to-HTTPS redirects are part of the edge behavior. Test the redirect from the bare HTTP listener and the certificate name from the public hostname. Do not infer success from a healthy process alone. ## A root is a publication boundary, not a sandbox Keep source control metadata, backups, credentials, and private configuration outside the published tree. Caddy follows symlinks inside the root, so a link can expose files outside it. Check both the artifact contents and the serving user’s read/traverse permissions. Also, merely placing `404.html` in the tree does not configure a custom error handler; verify the missing-route status and error-page behavior you intend. ## Sources - [Caddy `file_server` directive](https://caddyserver.com/docs/caddyfile/directives/file_server), file serving, root, and index defaults, checked 2026-10-15. - [Caddy Automatic HTTPS](https://caddyserver.com/docs/automatic-https), automatic TLS and issuance behavior, checked 2026-10-15. - [Static-site release checklist](/resources/), local companion resource. - [Caddy `header` directive](https://caddyserver.com/docs/caddyfile/directives/header), response header operations, checked 2026-10-15. - [RFC 7034 — X-Frame-Options](https://www.rfc-editor.org/rfc/rfc7034.html), the DENY framing policy, checked 2026-10-15. - [MDN: `X-Frame-Options`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options), framing options, checked 2026-10-15. - [RFC 6761 — Special-Use Domain Names](https://www.rfc-editor.org/rfc/rfc6761.html), reserved `.test` names, checked 2026-10-15. ======================================================================== ## certificate-chain.md --- title: A certificate chain is a path you should be able to print date: 2026-06-08 description: Check the server’s certificate list, the client’s trust anchor, and the requested hostname as separate parts of TLS validation. summary: Read the chain the edge actually presents, not just the leaf, before you trust the rollout. tags: - tls - certificates - debugging series: - Web security series_weight: 7 ShowToc: true cover: image: /visuals/cert-chain.svg alt: "The server sends the leaf and needed intermediates; the client builds a path to its own trust anchor and also checks hostname and validity." --- A TLS client must build a valid certificate path to an anchor it already trusts, and verify the requested hostname. Printing certificates is a useful diagnostic, but does not establish either result by itself. Diagram description: A server-sent leaf and needed intermediates feed client path building. A separately configured trust anchor supplies trust. Requested hostname and certificate validity are additional checks; printing or counting certificates proves none of these alone. The server supplies certificates; trust comes from the client’s anchor. Validate the path, requested hostname and validity period. Printed certificate counts cannot establish those results. ## The leaf is not the whole story The endpoint presents the *leaf* first. It contains the public key, validity period, and subject alternative names used for hostname checks. A public-CA deployment normally also sends the intermediates needed to build a path. Missing intermediates can break clients that do not already have them cached or cannot retrieve them. The self-signed root normally need not be sent: trust comes from the client’s configured anchors, not from a server including a certificate. A cross-signed certificate whose subject says “Root” can instead be an intermediate in the selected path. Certificate count alone is therefore not a completeness or trust test. ## Read the chain the edge actually sends OpenSSL documents `s_client -showcerts` as the server’s certificate list, not a verified chain. The earlier version of this note recorded this site snapshot: ```text certificate 0: CN=www.xbcatm.com issuer : C=US, O=Let's Encrypt, CN=YE1 notAfter : Nov 19 08:37:12 2026 GMT certificate 1: CN=YE1 (intermediate) issuer : C=US, O=ISRG, CN=Root YE certificate 2/3: Root YE / ISRG Root … (CA certificates) ``` The original report counted four blocks. Its raw handshake log is not retained here, and this review did not reproduce the live handshake; treat the values as a historical report, not current edge evidence. Let’s Encrypt’s chain documentation does describe YE1 and a cross-signed Root YE path, so those names alone are not evidence of an invalid chain. Which certificate terminates trust depends on the client’s trust store and path building. ## Keep issuance authorization separate Intermediates let a CA keep its root key offline and separate issuance operations. Revoking an intermediate can invalidate dependent leaf paths and may require replacement certificates; it does not preserve every leaf automatically. CAA authorizes a CA’s documented issuer-domain identifier at issuance time. It does not pin an intermediate, change browser trust, or prove that an existing certificate complied with historical CAA. The `dns-caa-records` note covers that separate policy check. ## Count for inventory, validate for trust The earlier inventory command was: ```text $ echo | openssl s_client -connect www.xbcatm.com:443 -servername www.xbcatm.com -showcerts 2>/dev/null | grep 'BEGIN CERT' | wc -l 4 ``` This is read-only network traffic, but it suppresses diagnostics and the final count can succeed even when an earlier command failed. Do not use `4`, or any other fixed count, as a health threshold. A leaf-only result warrants checking the expected issuer path rather than declaring every one-certificate chain invalid. For actual validation, configure hostname checking and failure on verification errors with OpenSSL’s `-verify_hostname` and `-verify_return_error` options, retain diagnostics, and record the trust store and client version. Also test the clients you support; one OpenSSL trust store cannot establish compatibility with all browsers. ## Sources - [RFC 8446 — TLS 1.3](https://www.rfc-editor.org/rfc/rfc8446.html), section 4.4.2 certificate-list order and optional trust-anchor omission, checked 2026-10-15. - [RFC 5280 — X.509 PKI](https://www.rfc-editor.org/rfc/rfc5280.html), section 6 trust-anchor input and path validation, checked 2026-10-15. - [Let's Encrypt FAQ](https://letsencrypt.org/docs/faq/), certificate compatibility overview, checked 2026-10-15. - [Let's Encrypt: Chains of Trust](https://letsencrypt.org/certificates/), YE1 and cross-signed Root YE paths, checked 2026-10-15. - [OpenSSL `s_client`](https://docs.openssl.org/3.0/man1/openssl-s_client/), presented-list and verification options, checked 2026-10-15. - [RFC 8659 — CAA](https://www.rfc-editor.org/rfc/rfc8659.html), issuance authorization rather than client trust, checked 2026-10-15. ======================================================================== ## certificate-expiry-check.md --- title: Certificate expiry is a date, not a dashboard color date: 2026-06-10 description: Check the certificate served for the real hostname and turn the remaining days into an actionable threshold. summary: Use SNI, inspect the public certificate, and alert before renewal becomes an incident. tags: - tls - caddy - monitoring series: - Web security ShowToc: true cover: image: /visuals/tls-expiry.svg alt: "Expiry evidence consists of notAfter and a chosen threshold. Separate verification asks about name, chain, and trust. SNI selects the virtual host and does not validate its certificate." series_weight: 1 --- An HTTPS process can be healthy while the public certificate is close to expiry, issued for the wrong name, or served differently on another edge. A useful check asks the public endpoint for the certificate it would give to a real client. Diagram description: A public hostname is sent as SNI to select a certificate. Observed subject, issuer, fingerprint, notBefore, and notAfter are evidence, not a validation result. Remaining time is compared with a service-specific threshold smaller than the normal certificate lifetime. Separate checks cover hostname, chain and trust store, notBefore, and revocation, with visible diagnostics. A pipeline can print a certificate while hiding verification errors, so its final exit code cannot establish TLS health. Inspect the certificate a real client sees, then verify separately. Expiry alerts need a service-specific response threshold, handshake checks, renewal-failure alerts, and sampling of relevant edges and address families. ## Include the hostname TLS virtual hosting depends on the server name. Pass the hostname as SNI and inspect the peer certificate: ~~~sh host=www.example.test openssl s_client -connect "$host:443" -servername "$host" /dev/null \ | openssl x509 -noout -subject -issuer -dates -fingerprint -sha256 ~~~ The command is read-only. It does not renew anything and it does not prove that every intermediate certificate is accepted by every client. It does give you the not-before, not-after, subject, issuer, and fingerprint of the certificate presented for that hostname. ## Alert on a threshold Choose a threshold that leaves time for a failed renewal, a DNS problem, and a human response. The exact number belongs to the service's recovery target; a universal “30 days” rule is not a substitute for that decision. The companion [TLS expiry check](/resources/) prints the expiry and exits non-zero when the remaining interval is below the threshold you pass. Run the check from outside the host when possible. A local process may see a different listener, certificate file, or proxy path than a visitor on the public network. ## Separate observation from verification SNI selects a virtual host; it does not verify that the certificate matches that name. This pipeline prints a certificate even when `s_client` reports verification problems, and stderr is suppressed. Use a separate hostname-and-chain verification check with visible diagnostics and a known trust store. Do not turn this inspection pipeline’s final exit code into a TLS-health verdict. The companion helper tests `notAfter` with OpenSSL `-checkend`; it does not establish hostname validity, trust, revocation status, or that `notBefore` has arrived. Pair expiry alerts with handshake checks and renewal-failure alerts. Choose a threshold smaller than the certificate’s normal lifetime, especially for short-lived certificates, and sample each relevant edge/address family. ## Sources - [Caddy Automatic HTTPS](https://caddyserver.com/docs/automatic-https), automatic TLS certificate management, checked 2026-10-15. - [OpenSSL `s_client` documentation](https://docs.openssl.org/3.0/man1/openssl-s_client/), SNI and certificate inspection options, checked 2026-10-15. - [OpenSSL `x509` documentation](https://docs.openssl.org/3.0/man1/openssl-x509/), certificate parsing, dates, and fingerprints, checked 2026-10-15. ======================================================================== ## content-type-and-nosniff.md --- title: Content-Type should describe the bytes you actually send date: 2026-06-11 description: Check media types at the public edge and use nosniff to make mismatches fail visibly. summary: A filename is only a hint; the response header is the browser-facing contract for the representation. tags: - http - web-security - static-web series: - HTTP foundations ShowToc: true cover: image: /visuals/content-type.svg alt: "A style request for /assets/site.css branches into schematic responses: 200 with CSS bytes declared text/css, or 404 with HTML bytes declared text/html. Nosniff is scoped to script and style destinations." series_weight: 1 --- A static server can return the correct bytes with the wrong `Content-Type`. The page may still look acceptable in one browser, while a download, stylesheet, feed reader, or security policy treats the same response differently. The public response header is the contract; the filename is only an input to server configuration. Diagram description: A CSS pathname leads to public-response inspection of status, actual bytes, and Content-Type. Correct CSS and an HTML error page are separate cases. The browser script/style destination boundary applies nosniff; failures require fixing responses, and public CDN/proxy headers must be checked independently of origin headers. A suffix cannot prove what arrived. Read the public response status, bytes, and media type together; an HTML error page at a CSS path is not a mapping test. Nosniff makes script and style type mismatches visible. ## Inspect representative formats Check one HTML document, stylesheet, SVG, feed, and downloadable script. Include headers but discard the body: ~~~sh for path in / /assets/site.css /favicon.svg /index.xml /downloads/check-http-headers.sh; do curl -sS -D - -o /dev/null "https://www.example.test$path" \ | awk -v path="$path" ' BEGIN { print "== " path } /^HTTP\// || tolower($1) == "content-type:" || tolower($1) == "x-content-type-options:" { print } ' done ~~~ Replace the example asset paths with files that actually exist in the release; an HTML 404 response at an invented CSS path is not a MIME-mapping test. Expected media types depend on the resource, but they should be specific and stable. HTML should normally include a character set. CSS and SVG should not be served as generic binary data. A shell helper offered for download should not be advertised as HTML. This loop displays headers; it does not assert success, inspect the discarded bytes, or exercise browser blocking. Confirm the resource against the release artifact and inspect an actual browser load as a stylesheet or script. Opening the same URL as a document tests a different request destination. ## Keep nosniff consistent `X-Content-Type-Options: nosniff` asks supporting browsers to respect the declared type for script and style destinations instead of guessing. For that Fetch check, a script-like destination needs a JavaScript MIME type, and a style destination needs `text/css`; a missing or unusable type also fails. This turns a mismatch into an observable failure rather than browser-dependent interpretation. The header does not repair a wrong type or make arbitrary content safe. Check the inventory while deploying it, and fix any rejected script/style response rather than removing the protection: ~~~text X-Content-Type-Options: nosniff ~~~ ## Check the server mapping Static servers usually derive types from extensions. If a new extension is served as `application/octet-stream`, decide whether that is intentional before adding a global override. A narrow mapping is easier to review than a rule that changes every response. Repeat the checks at the public hostname. A CDN or reverse proxy can replace a correct origin header, and a friendly error page can return HTML for a path whose suffix looks like CSS or JavaScript. ## Sources - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), representation metadata and `Content-Type`, checked 2026-10-15. - [MDN: `Content-Type`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Content-Type), media types and charset examples, checked 2026-10-15. - [MDN: `X-Content-Type-Options`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Content-Type-Options), `nosniff` behavior, checked 2026-10-15. - [WHATWG Fetch Standard](https://fetch.spec.whatwg.org/#should-response-to-request-be-blocked-due-to-nosniff), script-like and style destination blocking, checked 2026-10-15. ======================================================================== ## cookie-scope.md --- title: Cookie scope should be narrower than the application boundary date: 2026-06-15 description: Review Domain, Path, Secure, HttpOnly, and SameSite as one browser storage contract. summary: A cookie sent to more hosts or paths than necessary enlarges the impact of every mistake at those boundaries. tags: - http - web-security - privacy series: - Web security ShowToc: true cover: image: /visuals/cookie-scope.svg alt: "Omitting Domain keeps a cookie at its setting host. Domain=example.test broadens eligible destinations to that domain and eligible subdomains such as app and api. Path selects requests and is not authorization." series_weight: 5 --- Cookies are attached to requests according to browser rules, not according to the server component that originally set them. A broad `Domain` or `Path` can expose state to unrelated routes or subdomains, while missing transport and script restrictions enlarge the damage from other failures. Diagram description: Cookie scope begins with host-only versus Domain scope. Path selects request paths but is not authorization. Secure restricts transport without encrypting storage. SameSite controls cross-site context; None needs Secure in modern browsers. HttpOnly blocks document.cookie access but the browser still sends cookies. Supporting browsers require Secure, no Domain and Path=/ for __Host-. Deletion matches the name and domain/path scope, meets prefix and secure-origin requirements, and is separate from server session invalidation. Static-site checks include headers and browser storage. Review the dimensions together, but keep their roles separate: where cookies are sent, who can read them through JavaScript, and which stored identity a deletion targets. ## Prefer a host-only cookie When `Domain` is omitted, the cookie is scoped to the host that set it. Adding `Domain=example.test` allows it to be sent to eligible subdomains too. Use the broader form only when multiple hosts genuinely share the same application and trust boundary. Inspect the response without printing real cookie values into logs. Header names are case-insensitive, so redaction must work for lowercase HTTP/2-style names too: ~~~sh curl -sS -D - -o /dev/null https://app.example.test/login \ | awk ' tolower($0) ~ /^set-cookie:/ { sub(/^[^:]*:[^;]*/, "Set-Cookie: [value redacted]") print } ' ~~~ ## Review the attributes together - `Secure` restricts cookie transmission to secure transport, normally HTTPS; it does not encrypt the stored value. - `HttpOnly` prevents JavaScript access through `document.cookie`; it does not stop the browser from sending the cookie. - `SameSite` influences cross-site requests and must match the application's navigation or embedding needs. - `Path` controls when a cookie is sent, but it is not an authorization boundary between applications on the same host. A cookie using `SameSite=None` also needs `Secure` in modern browsers. Avoid copying that pair into applications that do not require cross-site use. Same-site is broader than same-origin: sibling HTTPS subdomains can be same-site while remaining different origins. `SameSite` therefore cannot replace the host boundary or the application’s CSRF defenses. Cookies are also not isolated by port, even with the `__Host-` prefix. ## Use prefixes when their constraints fit Cookie name prefixes such as `__Secure-` and `__Host-` let supporting browsers enforce additional requirements. A `__Host-` cookie must use `Secure`, omit `Domain`, and use `Path=/`, which makes the host-only intention explicit. ## Test deletion with the same scope Delete the same cookie name and domain/path scope using `Max-Age=0` or a past `Expires` value; not every attribute is part of its storage identity. Meet the prefix and secure-origin requirements when sending the deletion. Preserve whether `Domain` was omitted: expire a host-only cookie without adding `Domain`, and specify its original `Path`. The current 6265bis draft includes the host-only flag in storage identity, so matching only the displayed domain string is insufficient as a portable deletion recipe. Verify deletion in the target browsers. A logout that clears only one scope can leave another active, and server-side sessions or tokens may also need invalidation. ## Keep static sites cookie-free A genuinely static documentation site normally has no reason to set application cookies. Check representative public responses for `Set-Cookie` and inspect browser storage after navigation, including any third-party scripts. A curl header check alone cannot detect JavaScript-set cookies or other browser storage; avoid adding storage the site does not need. ## Sources - [RFC 6265 — HTTP State Management Mechanism](https://www.rfc-editor.org/rfc/rfc6265.html), cookie storage and scope, checked 2026-10-15. - [MDN: `Set-Cookie`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Set-Cookie), attributes and cookie prefixes, checked 2026-10-15. - [HTTPWG cookie specification draft](https://httpwg.org/http-extensions/draft-ietf-httpbis-rfc6265bis.html), SameSite and cookie-prefix requirements, checked 2026-10-15. ======================================================================== ## dns-caa-records.md --- title: CAA records should authorize the issuer you actually use date: 2026-06-17 description: Inspect inherited CAA policy before restricting which certificate authorities may issue for a domain. summary: CAA narrows certificate issuance policy, but a mistaken record can block renewal as effectively as an expired certificate. tags: - dns - tls - web-security series: - Web security ShowToc: true cover: image: /visuals/caa-policy.svg alt: "An illustrative effective CAA policy, issue letsencrypt.org, branches into CAA permits for an authorized CA and CAA blocks for an unauthorized CA. Existing certificates are unchanged." series_weight: 6 --- Certification Authority Authorization records let a domain holder state which certificate authorities may issue certificates for the domain. They are useful only when the record matches the real renewal path and the operator understands how policy can be inherited from parent names. Diagram description: Follow aliases and walk upward to the first relevant nonempty CAA set, then stop without merging parents. Use the CA issuer identifier; issuewild overrides issue for wildcard requests. CAA constrains issuance, does not perform an HTTPS trust test or revoke existing certificates. Stage renewal changes while the current certificate is healthy. CAA authorizes issuers at issuance time. Lookup stops at the first relevant nonempty record set; wildcard tags refine the rule. Existing certificates and HTTPS trust are separate boundaries. The displayed issuer is an example. ## Inspect the current policy Query the exact hostname and its parent names: ~~~sh for name in www.example.test example.test test; do printf '== %s\n' "$name" dig +noall +answer CAA "$name" done ~~~ These are placeholder names, queried through the configured resolver. An empty answer at the leaf does not necessarily mean there is no applicable policy: lookup follows aliases and stops at the first nonempty relevant CAA RRset while walking upward. It does not merge every parent’s restrictions. Inspect DNS status and authority information too; an empty answer-only display can hide `NXDOMAIN` or `SERVFAIL`. Compare authoritative answers with recursive results when diagnosing changes. ## Match the CA’s issuer identifier A record uses tags such as `issue` or `issuewild` and an issuer-domain value: ~~~text example.test. 3600 IN CAA 0 issue "letsencrypt.org" ~~~ The value is an authorization identifier, not the display name from a certificate. Confirm it in the certificate authority's current documentation. Without `issuewild`, `issue` also governs wildcard issuance. When `issuewild` is present, it takes precedence for wildcard requests; multiple records of the same tag grant alternatives rather than requiring every CA. CAA does not select a particular intermediate certificate. ## Stage the DNS change before renewal Add or tighten CAA while the current certificate is healthy. Wait through the previous TTL, query authoritative and recursive answers, and run a staging or renewal test supported by the ACME client. Keep a rollback record ready. Do not add a restrictive record during an expiry incident unless the issuer path has already been verified. A syntactically correct but wrong policy can turn automatic renewal into a hard failure. ## Observe issuance failures CAA checks happen at issuance time, not on every HTTPS request. Changing CAA neither revokes existing certificates nor proves that a historical issuance complied with the policy at that time. Certificate monitoring must therefore cover both current expiry and renewal attempts. Log the effective DNS answer with the ACME failure instead of recording only a generic authorization error. ## Sources - [RFC 8659 — DNS Certification Authority Authorization](https://www.rfc-editor.org/rfc/rfc8659.html), CAA lookup and property semantics, checked 2026-10-15. - [Let's Encrypt: CAA](https://letsencrypt.org/docs/caa/), issuer identifiers and operational guidance, checked 2026-10-15. ======================================================================== ## dns-change-window.md --- title: DNS changes need a cache-aware rollback window date: 2026-06-18 description: Lowering a TTL just before a change does not remove answers already cached with the old value. summary: Prepare the TTL ahead of time, observe authoritative and recursive answers, and keep both endpoints valid through the cache window. tags: - dns - deployment - operations series: - Release and recovery ShowToc: true cover: image: /visuals/dns-ttl.svg alt: "The old endpoint keeps serving across the DNS switch while the new endpoint is already ready. The overlap continues into observation; changing TTL cannot rewrite cached answers." series_weight: 8 --- Changing an address record is easy; predicting when every resolver stops using the old answer is not. The time-to-live value travels with a DNS response and controls its ordinary cache freshness. Lowering the TTL at the moment of a migration does not rewrite answers that were cached earlier. Diagram description: Lower the TTL at least one old TTL before switching. Two illustrative recursive caches retain old answers for different intervals after the authoritative switch. Old and new endpoints both serve the intended hostname through migration and observation. Authoritative evidence, recursive cache state, A and AAAA records, CNAME chains, and SOA-derived negative TTL are distinct. Stale answers, application caches, and long-lived connections prevent a universal shutdown deadline. TTL travels with each answer. Prepare before the switch, keep both endpoints valid, and use old-endpoint traffic plus a margin to guide retirement and rollback. ## Lower the TTL before the event If an existing record has a one-day TTL, lower it at least one old TTL before the planned switch. That gives caches time to fetch the shorter value while the authoritative record still points to the old endpoint. The exact interval belongs in the change plan rather than in an improvised command during the migration. Record authoritative answers separately. The following two probes compare the system-configured resolver with a public recursive resolver; neither is inherently an authoritative query: ~~~sh dig +noall +answer www.example.test A dig @1.1.1.1 +noall +answer www.example.test A ~~~ For authoritative evidence, query the zone’s actual nameserver without recursion and inspect the `AA` flag, status, answer, and authority sections. The remaining TTL shown by a recursive resolver is evidence of its current cache state; different resolvers can legitimately differ. Repeat for AAAA and any CNAME chain used by the service, not only A. ## Keep both sides valid During the switch, keep the old and new endpoints capable of serving the intended hostname. TLS names, redirects, and application configuration must agree on both sides. Removing the old endpoint immediately after the authoritative record changes creates an avoidable outage for clients that still hold the prior answer. Negative answers are cached too. Creating a name that recently returned `NXDOMAIN` can take time to become visible because the negative response may remain in recursive caches according to the zone's authority data. ## Raise the TTL after observation Once authoritative and representative recursive answers point to the new endpoint and the old cache window has elapsed, restore the normal TTL. Keep the change record with the old value, new value, authoritative answer, and rollback condition. TTL is not a universal shutdown deadline. Resolvers can serve stale data during resolution failures, and long-lived connections or application caches may outlast DNS freshness. Observe old-endpoint traffic and retain a margin; changing DNS back also has a cache window. For negative answers, inspect the SOA-derived negative TTL rather than assuming the new A record’s TTL applies. ## Sources - [RFC 1034 — Domain Names: Concepts and Facilities](https://www.rfc-editor.org/rfc/rfc1034.html), caching model, checked 2026-10-15. - [RFC 1035 — Domain Names: Implementation and Specification](https://www.rfc-editor.org/rfc/rfc1035.html), TTL field, checked 2026-10-15. - [RFC 2308 — Negative Caching of DNS Queries](https://www.rfc-editor.org/rfc/rfc2308.html), negative caching model and retained error TTL, checked 2026-10-15. - [RFC 8767 — Serving Stale Data](https://www.rfc-editor.org/rfc/rfc8767.html), stale answers during DNS resolution failures, checked 2026-10-15. - [BIND 9 `dig` manual](https://bind9.readthedocs.io/en/v9.20.23/manpages.html#dig-dns-lookup-utility), server selection and recursion controls, checked 2026-10-15. ======================================================================== ## head-request-parity.md --- title: HEAD should describe the same representation as GET date: 2026-06-22 description: Compare HEAD and GET metadata without confusing a missing body with a different resource. summary: HEAD is a metadata view of GET; status and representation headers should tell the same story. tags: - http - monitoring - testing series: - HTTP foundations ShowToc: true cover: image: /visuals/head-parity.svg alt: "GET and HEAD use matched URL, request headers, and release. Both response cards show type, validator, coding, and length categories. Only GET has content and checksum evidence. HEAD length can be omitted; if provided it describes GET." series_weight: 2 --- The `HEAD` method asks for the response a server would send to `GET`, without transferring the response content. It is useful for checks that need status, validators, type, or length, but only when the metadata remains aligned with the real `GET` path. Diagram description: Matched request context, including Accept-Encoding, feeds GET and HEAD metadata cards. A content/checksum arrow exists only under GET. An absent HEAD Content-Length is allowed; a present value describes corresponding GET content. A small GET checks payload, and method-specific or redirect differences need investigation. Match the URL, request headers, release, and content coding before comparing HEAD with GET. Permitted omissions are not automatic failures. Keep a small GET body or checksum check, because plausible metadata cannot prove that the payload is intact. ## Compare a normalized header set Dynamic headers such as `Date` will differ between requests. Compare the fields that describe the selected representation: ~~~sh url=https://www.example.test/downloads/manual.pdf curl -sS -I "$url" > head.headers curl -sS -D get.headers -o /dev/null "$url" for file in head.headers get.headers; do printf '== %s\n' "$file" awk ' /^HTTP\// || tolower($1) ~ /^(content-type:|content-length:|content-encoding:|cache-control:|etag:|last-modified:|vary:)$/ { print } ' "$file" done ~~~ Run this in a disposable working directory; the two header files are overwritten. Match the URL, request headers, and release version. Compression negotiation belongs in the comparison: send the same `Accept-Encoding` value to both requests. HEAD should describe the corresponding GET, but RFC 9110 permits omission of fields determined only while generating content. Missing `Content-Length` alone is not a failure. If present on HEAD, it must be the length the corresponding GET content would have, not zero just because HEAD has no body. Compare encoded lengths consistently and investigate changed validators or types rather than requiring byte-identical header lists. ## Do not use HEAD as the only release check A server can synthesize plausible headers while the file is missing, unreadable, or truncated. Keep one small `GET` in the smoke test and verify its body or checksum. `HEAD` reduces transfer cost; it does not prove the payload. ## Check redirects explicitly Without `--location`, `curl -I` reports the first response. That is useful when testing redirect policy. With `--location`, it prints header blocks for the followed responses, including intermediate redirects and the final destination. Record which question the command answers: ~~~sh curl -sS -I --max-redirs 0 https://old.example.test/manual curl -sS -I --location https://old.example.test/manual ~~~ ## Treat method-specific failures as configuration defects A static page that returns `200` to `GET` and `405` or `404` to `HEAD` usually exposes a proxy or routing inconsistency. Fix the method handling instead of teaching monitors to ignore it, unless the endpoint contract deliberately disallows `HEAD`. ## Sources - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html#name-head), HEAD method semantics, checked 2026-10-15. - [curl manual](https://curl.se/docs/manpage.html), `--head`, `--include`, and redirect options, checked 2026-10-15. ======================================================================== ## hsts-rollout.md --- title: HSTS should be rolled out with a recovery plan date: 2026-06-23 description: Increase max-age only after HTTPS, redirects, and every covered subdomain are ready. summary: HSTS is cached browser policy, so a configuration mistake can outlive the server change that created it. tags: - tls - web-security - deployment series: - Web security ShowToc: true cover: image: /visuals/hsts-rollout.svg alt: "Verify HTTPS, begin with a short HSTS lifetime, then observe before extending it. Subdomain scope and preload need separate decisions; recovery requires working HTTPS." series_weight: 2 --- HTTP Strict Transport Security tells a supporting browser to use HTTPS for a host during a cached period. That persistence is the feature and the risk: removing the header does not immediately erase policy already stored by clients. Diagram description: A rollout rail requires HTTPS and redirect checks, a short example policy and observation of site and renewal before choosing a duration. A domain tree confines includeSubDomains to the sending host’s descendants. Separate preload and recovery panels explain stored-policy limits. Extend HSTS only after observation. Decide descendant scope and preload separately. Recovery needs valid HTTPS and cannot instantly erase every stored or preloaded policy. ## Verify the HTTPS path first Before sending HSTS, confirm that the canonical hostname serves a valid certificate and that HTTP redirects to the intended HTTPS URL without depending on user input: ~~~sh curl -sS -D - -o /dev/null --max-redirs 0 http://www.example.test/ curl -sS -D - -o /dev/null https://www.example.test/ ~~~ Check representative deep routes and error pages. HSTS cannot repair a certificate error because the browser requires a valid HTTPS connection before accepting the policy. ## Start with a short max-age A staged rollout limits the recovery window while configuration is new: ~~~text Strict-Transport-Security: max-age=300 ~~~ After observing the site and renewal path, increase the duration deliberately. Record the date, chosen value, and rollback conditions. A final policy might use a year, but the number is a commitment rather than a scanner score. ## Treat includeSubDomains as a separate decision `includeSubDomains` extends policy to every subdomain. Inventory names first, including old services, delegated zones, and hostnames used only on internal networks. One HTTP-only descendant is enough to make the option disruptive for clients that received the parent policy. ## Preload is not ordinary header configuration Browser preload programs can distribute policy outside the normal response cycle. Their requirements and removal delays are stricter than setting a header. Do not request preload merely because a scanner recommends it; confirm long-term ownership of the domain and all covered names. ## Verify the public response ~~~sh curl -sS -D - -o /dev/null https://www.example.test/ \ | awk 'tolower($1) == "strict-transport-security:" { print }' ~~~ Test the canonical public edge, not only an origin port. Proxies may add, remove, or duplicate the header. ## Know what rollback can clear A valid HTTPS response with `max-age=0` clears that host’s learned policy only for clients that receive it. Removing the header or sending it over HTTP does not clear a cached policy. A child cannot opt out of a parent’s still-active `includeSubDomains`, and clearing a parent does not delete policies independently learned for children. Scope follows the host that sends the header: policy from `www.example.test` covers that host and, with `includeSubDomains`, its descendants, not the sibling `api.example.test` or the parent `example.test`. Keep HTTPS working during recovery. Preload removal is a separate browser-update process, not an immediate response-header rollback. ## Sources - [RFC 6797 — HTTP Strict Transport Security](https://www.rfc-editor.org/rfc/rfc6797.html), policy processing and scope, checked 2026-10-15. - [MDN: `Strict-Transport-Security`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Strict-Transport-Security), directives and deployment cautions, checked 2026-10-15. - [HSTS preload program](https://hstspreload.org/), submission requirements and removal constraints, checked 2026-10-15. ======================================================================== ## http-cache-headers.md --- title: Cache headers are a contract with two clocks date: 2026-06-25 description: Choose freshness and revalidation rules from how often a representation can change. summary: Separate browser freshness from origin validation, then verify the response headers that implement both. tags: - http - caching - web-performance series: - HTTP foundations ShowToc: true cover: image: /visuals/cache-clocks.svg alt: "A matrix distinguishes no-cache, which permits storage but requires validation before reuse, from no-store, which asks caches not to store the response." series_weight: 5 --- Caching is not a single switch. A response has a freshness decision made by a cache and a validation conversation with the origin. Mixing those two ideas produces either stale content or needless requests. Diagram description: A storage and reuse matrix is followed by stable and fingerprinted URL policies. A conditional GET sends the real ETag in If-None-Match: a weak match can return 304, while a changed validator normally returns a new 200. Weak equivalence need not mean equal bytes. Age, Vary, freshness-limited immutable, and previously cached lifetimes constrain reuse. Storage, freshness, and validation answer separate questions. Match the policy to how the URL changes, preserve validators exactly, and check shared-cache variants. ## Pick the change model first For an HTML document that may change after publication, `Cache-Control: no-cache` is usually a more accurate starting point than `no-store`. `no-cache` permits storage but requires a cache to validate before reuse; `no-store` asks caches not to store the response at all. The distinction matters when you want conditional requests and a cheap `304 Not Modified` response. For a fingerprinted asset such as `app.4f2c.js`, the URL changes when the bytes change. A long freshness lifetime can then be reasonable: ~~~text Cache-Control: public, max-age=31536000, immutable ~~~ That policy is unsafe for a stable URL whose contents can be replaced in place. If the URL is stable, prefer a shorter lifetime and a validator such as an entity tag (`ETag`) or a meaningful `Last-Modified` value. Choose who may store the response as well as how long it stays fresh. Use the unqualified `private` directive when shared caches must not store user-specific content; a browser’s private cache can still store it. `no-cache` alone does not provide that restriction. The `public` example above is for assets safe to share across users, not a default for personalized responses. ## Inspect both paths Make one ordinary request and one conditional request. The exact server flags vary, but the evidence you want is stable: ~~~sh curl -sS -D - -o /dev/null https://www.example.test/ curl -sS -D - -o /dev/null \ -H 'If-None-Match: "copy-a-real-etag-from-the-first-response"' \ https://www.example.test/ ~~~ Do not paste a made-up validator into a runbook. Copy the value from the first response or use a test fixture. For this conditional GET, a matching `If-None-Match` can yield `304` using weak comparison: it permits semantic equivalence, not necessarily byte identity. A changed validator normally produces a new `200` response. Keep the ETag’s quotes and any `W/` prefix intact. ## Keep intermediaries visible `Age`, `Vary`, and the cache-control directives explain why two clients can receive different results. If a response varies by `Accept-Encoding`, language, or another request field, that dimension belongs in `Vary`; otherwise a shared cache can reuse the wrong representation. The useful release check is not “the scanner found a cache header”. It is “the header matches the replacement policy, and a conditional request behaves as documented”. `immutable` applies during the advertised freshness lifetime; it is not a permanent ban on revalidation. Keep versioned URLs immutable and older assets reachable while old HTML can still reference them. Changing today’s cache header does not retroactively shorten a previously cached response’s lifetime, so a cache-policy mistake may require a new URL or an explicitly supported cache purge. ## Sources - [RFC 9111 — HTTP Caching](https://www.rfc-editor.org/rfc/rfc9111.html), sections 4–5, checked 2026-10-15. - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), sections 8 and 13, checked 2026-10-15. - [MDN: `Cache-Control`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cache-Control), directive semantics and defaults, checked 2026-10-15. - [RFC 8246 — Immutable Responses](https://www.rfc-editor.org/rfc/rfc8246.html), freshness-limited immutable semantics, checked 2026-10-15. ======================================================================== ## http-compression-negotiation.md --- title: Compression changes the selected HTTP representation date: 2026-06-29 description: Treat content encoding, validators, and Vary as one observable response contract. summary: Test compressed and uncompressed requests separately so caches and validators describe the bytes actually served. tags: - http - caddy - web-performance series: - HTTP foundations ShowToc: true cover: image: /visuals/compression.svg alt: "Accept-Encoding selects identity or encoded representations. The two byte-distinct variants carry different schematic strong ETags, A and B. Vary: Accept-Encoding tells shared caches to match the negotiated request field." series_weight: 3 --- Compression saves transfer bytes, but it also creates another representation of a resource. A client announces acceptable content codings with `Accept-Encoding`; the server describes the applied coding with `Content-Encoding`. Diagram description: A negotiation fork selects identity bytes A or encoded bytes B, requiring different strong tags. Vary controls matching; weak tags have a qualified alternative. A separate pipeline keeps encoded Content-Length in saved response headers while curl decodes the body before an artifact-checksum comparison. Compression selects representation bytes, so negotiation, cache matching, and validators must agree. Distinct coded bytes need distinct strong tags. Compare the decoded download with the artifact; an encoded Content-Length need not equal the decoded file size. ## Make negotiation visible When a response can change according to `Accept-Encoding`, include that request field in `Vary`. This tells shared caches that a compressed response and an identity response are not interchangeable cache entries. Compare the two paths instead of checking only a browser request: ~~~sh curl -sS -D - -o /dev/null \ -H 'Accept-Encoding: identity' https://www.example.test/ curl -sS --compressed -D - -o /dev/null \ https://www.example.test/ ~~~ Inspect `Content-Encoding`, `Vary`, `Content-Length` when present, and the validator. Strong entity tags must differ when content coding changes the representation bytes. A shared weak tag is possible for semantically equivalent variants, but it cannot support strong comparison. Check conditionals against the same negotiated representation. ## Compress appropriate content Text formats such as HTML, CSS, JSON, XML, and SVG usually benefit from compression. Formats that are already compressed might not. Caddy's `encode` directive negotiates supported encodings and normally avoids encoding a response without an appropriate content type or minimum size. Do not add compression solely to satisfy a score. Measure a representative response, confirm that the edge emits the intended headers, and verify that downloadable files retain the correct content type and checksum after transfer decoding. With `--compressed`, curl decodes the body automatically but leaves saved response headers unchanged. A reported `Content-Length`, when present, describes encoded content, so it need not equal the size of the decoded download. Compare a checksum with the artifact only after content decoding; HTTP transfer framing is a separate layer. The option also does not guarantee compression: inspect the actual response, especially for small or already-compressed files. ## Sources - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), content codings and `Vary`, checked 2026-10-15. - [Caddy `encode` directive](https://caddyserver.com/docs/caddyfile/directives/encode), content encoding selection and negotiation, checked 2026-10-15. - [curl manual](https://curl.se/docs/manpage.html), `--compressed`, checked 2026-10-15. ======================================================================== ## http-conditional-requests.md --- title: Validators decide whether a cached copy is still the answer date: 2026-06-30 description: ETags and Last-Modified let caches check whether a stored copy is current and avoid transferring the body when revalidation confirms it is unchanged. summary: Conditionals are the cache's second clock; a 304 means the stored copy is still the answer. tags: - http - caching - debugging series: - HTTP foundations series_weight: 9 ShowToc: true cover: image: /visuals/etag-validation.svg alt: "A cache sends a GET with If-None-Match and illustrative tag W/\"release-a\" to the origin. A matching response returns 304 so the cache can reuse its stored body; a changed representation returns 200 with a body." --- A cache can ask whether its stored copy is still usable without always downloading the body again. It sends a validator; a matching conditional GET can receive `304 Not Modified`, while a changed representation normally needs a new `200` response. Validation saves bytes on a hit, not on every request. Diagram description: The cache sends the exact weak ETag W/"release-a" in If-None-Match. Weak comparison matches the origin tag "release-a", giving a body-free 304; the cache updates metadata and reuses stored body A. Alternatively, origin tag "release-b" produces 200 and new body B for GET. HEAD uses the same status test but returns no body. Weak matching implies neither byte identity nor freshness. Illustrative tags; otherwise successful requests. GET/HEAD use weak comparison. HEAD has no body. Matching is neither byte identity nor freshness. Other methods normally return 412; If-Range cannot use weak ETags. ## Validator type and strength are different questions An `ETag` is an opaque, quoted value. It can be strong (`"release-a"`) or explicitly weak (`W/"release-a"`). Strong comparison requires both tags to be strong and equal; weak comparison compares the opaque values regardless of the `W/` marker. Weak tags can describe semantic equivalence even when bytes differ. `Last-Modified` is a timestamp with one-second HTTP-date resolution. It is usually weak unless the conditions in RFC 9110 section 8.8.2.2 establish it as strong. Do not classify every ETag as strong or every timestamp as unconditionally weak. The earlier site observation recorded: ```text etag: "dkx84jwth7nkc9e" last-modified: Mon, 24 Aug 2026 13:55:23 GMT ``` These are historical values; the original response log is not retained here and the live result was not rechecked in this review. Caddy’s file server normally sets both fields, subject to file metadata and configuration. ## Conditional GET uses weak comparison Send the ETag exactly as received in `If-None-Match`, including its quotes and any `W/`. The original note reported `HTTP 304` for the value above but printed an unquoted request value. That request text was not valid entity-tag syntax and should not be copied as a working probe. Obtain a current validator before testing a live endpoint. This socket-free exercise illustrates the comparison for individual, already-parsed tags; it is not a complete HTTP header parser: ~~~python def weak_value(tag): return tag[2:] if tag.startswith('W/') else tag def strong_equal(a, b): return not a.startswith('W/') and not b.startswith('W/') and a == b stored = 'W/"release-a"' current = '"release-a"' assert weak_value(stored) == weak_value(current) assert not strong_equal(stored, current) assert weak_value(stored) != weak_value('"release-b"') print("weak match: True; strong match: False; changed tag: no match") ~~~ For an otherwise successful GET or HEAD, a matching `If-None-Match` produces a body-free `304`. The same false precondition on another method normally produces `412 Precondition Failed`. If both `If-None-Match` and `If-Modified-Since` are present, the ETag condition takes precedence. Byte-range validation with `If-Range` cannot use a weak ETag. ## Republish without preserving a misleading validator An ETag is not necessarily a content hash. Caddy v2.10.2’s default file tag is derived from modification time and size; replacing bytes while preserving both can preserve the tag. Inspect the deployed version and any configured overrides, then make the release process update the validator’s inputs or use content-based versioning. Keep byte checksums in the release manifest and test both unchanged and changed responses. A checksum does not automatically become the server’s ETag. A `304` also needs the applicable cache metadata so a cache can update its stored response; it is not merely an empty success status. ## Sources - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), sections 8.8 and 13 on validator strength and preconditions, checked 2026-10-15. - [RFC 9111 — HTTP Caching](https://www.rfc-editor.org/rfc/rfc9111.html), section 4.3 on validation and updating stored responses, checked 2026-10-15. - [MDN: HTTP conditional requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Conditional_requests), conditional-request overview, checked 2026-10-15. - [Caddy v2.10.2 file-server source](https://raw.githubusercontent.com/caddyserver/caddy/v2.10.2/modules/caddyhttp/fileserver/staticfiles.go), `calculateEtag` inputs and default response fields, checked 2026-10-15. ======================================================================== ## http-method-semantics.md --- title: HTTP methods are a promise about side effects date: 2026-07-02 description: The HTTP method names what a request may change; a static origin can answer most of them without pretending to have a stateful contract. summary: Get the method boundary right once, and OPTIONS, HEAD, and idempotent writes stop being a surprise. tags: - http - semantics - debugging series: - HTTP foundations series_weight: 8 ShowToc: true cover: image: /visuals/http-methods.svg alt: "GET and HEAD request no state change; PUT and DELETE are not safe even though their intended repeat effect is idempotent." --- A request method tells the server what the client is asking it to do. A read-only file origin has a narrow contract, but “static site” alone does not prove that proxies or imported handlers expose no write operations. Check the route actually serving the resource. Diagram description: A semantic matrix separates safe GET and HEAD from unsafe but idempotent PUT and DELETE. A scoped Caddy v2.10.2 file route accepts GET and HEAD and rejects other methods with 405 and Allow, while earlier handlers may differ. Safe methods can still produce incidental logs; idempotency does not promise identical statuses or authorize every retry. Any supplied HEAD Content-Length describes the corresponding GET with the same coding; omission may be legal. Method support is separate from CORS permission. Separate requested state change from intended repeat effect, then inspect the actual route. The file-handler boundary is scoped; method support is separate from CORS permission. ## Safe and idempotent answer different questions GET and HEAD are safe: the client does not request a state-changing action. Incidental logging, metrics, or accounting can still change server state. PUT and DELETE are idempotent in their defined semantics, but are not safe: repeating the same request is intended to have the same effect as performing it once. That does not require identical response statuses, and it does not make every retry after a network failure appropriate. For a read-only static origin: - **GET** retrieves a representation. - **HEAD** describes the corresponding GET without response content. - **OPTIONS** asks about the target’s communication options; a file handler may reject it. - **POST / PUT / DELETE / PATCH** should not fall through to successful file serving unless a separate, intended handler implements their semantics. ## A Caddy file handler’s method boundary Caddy v2.10.2’s file-serving path accepts GET and HEAD and rejects other methods with `405` and `Allow: GET, HEAD`. Redirects, missing paths, error handlers, and other configured routes can answer earlier or differently, so this is not a universal response for every static-site URL. The earlier note recorded: ```text $ curl -s -o /dev/null -w 'OPTIONS -> HTTP %{http_code}\n' -X OPTIONS https://www.xbcatm.com/ OPTIONS -> HTTP 405 ``` The original response log is not retained here, and this review did not reproduce that live result. The command reports only status; it does not establish that the required `Allow` field was present. Inspect that field separately. `405` means a known method is not supported for this target; `501 Not Implemented` is the usual distinction when the server does not recognize or implement a method at all. ## HEAD parity allows documented omissions HEAD should describe the same selected representation as GET, while omitting content. Fields calculated only while generating the content may be omitted. If `Content-Length` is present, it must describe the corresponding GET content, using the same content coding. The earlier note reported this comparison: ```text HEAD content-length: 15890 GET bytes read : 15890 ``` Those historical numbers are not a fixed expected size or a current verification result. A missing HEAD length alone is legal; a wrong supplied length needs investigation. Follow the `head-request-parity` note for paired probes. The downloadable `check-http-headers.sh` helper inspects a GET response and does not perform this comparison automatically. ## Reject unintended writes without hiding the contract For a method recognized but disallowed on the target, return `405` with `Allow` listing the methods actually supported. The list is not necessarily restricted to safe methods on an application route. Do not return a plausible `200` page for an ignored write request. If browser clients need CORS preflight, define that OPTIONS behavior explicitly; method support and CORS permission are separate decisions. Test write rejection only against an authorized disposable fixture rather than assuming a public endpoint is harmless because the site looks static. ## Sources - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), safe/idempotent methods and HEAD/405/501 semantics, checked 2026-10-15. - [MDN: HTTP request methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods), method overview, checked 2026-10-15. - [curl manual](https://curl.se/docs/manpage.html), method and response-inspection options, checked 2026-10-15. - [Caddy v2.10.2 file-server source](https://raw.githubusercontent.com/caddyserver/caddy/v2.10.2/modules/caddyhttp/fileserver/staticfiles.go), method rejection after file selection, checked 2026-10-15. - [WHATWG Fetch Standard](https://fetch.spec.whatwg.org/#cors-preflight-fetch), CORS preflight processing, checked 2026-10-15. ======================================================================== ## http-range-requests.md --- title: Range requests should prove which bytes were selected date: 2026-07-06 description: Verify partial responses, Content-Range, and the behavior of unsatisfiable byte ranges. summary: A 206 response is useful only when its boundaries and total size describe the bytes that were returned. tags: - http - downloads - testing series: - HTTP foundations ShowToc: true cover: image: /visuals/range-request.svg alt: "Inclusive byte offsets 0 through 15 select exactly 16 bytes from a complete representation of N bytes." series_weight: 4 --- Range requests let a client ask for part of a representation. Media players, resumable download tools, and diagnostic clients use them, but a server that advertises or returns ranges incorrectly can produce silent corruption instead of a clean failure. Diagram description: A stable encoded representation of N bytes supplies the reference size. A ruler selects inclusive offsets 0 through 15, exactly 16 bytes. Outcomes distinguish matching 206 bytes and Content-Range, complete 200 when Range is ignored, and rejected unsatisfiable ranges starting at offset N with 416 and bytes */N. Strong If-Range validators prevent splicing different versions. Weak ETags are forbidden; a strong HTTP date qualifies only when no ETag exists. Changed representation requires a complete current body. Keep the representation and content coding stable. Match interval, total size and actual body bytes; a complete 200 can be valid when Range is ignored. Resume only with an appropriate strong validator. ## Establish the complete size Use an immutable test download of at least 16 bytes. Run the fragments in the same disposable shell session so `url` and `size` remain available. Their explicit `set -eu` stops after a failed transfer, size check, or comparison; inspect the HTTP status and range fields separately. Save the complete body, headers, and validator in that temporary working directory: ~~~sh set -eu url=https://www.example.test/downloads/archive.tar.gz curl -fsS -H 'Accept-Encoding: identity' -D full.headers -o full.body "$url" size=$(wc -c < full.body) test "$size" -ge 16 awk 'tolower($1) ~ /^(content-length:|content-encoding:|etag:|last-modified:|accept-ranges:)$/ { print }' full.headers ~~~ Check that the full response is `200` and has no unexpected content coding. The commands request the identity representation and do not use automatic decompression; byte ranges apply to the selected encoded representation, so changing encoding invalidates the comparison. `Content-Length` may be absent, which is why the complete body supplies the size. `Accept-Ranges: bytes` is an explicit signal, but absence of the header does not by itself prove that range requests are unsupported. Exercise the request. ## Request a known interval Ask for the first 16 bytes and keep both headers and body: ~~~sh set -eu curl -sS -D range.headers -o range.body \ -H 'Accept-Encoding: identity' -H 'Range: bytes=0-15' "$url" wc -c range.body head -c 16 full.body > expected.body cmp expected.body range.body awk 'tolower($1) ~ /^(content-range:|content-length:|etag:)$/ || /^HTTP\// { print }' range.headers ~~~ A successful byte-range response uses `206 Partial Content`. `Content-Range` should identify `0-15` and the complete representation length, while the body should contain exactly 16 bytes. The byte comparison deliberately tests the partial-body case. A valid full `200` larger than 16 bytes will fail it and stop this shell session; inspect the saved headers before calling that corruption. Conversely, a complete 16-byte file can pass `cmp` even with `200`, so status and `Content-Range` still need checking. ## Test an impossible range Start at the complete byte count: for a file of size N, the last valid offset is N−1. Unlike a fixed large constant, this is outside the known representation: ~~~sh set -eu curl -sS -D - -o /dev/null \ -H 'Accept-Encoding: identity' -H "Range: bytes=${size}-" "$url" ~~~ When the server rejects an unsatisfiable byte range with `416 Range Not Satisfiable`, it should include `Content-Range`, for example `bytes */12345`, to report the current size. A `200` response may be valid when the server ignores the range, but it must return the complete representation rather than a mislabeled fragment. ## Keep validators in the story Resumed downloads can use `If-Range` with a strong ETag. A weak ETag is forbidden; an HTTP date is allowed only when no ETag is available and the date qualifies as a strong validator. If the representation changed, the safe response is the full current representation, not bytes spliced from two versions. Test this behavior before calling a download resumable. ## Sources - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html#name-range-requests), range units, partial responses, and `Content-Range`, checked 2026-07-06. - [curl manual](https://curl.se/docs/manpage.html), `--range` and header options, checked 2026-07-06. ======================================================================== ## postmortem-leave-a-fact-record.md --- title: A postmortem should leave a fact record date: 2026-07-07 description: Structure an incident write-up around captured output so the review thread stays on evidence instead of memory. summary: A postmortem is a chronological map from detected symptom to verified fix, backed by the commands that produced each fact. tags: - postmortem - operations - incident series: - Release and recovery series_weight: 13 ShowToc: true cover: image: /visuals/postmortem.svg alt: "Detect, Scope, Fix and Verify appear on a left-to-right chronology, with a separate evidence sheet attached to each phase. Facts, uncertainty and prevention close the record." --- A postmortem earns trust when its claims can be re-checked. Record each observation’s source, collection time and timezone, command or query, and exit status where relevant. Logs, metrics, and clearly attributed human observations also count; distinguish confirmed facts from hypotheses and preserve redacted evidence rather than secrets. The four phases below keep a write-up reviewable at any later time. Diagram description: Four entries on a vertical incident timeline attach captures to Detect, tested boundaries to Scope, a configuration diff and intended result to Fix, and repeat evidence to Verify. Each entry preserves source, collection time and context. A dashed uncertainty box is separate from an owned preventive action with due date and acceptance check. Tie every phase to captured evidence. A post-fix sample is a bounded observation; causes, uncertainty and owned prevention still need explicit treatment. ## Detect: the symptom, with evidence Start with the smallest verifiable symptom and the command that showed it. Save the actual capture. The following is an illustrative excerpt, not evidence of a fresh probe: ~~~text $ check-http-headers.sh https://www.xbcatm.com/ HTTP/2 200 content-security-policy: ... script-src 'none' ... last-modified: Mon, 24 Aug 2026 11:20:54 GMT ~~~ If the symptom is that a header changed, the output showing the previous policy and the new one belongs side by side. If it is a failure, include the exit status and the error line. ## Scope: how wide, and how you know State the blast radius with evidence, not adjectives. Define the boundary you can inspect: - Which hosts and paths show the symptom? Record each probed URL and observation time; one successful request cannot establish the state of every edge or visitor. - Does the cache differ from origin? Compare actual responses, cache metadata, and validators from both paths using the same request headers. `Vary` describes selection inputs, not proof that the response came from cache. - What delays recovery? For a DNS change, record the old TTL and resolver observations. For HTTP caches, record freshness and revalidation policy separately; a DNS TTL is not a universal rollback deadline. ## Fix: the change and its intended effect Record the actual configuration diff and deployment reference so the corrective action can be understood and, where safe, replayed. This header fragment is an expected observation, not a shell command or a complete policy. Prefer the smallest change that restores the contract, and name what the post-fix run should show: ~~~text Content-Security-Policy: script-src 'none' ~~~ ## Verify: the post-incident run Repeat the detection command after the change and show that the observed output now matches the intended contract. A postmortem without this final section is a story; with it, it is a record that approves its own future re-check. Close the record with contributing causes, what remains uncertain, and preventive actions with an owner, due date, and acceptance check. A successful probe confirms that sampled path at that time; it does not establish the cause or prevent recurrence by itself. ## Sources - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), response metadata and conditional requests, checked 2026-07-07. - [artifact HTTP header check](/resources/), read-only inspection used throughout this record. - [Google SRE: Postmortem Culture](https://sre.google/sre-book/postmortem-culture/), impact, contributing causes, evidence review, and preventive actions, checked 2026-10-15. - [RFC 9111 — HTTP Caching](https://www.rfc-editor.org/rfc/rfc9111.html), cache selection and freshness are distinct from DNS cache lifetimes, checked 2026-10-15. ======================================================================== ## read-only-release-smoke-tests.md --- title: A release is not complete until the public edge agrees date: 2026-07-09 description: Compare the generated artifact with the responses a visitor can actually receive. summary: Validate one staged artifact, publish it, then compare representative public responses with those exact bytes. tags: - release - testing - static-web series: - Release and recovery ShowToc: true cover: image: /visuals/release-edge.svg alt: "HTML, download, and error-page artifact samples point to corresponding public GET bodies. Expected statuses are shown separately as 200, 200, and 404. These are post-selection samples to compare in English and Chinese." series_weight: 7 --- Static publishing removes application runtime complexity, but it does not remove the boundary between a generated directory and a public response. A release check should cover both. ![A small static release path](/diagrams/release-path.svg) Diagram description: Check the exact artifact at a staged server with intended configuration, then repeat at the public edge after switching directories. Pair index.html, the downloadable resource, and 404.html with decoded GET bodies. Compare their bytes separately from expected status, type, cache, and security headers. Repeat English and Chinese samples and retain the previous known-good directory until the new one passes. Use the same artifact before and after selection. The download row refers to public/downloads/check-http-headers.sh in the article. Supported content codings are decoded before cmp; server transformations need a documented rule. Status and header checks are separate. Retain the previous directory for a deliberate rollback. ## Start from a clean artifact Build with the pinned generator into a clean destination. Record the source commit, generator version, output file list, and checksums of files that users download. A clean build catches stale files that a copy-over deployment can accidentally preserve. ~~~sh set -eu make verify find public -type f -print | sort ~~~ The file list is evidence, not decoration. If a route disappears, the diff should explain whether the source was removed or the generator failed to emit it. ## Exercise three response classes Before publishing, test the staged artifact with the intended server configuration. After switching directories, repeat these checks at the public edge; probing the old live site cannot validate the new release. Request one HTML page, one static resource, and one missing path. Inspect status, content type, cache policy, and security headers: ~~~sh set -eu curl --compressed -fsS -D page.headers -o page.body https://www.example.test/ curl --compressed -fsS -D asset.headers -o asset.body \ https://www.example.test/downloads/check-http-headers.sh curl --compressed -sS -D missing.headers -o missing.body \ https://www.example.test/does-not-exist cmp public/index.html page.body cmp public/downloads/check-http-headers.sh asset.body cmp public/404.html missing.body ~~~ Run each fragment as a standalone shell script or in a disposable shell session. `set -eu` stops this sequential script after a failed build, transfer, or comparison, so a later successful command cannot hide an earlier failure. The missing-path request intentionally does not use `-f`: a 404 is expected and should still be captured. Check the final status in each header file: `200`, `200`, then `404`; the byte comparisons do not automate that status check. No redirect is followed here. `--compressed` decodes supported content codings before comparison; a deliberate server-side HTML transformation needs its own documented comparison rule. `cmp` is silent on equality and returns nonzero on a difference. Repeat for `/zh/` and its error page rather than assuming language parity. ## Keep rollback simple Do not delete the previous directory until the new one has passed the checks. A rollback should select a known-good directory, not reconstruct one from memory during an incident. The [release checklist](/resources/) and [HTTP header check](/resources/) are deliberately small enough to run from a clean shell. ## Sources - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), status and representation semantics, checked 2026-07-09. - [curl manual](https://curl.se/docs/manpage.html), response inspection options, checked 2026-07-09. - [Static-site release checklist](/resources/), local companion resource. ======================================================================== ## read-the-public-edge-in-two-commands.md --- title: Read the public edge in two commands date: 2026-07-13 description: Read response headers and certificate dates while keeping expiry, identity, and byte verification distinct. summary: Two small read-only helpers turn the public endpoint into evidence you can inspect before and after a release. tags: - debugging - operations - https series: - Release and recovery series_weight: 6 ShowToc: true cover: image: /visuals/field-output.svg alt: "An illustrative two-column ledger separates HTTP headers, final URL and status from TLS dates and expiry. Body equality requires another check." --- A deployment is complete when public responses match the release contract, not when the last upload exits zero. Two read-only helpers from the [operator kit](/resources/) expose useful evidence, but they do not prove the whole artifact was published correctly. Diagram description: Illustrative source excerpts show HTTP_STATUS 200 and FINAL_URL, then a full 2026 certificate expiry timestamp and a 30-day expiry-only result. Annotations distinguish shell success from HTTP status and expiry from notBefore, hostname and trust checks. Body comparison and SAN identity policy remain separate. These are illustrative excerpts. Evaluate HTTP status and expiry separately; verify hostname, trust and body equality with the checks that actually test them. ## Inspect the response headers `check-http-headers.sh` performs a GET, discards the body, follows redirects, and prints the response headers for each hop plus the final status and URL: ~~~sh check-http-headers.sh https://www.xbcatm.com/ ~~~ This illustrative output shows the fields to inspect. It is not a fresh production observation: ~~~text HTTP/2 200 accept-ranges: bytes content-security-policy: default-src 'self'; base-uri 'self'; form-action 'none'; frame-ancestors 'none'; object-src 'none'; script-src 'none'; style-src 'self'; img-src 'self' data: content-type: text/html; charset=utf-8 last-modified: Mon, 24 Aug 2026 11:20:54 GMT permissions-policy: accelerometer=(), camera=(), geolocation=(), gyroscope=(), microphone=(), payment=(), usb=() referrer-policy: strict-origin-when-cross-origin strict-transport-security: max-age=31536000; includeSubDomains x-content-type-options: nosniff x-frame-options: DENY HTTP_STATUS 200 FINAL_URL https://www.xbcatm.com/ ~~~ Check the expected status, final host and path, and the policy on the final response. The helper does not use curl's `--fail`: an HTTP 404 or 500 can still leave the shell command successful. Treat its printed status as data to evaluate. A header mismatch warrants investigation of server, proxy, or cache configuration; it does not by itself prove the wrong output directory is live. ## Inspect the certificate dates `check-tls-expiry.sh HOST [DAYS]` extracts the certificate presented with SNI and checks whether its expiry is at least the requested number of seconds away: ~~~sh check-tls-expiry.sh www.xbcatm.com 30 ~~~ An illustrative excerpt, with the fingerprint line omitted: ~~~text subject=CN=www.xbcatm.com issuer=C=US, O=Let's Encrypt, CN=YE1 notBefore=Aug 21 08:37:13 2026 GMT notAfter=Nov 19 08:37:12 2026 GMT certificate remains valid for at least 30 day(s) ~~~ The helper prints subject, issuer, dates, and SHA-256 fingerprint. It does not print the remaining day count. Its success message describes the expiry test only: the script does not enforce hostname or trust-chain validation, and `-checkend` does not validate `notBefore`. The subject CN is not sufficient hostname evidence; service identity is matched against the certificate's subject alternative names. A valid wildcard may legitimately cover the host. Use a separate verification when those properties matter: ~~~sh openssl s_client -connect www.xbcatm.com:443 -servername www.xbcatm.com \ -verify_hostname www.xbcatm.com -verify_return_error "$manifest" test -s "$manifest" ~~~ Sort filenames before hashing. NUL-delimited paths preserve spaces, tabs, and embedded newlines between tools; GNU `sha256sum` escapes special names in its ordinary checkable output. Do not parse that output with `awk '{print $2}'`: whitespace is legal in filenames. The separate assignment preserves a failed `find` exit status; testing only its output could mistake a failed scan for an empty result. Treat any failed scan or pipeline as an incomplete manifest and do not publish it. Stop writers while measuring; these commands do not create a filesystem snapshot. ## Verify after transport In one disposable Bash session on the destination, run the remaining fragments in order so the paths and `set` options stay in scope. Set the manifest and artifact paths to their actual locations: ~~~bash set -euo pipefail manifest=/path/to/release.sha256 artifact=/srv/www/site.new ( cd "$artifact" sha256sum -c "$manifest" ) ~~~ Checking listed files catches missing or changed bytes, but not extra files. Recompute the destination manifest with the same procedure and compare the complete records: ~~~bash observed=$(mktemp "${manifest}.observed.XXXXXX") trap 'rm -f "$observed"' EXIT ( cd "$artifact" nonregular=$(find . ! -type d ! -type f -print -quit) test -z "$nonregular" find . -type f -print0 | LC_ALL=C sort -z | xargs -0 -r sha256sum -- ) > "$observed" diff -u "$manifest" "$observed" ~~~ No diff and exit status zero means the regular-file paths and hashes agree. An extra stale file now fails too. The temporary output remains outside the artifact so the check never includes itself. Empty directories, permissions, ownership, and timestamps are not measured; validate those separately if deployment depends on them. ## Add an aggregate digest for release records Hash the manifest itself for a compact release identifier: ~~~sh sha256sum "$manifest" ~~~ Its meaning depends on fixed rules: relative paths, sorting locale, algorithm, tool format, and included metadata. Keep the trusted manifest with the source commit and builder version. ## Do not confuse integrity with authenticity Matching SHA-256 digests are strong practical evidence of matching bytes, not proof of authorship. An attacker able to replace both artifact and manifest can make the check pass. Signing and trusted distribution are separate controls; verify only a manifest obtained through the intended trusted channel. ## Sources - [GNU Coreutils: sha2 utilities](https://www.gnu.org/software/coreutils/manual/html_node/sha2-utilities.html), checksum generation and verification; reviewed through the linked upstream manual source, checked 2026-10-15. - [GNU Coreutils manual source](https://raw.githubusercontent.com/coreutils/coreutils/master/doc/coreutils.texi), untagged checksum escaping, `--check`, and NUL-delimited sorting, checked 2026-10-15. - [sha256sum(1)](https://man7.org/linux/man-pages/man1/sha256sum.1.html), command behavior and output format, checked 2026-07-16. - [GNU Bash: Simple Command Expansion](https://www.gnu.org/software/bash/manual/html_node/Simple-Command-Expansion.html), assignment-only commands retain command-substitution exit status, checked 2026-10-18. ======================================================================== ## reliable-defaults.md --- title: Reliable defaults beat clever recovery date: 2026-07-20 description: A predictable first path prevents more incidents than a complicated fallback can repair. summary: Keep the common path explicit, observable, and safe to repeat before adding recovery branches. tags: - reliability - operations - design series: - Release and recovery ShowToc: true cover: image: /visuals/reliable-path.svg alt: "One main build, check and publish path has a bounded retry side branch and a separate return to a retained known-good release." series_weight: 12 --- Small services often accumulate recovery logic before their normal path is fully understood. Every retry, alternate endpoint, and compatibility branch adds another state that must be observed and tested. Diagram description: The common path builds from pinned tools, validates a complete directory, publishes and verifies public responses, and retains the previous directory. Configuration, runtime and documentation agree on one selected release. Failure first asks whether it is transient and safe to repeat. Only then can retries consume an attempt-and-time budget; otherwise stop and correct. Permission errors are excluded. A lost response may already conceal success. Recovery explicitly returns to the retained known-good directory and records the outcome. Keep the common path visible. Qualify and bound retries, then recover to a retained known-good state rather than a partial copy. ## Make the common path boring A useful default is explicit, easy to inspect, and safe to repeat. For a static site, that can be as simple as: 1. Build from a pinned toolchain. 2. Check the generated HTML, assets, and complete staged directory. 3. Publish that directory, then verify public responses against it. 4. Keep the previous directory available for rollback. The sequence is less impressive than a multi-stage deployment system, but its state is visible. When something fails, the operator can answer whether the problem is in the source, the build, the copy, or the web server. ## Retry only a bounded operation Retries are useful when the failure may be transient and repeating the operation preserves its intended effect. Idempotence does not mean every response or incidental log entry is identical; after a lost response, the first attempt may already have succeeded. A download from an origin may be safe to retry; a database migration may not be. Treating every error as retryable hides the boundary that needs attention. When a retry is appropriate, record the attempt count and the final reason. A log entry such as "publish failed: checksum mismatch" is more actionable than a sequence of silent retries followed by a generic timeout. ## Prefer one authoritative state Configuration, runtime state, and documentation should agree about which directory is live and which command produces it. If a release can be copied to three different paths depending on who runs it, recovery becomes guesswork. For a static deployment, a short release record is enough: ~~~text release: 2026-07-20 source: git commit builder: Hugo 0.165.0 output: public/ verification: links + headers + checksums ~~~ The placeholder commit ID is deliberately resolved at release time instead of being invented in documentation. ## Recovery should restore a known state Recovery is easiest when it returns the service to the last known-good directory, not when it chooses among several partially built alternatives. Keep the previous output until the new output has passed its checks, then remove old copies according to a retention policy. The goal is not to eliminate failure. It is to make the failure small enough to explain and the recovery small enough to trust. Bound retries by both attempts and elapsed time. Use backoff with jitter when many clients could retry together, and avoid stacking retries at every layer. Permission errors and deterministic bad input need correction, not a longer retry loop. ## Sources - [RFC 9110 — HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), section 9.2.2 on idempotent methods and retries, checked 2026-07-20. - [Static-site release checklist](/resources/), local companion resource. - [AWS Builders’ Library: Timeouts, retries, and backoff with jitter](https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/), retry amplification, bounded retry budgets, backoff, and jitter, checked 2026-10-15. ======================================================================== ## robots-and-sitemaps.md --- title: robots.txt and sitemaps answer different questions date: 2026-07-21 description: Keep crawler preferences, URL discovery, and access control separate. summary: robots.txt is a crawler-facing preference; a sitemap is a URL inventory; neither is an authorization layer. tags: - web - discovery - operations series: - HTTP foundations ShowToc: true cover: image: /visuals/discovery-map.svg alt: "Three separate roles: robots.txt gives cooperating crawler preferences, sitemap gives a URL inventory, and the server provides access authorization. Neither file controls access." series_weight: 7 --- Two small files are often treated as if they were a security boundary. They are not. A `robots.txt` file describes crawler preferences, while a sitemap gives search systems a list of URLs that belong to the site. Diagram description: robots.txt at the origin root is a UTF-8 preference file for cooperating crawlers, not authentication or reliable de-indexing. Inspect the sitemap root: follow sitemapindex children such as language sitemaps, or read a direct urlset. Use canonical absolute URLs from one host. XML syntax alone is insufficient: check namespace, count, host and page existence; each sitemap is limited to 50,000 URLs and 52,428,800 uncompressed bytes, unchanged by gzip. Private routes need server authorization or a network boundary. RFC 9309 permits crawling for unavailable robots with 4xx; network/server errors initially require complete disallow. A reachable 200 HTML fallback is not proof of a correct robots file. Discovery aids do not grant or deny access. Follow every listed sitemap, validate more than XML syntax, and handle robots error responses according to their actual class. ## What robots.txt can and cannot do The rules are published at the origin's `/robots.txt` path and are read by cooperating crawlers. They do not protect a private directory, remove a URL from the network, or replace authentication. A route that must not be public needs server authorization or a network boundary. Keep the file boring and test its actual location: ~~~text User-agent: * Disallow: Sitemap: https://www.example.test/sitemap.xml ~~~ The file should be UTF-8. If a deployment redirects it, verify the crawler behavior you intend rather than assuming every client follows the same chain. ## What a sitemap adds A sitemap is an inventory, not a permission list. Use absolute URLs from one host, encode XML values correctly, and include only canonical pages that you are prepared to publish. Do not list a URL merely because it exists in a build directory. Each sitemap may contain at most 50,000 URLs and 50 MB (52,428,800 bytes) uncompressed; gzip does not raise that limit. An index is optional for a small site, but multilingual generators may legitimately produce one pointing to separate language sitemaps. Inspect the root element and follow each listed sitemap rather than assuming `/sitemap.xml` is a URL set. Validate it as XML and request it through the same HTTPS edge as a normal page. ~~~sh curl -fsS https://www.example.test/robots.txt curl -fsS -o sitemap.xml https://www.example.test/sitemap.xml && \ xmllint --noout sitemap.xml ~~~ Separate download and parsing so a failed fetch cannot be hidden by a successful parser. Well-formed XML is only a syntax check: also validate the namespace, URL count, canonical hosts, and whether listed pages exist. If `xmllint` is not installed, parse the XML in the same CI environment that publishes it. The important part is that the check runs before the file is replaced at the edge. A robots `Disallow` is not a reliable de-indexing instruction. Also test error responses: RFC 9309 permits crawling when robots is unavailable with a 4xx response, while a network/server error initially requires complete disallow. Accidentally serving the HTML fallback with status 200 can therefore change crawler behavior even though the URL appears reachable. ## Sources - [RFC 9309 — Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309.html), robots rules, file location, and user-agent matching, checked 2026-07-21. - [Sitemaps Protocol](https://www.sitemaps.org/protocol.html), sitemap XML structure and discovery, checked 2026-07-21. ======================================================================== ## security-headers-for-a-static-site.md --- title: Security headers should describe the actual page date: 2026-07-23 description: A compact policy is safer when it matches the resources a static page really loads. summary: Start with a narrow policy, then expand it only when a documented feature needs another resource. tags: - web-security - caddy - static-web series: - Web security ShowToc: true cover: image: /visuals/security-layers.svg alt: "An HTML page loads a local stylesheet and favicon. script-src none is appropriate only when no executable scripts are needed." series_weight: 3 --- Security headers are not a substitute for secure code, but they make the browser enforce useful boundaries. The policy should describe the page that is actually served, not a generic template copied from another project. Diagram description: The actual HTML, local stylesheet and favicon determine CSP resource allowances. Content-Security-Policy enforces; Report-Only reveals violations and needs separate reporting setup. Deliver CSP as a header: frame-ancestors is ignored in meta and has no default-src fallback, form-action needs its own directive, and style-src self alone does not allow inline styles. HSTS is learned from valid HTTPS responses; includeSubDomains at www.example.test covers that host and descendants, not example.test or api.example.test. Check normal, missing and asset responses and verify enforcement in a real browser. No browser test is claimed. Treat headers as a deployment contract: match actual resources, distinguish enforced and report-only policy, and verify exact host scope and browser behavior. ## Start from the resource graph If a site serves HTML, a local stylesheet, a favicon, and no executable JavaScript, a narrow policy can be understandable: ~~~text Content-Security-Policy: default-src 'self'; base-uri 'self'; form-action 'none'; frame-ancestors 'none'; object-src 'none'; script-src 'none'; style-src 'self'; img-src 'self' data: ~~~ The `script-src 'none'` choice is only correct while the site does not require executable scripts. Adding search, comments, or an analytics tag changes the resource graph and should trigger a deliberate policy review. ## Add transport and framing protections For an HTTPS-only site, the following headers are common starting points: ~~~text Strict-Transport-Security: max-age=31536000; includeSubDomains X-Content-Type-Options: nosniff X-Frame-Options: DENY Referrer-Policy: strict-origin-when-cross-origin Permissions-Policy: camera=(), microphone=(), geolocation=(), payment=(), usb=() ~~~ Only enable HSTS with includeSubDomains when every subdomain is prepared to serve HTTPS. A header is a contract with future deployments, not merely a score in a scanner. ## Check behavior, not just presence Request a normal page, a missing page, and a static asset. Error responses should carry the same important security headers as successful responses. Also confirm that compression and caching do not change the content type or expose an internal path. The [Caddy example](/resources/) in this site is a starting point, not a universal policy. Adapt the host names, certificate handling, and application routes to your own deployment. Deliver the CSP as a response header. `frame-ancestors` is ignored in a meta policy and does not inherit `default-src`; `form-action` also needs its own directive. `style-src 'self'` does not allow inline styles by itself, so test the generated theme, including dark mode. Keep `data:` in `img-src` only when the page actually needs data images. `Content-Security-Policy-Report-Only` can reveal violations during a rollout, but does not enforce the policy; a reporting endpoint needs separate configuration. Verify enforcement in a real browser as well as checking headers. HSTS is learned from valid HTTPS responses; with `includeSubDomains` it also applies to descendants of the host that sets it: a policy on `www.example.test` does not cover the parent `example.test` or sibling `api.example.test`. Gradually increase the lifetime after checking that exact scope. ## Sources - [MDN: Content Security Policy guide](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CSP), CSP directive names and syntax, checked 2026-07-23. - [MDN: `Strict-Transport-Security`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Strict-Transport-Security), HSTS directives and deployment cautions, checked 2026-07-23. - [Caddy `header` directive](https://caddyserver.com/docs/caddyfile/directives/header), response header manipulation directives, checked 2026-07-23. - [W3C Content Security Policy Level 3](https://www.w3.org/TR/CSP3/), response-header delivery, directive fallbacks, framing, and report-only policies; current working draft, checked 2026-10-15. - [RFC 6797 — HTTP Strict Transport Security](https://www.rfc-editor.org/rfc/rfc6797.html), valid HTTPS delivery and host/descendant matching in sections 8.1–8.3, checked 2026-10-15. ======================================================================== ## small-systems.md --- title: The advantage of small systems date: 2026-07-27 description: A system is easier to trust when its important behavior fits inside a single mental model. summary: Small does not mean unfinished; it means each boundary has a clear job and a visible owner. tags: - architecture - operations - simplicity series: - Release and recovery ShowToc: true cover: image: /visuals/small-system.svg alt: "A DNS lookup branch attaches only to the client. A separate HTTPS request travels from the client to the TLS edge and a known output directory, making the serving boundaries visible." series_weight: 1 --- Small does not mean unfinished. It means each component has a clear job, dependencies are deliberate, and the path from request to result remains visible. Diagram description: DNS lookup returns an address to the client without carrying HTTP. The request reaches the intended TLS edge and one known output directory. A file decision leads to intended content or a site-handled 404. Responsibility fields name DNS, edge and release owners; file provenance and recovery requirements remain explicit. Independent backup and recovery drills address single points of failure. Name each request boundary, its evidence and its owner. Fewer components reduce the paths to understand; backups and recovery practice still carry the recovery promise. ## Visibility is a feature When operators can name every boundary, they can also identify which evidence is missing. Logs become more useful, alerts become more specific, and routine maintenance becomes less risky. For a static website the important boundaries are usually easy to list: - DNS points the hostname to the intended edge. - TLS terminates at the intended proxy. - The proxy serves one known output directory. - The output contains only the files the site needs. - A 404 response is handled by the site, not by an accidental upstream. Writing this list down is more valuable than adding a dashboard that does not answer any of those questions. ## Complexity should earn its place A new layer is worthwhile when it removes more uncertainty than it creates. A build step that fingerprints assets can be useful when caching is important. A database is useful when content needs transactions. Neither is automatically an improvement for a small, mostly static site. The same rule applies to client-side JavaScript. If a page genuinely requires client-side scripting, choose the smallest implementation that meets the requirement and review its content security policy deliberately. For this site, that would first require revisiting the accepted zero-script architecture and its verification contract; it is not an exception to the current policy. If the page is a document, plain HTML is often easier to inspect and more resilient. ## Make ownership explicit Every file should have a reason to exist and a clear way to update it. A vendored theme should record its upstream version and license. A generated directory should be marked as generated. A resource download should say what it contains, when it was updated, and how its checksum was produced. These small statements prevent future operators from treating an accidental artifact as a supported interface. ## Leave room for the next change The best small system is not the one with the fewest lines. It is the one where the next change has a visible place to go. A notes site can start with Markdown, a static generator, and a web server; if it later needs search or comments, the new requirement can be evaluated on its own rather than assumed from the beginning. Small is not automatically fault-tolerant. A single edge, disk, or operator can still be a single point of failure. Choose acceptable recovery time and data loss, keep an independent backup, and rehearse recovery. The benefit of fewer components is a smaller set of failure paths to understand, not immunity to outages. ## Sources - [Hugo directory structure](https://gohugo.io/getting-started/directory-structure/), directory conventions for content and assets, checked 2026-07-27. - [Caddy `file_server` directive](https://caddyserver.com/docs/caddyfile/directives/file_server), static file serving defaults referenced by the design, checked 2026-07-27. ======================================================================== ## ssh-host-key-policy.md --- title: SSH host keys are part of the deployment record date: 2026-07-28 description: Keep first contact explicit and reject unexpected host-key changes. summary: Record the expected fingerprint, use a deliberate StrictHostKeyChecking policy, and treat changes as an incident until explained. tags: - ssh - security - operations series: - Web security ShowToc: true cover: image: /visuals/ssh-host-key.svg alt: "A trusted owner or console reference and an observed server host fingerprint feed a comparison before key installation. Under strict checking, a known matching key passes the host-key check, while new or changed keys stop." series_weight: 4 --- In public-key authentication, a client proves possession of its private key; it does not tell the client which server it reached. The server's host key and the local `known_hosts` database provide that second half of the identity check. Diagram description: Client private-key authentication and server host-key identity are separate checks. Compare the observed candidate with an independently trusted owner or console fingerprint before installation. yes refuses unknown and changed keys; accept-new accepts unseen keys without independent proof but refuses changes. Review rotations, keep private operational records, and do not treat ssh -G as a handshake. SSH server identity depends on comparing the observed host key with a separately trusted reference. Strict checking rejects unknown or changed keys. Trust on first use and a network key scan do not independently authenticate first contact. ## Separate first contact from rotation With `StrictHostKeyChecking=yes`, SSH refuses unknown hosts and changed keys. That is the safest default for a fixed production endpoint, provided the expected key is installed through a trusted channel. `accept-new` is a narrower convenience for a fleet where new hosts are expected: it accepts a never-seen key but still refuses a changed key. ~~~sshconfig Host production-site HostName www.example.test User deploy IdentityFile ~/.ssh/site_ed25519 StrictHostKeyChecking yes UserKnownHostsFile ~/.ssh/known_hosts ~~~ Do not “fix” a warning by adding `StrictHostKeyChecking=no` to a deployment script. It permits connections despite changed host keys, subject to client restrictions; warnings may still be printed, but the intended fail-closed identity check has been weakened. If a host is rebuilt, record the replacement fingerprint and review why the old key is no longer valid. ## Make the fingerprint reviewable Obtain the fingerprint from the host owner or console before the first connection, then compare it with what the client sees. A key change should have a ticket, maintenance note, or other explanation that another operator can inspect. The same rule applies to automation: the known-hosts file is configuration with security consequences, not disposable cache data. Back it up with the deployment record, but do not publish private keys or copied credentials with the site artifact. `accept-new` is trust on first use, not independent proof that the first server was genuine. `ssh-keyscan` can collect a candidate public key, but fetching it over the same untrusted network does not authenticate it. Compare its fingerprint with the separately trusted value before installing it. Inspect the effective configuration without connecting, using a standalone copy of the example saved as `deploy.conf`: ~~~sh ssh -G -F ./deploy.conf production-site | \ grep -E '^(hostname|user|stricthostkeychecking|userknownhostsfile) ' ~~~ Expect the configured host, user, known-hosts path, and strict checking (`true` or `yes`, depending on client output). This parses configuration only; it does not test an actual handshake. For a nonstandard port, known-hosts entries normally use `[hostname]:port`. Keep public-key fingerprints with the private operational release record, not the public website; hostnames can reveal internal topology even though host public keys are not secrets. ## Sources - [OpenBSD `ssh_config`](https://man.openbsd.org/ssh_config), `StrictHostKeyChecking` and `UserKnownHostsFile`, checked 2026-07-28. - [OpenSSH `ssh-keygen` manual](https://man.openbsd.org/ssh-keygen), fingerprint inspection, checked 2026-07-28. - [OpenSSH ssh-keyscan](https://man.openbsd.org/ssh-keyscan), unverified collected keys do not prevent man-in-the-middle attacks, checked 2026-10-15. - [OpenSSH ssh](https://man.openbsd.org/ssh), -G prints evaluated client configuration without opening a session, checked 2026-10-15. ======================================================================== ## static-site-release-checklist.md --- title: A release checklist for a static site date: 2026-07-30 description: A compact pre-publish check for generated HTML, links, assets, and response headers. summary: Verify the artifact before replacing the live directory, and keep the checks close to the release command. tags: - static-web - release - testing series: - Release and recovery ShowToc: true cover: image: /visuals/release-checklist.svg alt: "One clean public/ artifact feeds required-file inspection, staged-to-public route checks, and a release record. A separate reminder states that a source preview rebuilds. No check is marked complete." series_weight: 3 --- A static site is simple to publish, but simplicity does not remove the need for a release check. The useful checks are small enough to run on every release and specific enough to fail for a real reason. Diagram description: Four numbered release phases build a clean artifact with a pinned generator, require nonempty index.html, 404.html, robots.txt, and sitemap.xml plus reviewed links, test the exact artifact at staged and then public routes, and record commit, generator version, and download checksums. A separate Hugo preview rebuilds source; HEAD checks metadata while GET enables body comparison. The checks belong to the generated artifact. Clean output does not establish reproducibility; placeholder searches are review aids, with parser-based link checks as the gate. Serve the exact artifact before publishing, compare public GET bodies afterward in both languages, and keep the commit, generator version, and download checksums. ## Build a clean artifact Use a pinned generator version and build into a disposable output directory: ~~~sh set -eu hugo version hugo --cleanDestinationDir --gc --minify --destination public ~~~ `--gc` collects unused build-cache resources; it does not clean stale published pages. `--cleanDestinationDir` removes stale destination files, so use it only on the disposable artifact directory, never a directory containing source or unrelated data. Compare the printed version with the project's pin; printing a version does not enforce it. Record the build inputs and verify the resulting artifact. A clean destination does not establish byte-for-byte reproducibility: timestamps and other nondeterministic inputs need separate control and comparison. Do not edit generated HTML by hand after the build. ## Check the artifact, not only the source Inspect the generated tree and verify that expected entry points exist. Run this assertion block as a standalone shell script; it must stop before printing the inventory if any required file is missing or empty: ~~~sh set -eu test -s public/index.html test -s public/404.html test -s public/robots.txt test -s public/sitemap.xml find public -type f -print | sort ~~~ Then search for accidental placeholders, local paths, and development-only hosts: ~~~sh rg -l 'localhost|127\.0\.0\.1|TODO|example\.com|/Users/' public --glob '*.html' ~~~ This is a review aid, not an automatic release gate: examples like this checklist deliberately match, including in translated pages. This HTML-only scan does not cover XML feeds or JSON search output. `rg` exits 1 when there are no matches and 2 on an error. Review each listed file, distinguish fenced examples from active links or attributes, and use a parser-based link check for the actual gate. A private path in a published link is a release mistake. ## Exercise the public routes An isolated production-like server pointed at this exact artifact gives the strongest pre-release routing check. Hugo’s development server rebuilds content; the following is a separate source preview, not evidence that the already checked `public/` bytes were served. Run the first command in one terminal and the requests in another: ~~~sh hugo server --bind 127.0.0.1 --port 1313 --baseURL http://127.0.0.1:1313/ \ --disableFastRender --disableLiveReload --renderToMemory curl -I http://127.0.0.1:1313/ curl -I http://127.0.0.1:1313/notes/ curl -I http://127.0.0.1:1313/resources/ curl -I http://127.0.0.1:1313/missing-page/ ~~~ HEAD checks do not compare bodies. Repeat with GET against the staged server, then after publishing compare public bodies with the artifact and check both languages. Confirm that the missing route returns a 404 status in the production server. A pretty error page with a 200 status still breaks monitoring and caches. ## Record the result Keep the source commit, generator version, and checksum of each downloadable resource in the release record. This makes it possible to compare a live file with the source that produced it without relying on memory. The downloadable version of this checklist is in the [Resources](/resources/) section. ## Sources - [Hugo command reference](https://gohugo.io/commands/hugo/), generator flags for a clean production build, checked 2026-07-30. - [curl manual](https://curl.se/docs/manpage.html), HTTP verification and header inspection options, checked 2026-07-30. - [Static-site release checklist](/resources/), downloadable companion resource. - [Hugo server command reference](https://gohugo.io/commands/hugo_server/), development rebuilds, memory rendering, and live-reload controls, checked 2026-10-15. ======================================================================== ## systemd-restart-policy.md --- title: Restart policies need a rate limit date: 2026-08-03 description: Let systemd recover transient failures without hiding a permanent configuration error. summary: Use an explicit service type, bounded restart behavior, and start-rate limits that leave evidence in the journal. tags: - systemd - operations - reliability series: - Release and recovery ShowToc: true cover: image: /visuals/restart-loop.svg alt: "Five numbered starts lie within a 60-second bracket. The next attempt is blocked. The configured limit is a rate cap, not a lifetime retry budget." series_weight: 9 --- Automatic restart is useful for a long-running service, but an unbounded loop can turn a clear failure into a noisy one. The unit should describe both the normal process lifecycle and the amount of recovery that is acceptable. Diagram description: Execution, application readiness and healthy responses are separate checks. An illustrative configured window allows five starts in 60 seconds, with five-second delays after rapid failures. A further attempt hits the limiter and automatic restarts stop. Slower failures can remain below the rate indefinitely. Time passing alone does not schedule another attempt; fix the cause and deliberately start or clear the counter before starting. Retain the first failure log and verify readiness and healthy responses. Count starts within the configured window. A restart rate limit stops rapid churn; it neither proves health nor caps lifetime retries. ## Track setup failures For a long-running process, `Type=exec` makes failures to execute the configured program visible during start-up rather than treating a fork as success. Pair it with an explicit `ExecStart` and the least-privileged `User` that can perform the work. ~~~ini [Unit] Description=Example foreground site service StartLimitIntervalSec=60s StartLimitBurst=5 [Service] Type=exec User=site ExecStart=/srv/site/bin/server Restart=on-failure RestartSec=5s ~~~ `Type=exec` confirms execution, not application readiness. Keep the program in the foreground; use `Type=notify` when the application supports explicit readiness notification. `Restart=on-failure` is a recovery policy, not a health check. If the process stays alive while serving bad responses, systemd will not know that the site is broken; an external smoke test still has a job. ## Bound the loop Systemd applies start-rate limiting through `StartLimitIntervalSec=` and `StartLimitBurst=`. These directives belong in `[Unit]`, while `RestartSec` belongs in `[Service]`. The example allows five starts within 60 seconds; rapid failures separated by five-second restart delays can therefore hit the limit. A longer delay can keep a loop below that rate forever: this is not a lifetime retry budget. Choose limits from the expected failure pattern. When the limit is hit, the journal should contain the evidence needed to distinguish “restarted successfully” from “gave up after repeated failure”. After installing or editing the unit on an authorized test host, validate the file, reload the manager, and inspect its effective configuration before restarting. The commands below change service state and can interrupt traffic: ~~~sh systemd-analyze verify /etc/systemd/system/example.service systemctl daemon-reload systemctl cat example.service systemctl show example.service -p Restart -p RestartUSec \ -p StartLimitIntervalUSec -p StartLimitBurst systemctl restart example.service systemctl status --no-pager example.service journalctl -u example.service -n 80 --no-pager ~~~ Do not use a restart policy to conceal missing files, invalid permissions, or a bad environment. Fix the first failure, then test the recovery path. Once start limiting stops automatic restarts, waiting for the interval alone does not schedule another attempt. After fixing the cause, either start it after the interval or deliberately use `systemctl reset-failed example.service` to clear the counter before starting. Test missing-executable and rapid-crash cases in an isolated service, and retain the first failure log rather than only the final “start limit hit” message. ## Sources - [systemd.service](https://man7.org/linux/man-pages/man5/systemd.service.5.html), service type, restart, and start-rate limiter, checked 2026-08-03. - [systemd.unit](https://man7.org/linux/man-pages/man5/systemd.unit.5.html), common unit sections and drop-in semantics, checked 2026-08-03. - [systemd.service upstream manual (v257)](https://raw.githubusercontent.com/systemd/systemd/v257/man/systemd.service.xml), Type=exec/oneshot, readiness, activation timeout, and restart semantics, checked 2026-10-15. - [systemd.unit upstream manual (v257)](https://raw.githubusercontent.com/systemd/systemd/v257/man/systemd.unit.xml), start-limit section placement, rate window, and reset-failed behavior, checked 2026-10-15. ======================================================================== ## systemd-timer-maintenance.md --- title: A maintenance timer should leave its last result visible date: 2026-08-04 description: Pair a oneshot service with a timer, bounded runtime, and journal evidence. summary: Scheduling is only half the job; operators also need to see when the task last ran and how it ended. tags: - systemd - maintenance - operations series: - Release and recovery ShowToc: true cover: image: /visuals/maintenance-timer.svg alt: "A timeline separates calendar scheduling, oneshot work and the fresh result. Timer firing alone does not establish success." series_weight: 10 --- Cron-like syntax is not the whole contract for a maintenance task. The command needs a clear service identity, a bounded runtime, and evidence that distinguishes “not scheduled” from “ran and failed”. A systemd timer and oneshot service keep those concerns separate. Diagram description: Hourly scheduling, randomized delay, oneshot activation and result evidence appear as separate timeline stages. Missed calendar events coalesce into one catch-up and still incur random delay. A two-minute activation timeout is qualified by stop handling and bounded helper requests. Success can end inactive; Result, exit status and a fresh journal timestamp must agree. An already active service is not started again by the recurring timer. A fired timer is only a scheduling event. Judge this run by its service result, exit status and fresh timestamp. ## Define the work as a service Use an explicit user and absolute command path: ~~~ini # /etc/systemd/system/site-check.service [Unit] Description=Read-only public site check [Service] Type=oneshot User=sitecheck ExecStart=/usr/local/bin/check-public-site TimeoutStartSec=2min ~~~ `TimeoutStartSec` limits the oneshot activation, which includes its command runtime; oneshot services otherwise have no start timeout by default. Timeout handling can then require a stop timeout and cannot instantly end uninterruptible kernel I/O, so the helper also needs bounded network requests. The helper should return non-zero when its contract fails and must not modify the service it checks. ## Schedule the service ~~~ini # /etc/systemd/system/site-check.timer [Unit] Description=Run the public site check every hour [Timer] OnCalendar=hourly Persistent=true RandomizedDelaySec=5min Unit=site-check.service [Install] WantedBy=timers.target ~~~ `Persistent=true` triggers one catch-up activation when the timer is activated after missing at least one calendar event; it does not replay every missed hour. The catch-up is still subject to the randomized delay. `RandomizedDelaySec` spreads routine work across a window; it is not appropriate when the task must run at an exact instant. ## Validate before enabling After creating both unit files, validate them and reload the manager before starting the service manually. These are administrative operations on the intended test host, not read-only checks: ~~~sh systemd-analyze verify /etc/systemd/system/site-check.service \ /etc/systemd/system/site-check.timer systemd-analyze calendar hourly systemctl daemon-reload systemctl start site-check.service systemctl status site-check.service journalctl -u site-check.service -n 50 --no-pager systemctl show site-check.service -p Result -p ExecMainCode -p ExecMainStatus ~~~ Enable the timer only after the service succeeds with the intended user, environment, paths, and network access: ~~~sh systemctl enable --now site-check.timer systemctl list-timers site-check.timer ~~~ ## Alert on the service result The timer firing is not success. Monitor the oneshot service's exit status or journal result. Keep stdout concise and send detailed artifacts to a bounded path if they are needed for debugging. A successful oneshot normally returns to `inactive (dead)`; that alone is not a failure, and `systemctl status` can return nonzero for an inactive service. Correlate `Result` and exit status with a fresh journal timestamp so an old successful run is not mistaken for this one. Do not add `RemainAfterExit=yes` merely to keep it green: a recurring timer does not start a fresh instance while its service is already active. Calendar timing uses the host timezone unless specified, and normal timer accuracy adds scheduling slack beyond the random delay. ## Sources - [systemd.timer(5)](https://man7.org/linux/man-pages/man5/systemd.timer.5.html), calendar timers, persistence, and delay behavior, checked 2026-08-04. - [systemd.time(7)](https://man7.org/linux/man-pages/man7/systemd.time.7.html), calendar expression syntax, checked 2026-08-04. - [systemd.service](https://man7.org/linux/man-pages/man5/systemd.service.5.html), oneshot service semantics, checked 2026-08-04. - [systemd.service upstream manual (v257)](https://raw.githubusercontent.com/systemd/systemd/v257/man/systemd.service.xml), Type=exec/oneshot, readiness, activation timeout, and restart semantics, checked 2026-10-15. - [systemd.timer upstream manual (v257)](https://raw.githubusercontent.com/systemd/systemd/v257/man/systemd.timer.xml), persistence coalesces missed runs; active units, randomized delay, and accuracy, checked 2026-10-15.