Services · XB Field Notes

Kubernetes Services give pods a stable home

Mon, 23 Aug 2027 00:00:00 +0000

A Service separates a stable discovery interface from replaceable backends.

The before/after sets are model results, not a transport test. Stable discovery does not prove every request succeeds. Endpoint conditions, ports and datapaths need separate checks. The exact local model passed again; no Kubernetes or transport test was run.

One address in front of many

An ordinary ClusterIP Service keeps a virtual address during that object’s lifetime, and cluster DNS provides a discovery name. Callers can depend on that name while interchangeable Pods are replaced. This separates a stable client-facing contract from the lifecycle of individual backends, without requiring a config push for every replacement.

Headless and ExternalName Services have different discovery behavior, and Services without selectors can use separately managed EndpointSlices. Scope the stable-VIP claim to the ordinary case. A Deployment manages workloads rather than acting as a network client itself; applications, gateways and other components use the resulting service discovery.

Selection drives which pods respond

A selector-based Service identifies matching Pods in its namespace. EndpointSlice controllers reconcile their addresses and conditions asynchronously. Readiness normally determines eligibility for ordinary traffic, but an unready endpoint may remain present with a false readiness condition rather than being erased from the API object.

The selector does not authorize network access, and a label match does not prove a correct targetPort or useful application response. publishNotReadyAddresses and terminating-endpoint behavior also affect interpretation. Check the selected set and its conditions, then inspect the actual datapath; these are related observations, not interchangeable evidence.

Traffic goes to whatever matches

Scaling and replacement can happen behind one name, but request continuity is not guaranteed by that name. A connection already established to a backend can fail when the backend terminates. Updates take time to reach proxies, and an empty eligible set cannot serve traffic merely because DNS still resolves.

Plan readiness, rollout capacity, termination draining and client retries together. Retries also need application-level safety for non-idempotent operations. Observe availability during replacement rather than asserting that a precise selector prevents every interruption. The local model below demonstrates set selection only, not those timing and transport properties.

The stable name is a promise to callers

Treat the Service name as the supported discovery interface, rather than caching a Pod IP seen once. A caller using an ordinary Service VIP does not need to resolve again for every backend change, but resolving only once forever fails when the Service is recreated with a new IP. Respect resolution and connection lifetimes appropriate to the client.

Headless clients need their own discovery and balancing behavior because answers can contain changing endpoint addresses. Test how the library refreshes DNS, replaces connections and handles stale results. Stability is an API contract with lifecycle conditions, not an instruction to disable refresh or ignore failures.

When it breaks

The quietest failure is a selector that matches nothing. A typo such as app: webs instead of app: web, or a Deployment that stopped setting the expected label, leaves the Service with zero endpoints. The cluster IP and DNS name still resolve, so the outage looks like refused connections rather than a missing name. When a Service answers lookups but carries no traffic, inspect its endpoint set first; an empty eligible set can reflect labels, readiness, Pod addresses or reconciliation; it does not by itself rule out a separate network problem.

A headless Service chosen by habit is the next trap. Setting clusterIP: None removes the virtual IP, so DNS returns the changing pod addresses and each client must balance across them. Code written for one stable endpoint suddenly faces many shifting ones, with uneven retries and broken connection reuse. Choose headless only when the caller explicitly wants discovery.

Ports break the promise from the other side. A Service mapped to an incorrect container targetPort cannot deliver requests to the intended listener, and an already-allocated NodePort makes the API operation fail. ClientIP affinity can concentrate traffic from clients sharing an address. It does not promise permanent attachment to a removed endpoint or automatically turn every backend failure into a total outage. Choose affinity from the application requirement and test failover.

Check it quickly

Save the following as service_endpoints.py in a scratch directory and run python3 service_endpoints.py. It models endpoint convergence with plain dictionaries and changes nothing outside the process: one function derives the endpoint set from a selector plus pod labels and readiness, and four scenarios exercise it, including a deliberate empty selector as the negative control.

"""Simulate Service endpoint convergence from selector plus readiness."""

PODS = [
 {"name": "web-1", "labels": {"app": "web"}, "ready": True},
 {"name": "web-2", "labels": {"app": "web"}, "ready": True},
 {"name": "web-3", "labels": {"app": "web"}, "ready": False},
 {"name": "worker-1", "labels": {"app": "worker"}, "ready": True},
]


def select_endpoints(pods, selector):
 """Derive the endpoint set; an empty selector warns instead of matching."""
 if not selector:
 return [], "warning: empty selector matches nothing"
 matched = sorted(
 pod["name"]
 for pod in pods
 if pod.get("ready")
 and all(pod.get("labels", {}).get(key) == value for key, value in selector.items())
 )
 return matched, ""


def check(condition, message):
 if not condition:
 raise AssertionError(message)


def main():
 endpoints, warning = select_endpoints(PODS, {"app": "web"})
 check(endpoints == ["web-1", "web-2"], "ready web pods should form the set")
 check(warning == "", "a precise selector must not warn")
 print("steady state:", endpoints)

 rolling = [dict(pod, ready=True) if pod["name"] == "web-3" else pod for pod in PODS]
 rolling = [pod for pod in rolling if pod["name"] != "web-1"]
 endpoints, _ = select_endpoints(rolling, {"app": "web"})
 check(endpoints == ["web-2", "web-3"], "rollover must swap one ready pod for another")
 print("after rollover:", endpoints)

 endpoints, _ = select_endpoints(PODS, {"app": "web", "track": "stable"})
 check(endpoints == [], "an over-narrow selector should converge to empty, not error")
 print("over-narrow selector:", endpoints)

 endpoints, warning = select_endpoints(PODS, {})
 check(endpoints == [] and "empty selector" in warning, "empty selector must warn loudly")
 print("empty selector:", repr(warning))

 print("PASS: four convergences; readiness-gated selection, rollover, and empty warning hold")


if __name__ == "__main__":
 main()

The four scenarios verify this local selection model: the steady state holds the two ready web pods while the unready one stays out, and the rollover swaps one ready pod for another without changing the selector. The empty selector is the negative control: it warns loudly instead of silently matching everything or nothing. Both language fixtures passed with Python 3.12.14 on Linux in this review. The empty-selector warning is a local safeguard; Kubernetes can serve selectorless Services through explicitly managed EndpointSlices.

The simulation checks label-plus-readiness convergence only. It does not run kube-proxy, cluster DNS, or real probe timing. Before deploying, inspect the live endpoint set alongside pod labels and readiness, and repeat the lookup against the real Service name.

Sources

Kubernetes: Services, the Service abstraction, checked 2026-10-15.
Kubernetes: Pods, the pods a Service selects, checked 2026-10-15.
Kubernetes: Deployments, managing the pod set, checked 2026-10-15.
Primary reference, endpoint conditions and readiness exceptions, checked 2026-10-15.
Primary reference, ordinary and headless DNS answers, checked 2026-10-15.

Linger extends a systemd user manager beyond login

Tue, 20 Apr 2027 00:00:00 +0000

Linger changes when an account’s systemd user manager is kept available. With lingering enabled, logind can start that manager at boot and retain it after the account’s login sessions end. This supports unattended work owned by a normal account. The flag controls manager lifetime; individual services still have their own activation, failure, shutdown, and installation rules.

Linger controls manager lifetime. Unit activation is separate, and manager restart creates fresh processes from enabled units. PID A and PID B are schematic identities, not measured values. This artwork explains the source; the historical runtime check remains blocked.

When it breaks

Enabling linger does not enable every unit file found in the account’s configuration directory. A service must be requested through enablement, a dependency, a timer, a socket, or another explicit activation path. A manager can be running correctly while the intended application remains inactive. Check both objects instead of interpreting the linger flag as a list of running services.

Processes do not survive a machine reboot. Linger can arrange for a new user manager to start at boot, and enabled units can then start new processes. Their previous in-memory state is gone. An encrypted or late-mounted home can also make unit files or application data unavailable at the time the manager starts. Persist required state and make boot-time dependencies intentional.

Linger is not an instruction to restart a crashed application forever or to adopt arbitrary shell jobs into managed services. Restart policy, resource limits, and logind’s treatment of session processes are separate controls. A stable background role needs a real user unit and an observed result, not just a shell command launched before logout.

Check it quickly

Run this Python script as an administrator in a disposable Linux system with systemd, logind, user D-Bus support, and account-management tools. The recovered text reports Debian 12 with systemd 252.39, but its original run log is unavailable. This review did not execute the fixture; the outcomes below describe its intended assertions. It creates a temporary account and home, enables lingering only for that account, and removes its changes afterward. No login session is opened for the account. The final 20-second wait is a fixture timeout, not a promised stop delay. This no-login experiment does not establish last-logout UserStopDelaySec behavior:

from pathlib import Path
from tempfile import TemporaryDirectory
import os,pwd,subprocess,time,uuid

name='xbcatm-linger-'+uuid.uuid4().hex[:8];unit='field-worker.service'
def run(*args):
 result=subprocess.run(args,capture_output=True,text=True,timeout=20)
 assert result.returncode==0,(args,result.stderr)
 return result.stdout.strip()
def wait_for(check,seconds=5):
 deadline=time.monotonic()+seconds
 while not check():
 assert time.monotonic()<deadline,'state transition timed out';time.sleep(0.05)
with TemporaryDirectory(prefix=name+'-',dir='/run') as directory:
 root=Path(directory);root.chmod(0o755);home=root/'home';home.mkdir()
 run('useradd','--no-create-home','--user-group','--home-dir',str(home),'--shell','/bin/sh',name)
 account=pwd.getpwnam(name);uid=account.pw_uid;gid=account.pw_gid;manager=f'user@{uid}.service'
 def userctl(*args):
 return run('setpriv','--reuid='+str(uid),'--regid='+str(gid),'--init-groups',
 'env',f'XDG_RUNTIME_DIR=/run/user/{uid}',f'DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/{uid}/bus',
 'systemctl','--user',*args)
 def manager_value(key):return run('systemctl','show',manager,'-p',key,'--value')
 try:
 units=home/'.config/systemd/user';units.mkdir(parents=True)
 (home/'worker.py').write_text("from pathlib import Path\nimport os,time\np=Path(__file__).with_name('worker-pid');t=p.with_suffix('.tmp');t.write_text(str(os.getpid()));t.replace(p)\ntime.sleep(60)\n")
 (units/unit).write_text('[Service]\nType=exec\nExecStart=/usr/bin/python3 %h/worker.py\n[Install]\nWantedBy=default.target\n')
 for path in [home,*home.rglob('*')]:os.chown(path,uid,gid)
 run('loginctl','enable-linger',name)
 wait_for(lambda:Path(f'/run/user/{uid}/bus').exists())
 assert run('loginctl','show-user',name,'-p','Linger','--value')=='yes'
 assert run('loginctl','show-user',name,'-p','Sessions','--value')==''
 assert manager_value('ActiveState')=='active'
 assert userctl('show',unit,'-p','ActiveState','--value')=='inactive'
 print('linger starts a manager with zero login sessions; merely installing a user unit does not start it')
 userctl('enable','--now',unit);wait_for(lambda:(home/'worker-pid').exists())
 first=(home/'worker-pid').read_text();first_manager=manager_value('MainPID')
 run('systemctl','restart',manager)
 wait_for(lambda:Path(f'/run/user/{uid}/bus').exists() and (home/'worker-pid').read_text()!=first)
 assert manager_value('MainPID')!=first_manager
 assert userctl('show',unit,'-p','MainPID','--value')==(home/'worker-pid').read_text()
 assert run('loginctl','show-user',name,'-p','Sessions','--value')==''
 print('enabled user service starts as a new process after a real manager restart')
 userctl('disable',unit)
 assert userctl('show',unit,'-p','ActiveState','--value')=='active'
 print('disable removes future enablement but does not stop the current service')
 userctl('stop',unit);run('loginctl','disable-linger',name)
 wait_for(lambda:manager_value('ActiveState')=='inactive',seconds=20)
 print('with no sessions and linger disabled, user manager stops after policy delay')
 finally:
 run('loginctl','disable-linger',name)
 run('systemctl','stop',manager,f'user-runtime-dir@{uid}.service')
 run('userdel',name)

After enable-linger, the user manager becomes active and its bus appears. logind reports lingering enabled and an empty session list. However, the installed worker unit is still inactive. This confirms that keeping the manager available is different from requesting the application to run, even though the configuration file already exists.

The script then enables and starts the worker through the correct user manager. Its installation section links it to the user’s default target, while the explicit start begins current execution. A PID marker identifies the actual worker. The controller uses setpriv and the user’s runtime bus, so the control command itself does not quietly create a login session.

Restarting the system-level user@UID.service replaces the account manager. The enabled worker subsequently publishes a different PID, and the new user manager reports that same process as its main worker. The session list remains empty. This tests reconstruction from persistent unit enablement across a real manager restart, rather than merely observing the same process for a little longer.

Disabling the worker without a stop request leaves the current service active. Disablement removes future enablement links; it is not an implicit termination command. The script checks that distinction, then explicitly stops the worker and disables lingering. The fixture expects the manager to become inactive after linger is disabled; it does not measure a last-logout delay.

Cleanup disables lingering again if needed, stops the experiment’s manager and runtime-directory unit, and deletes the temporary account and home. The exercise deliberately uses temporary configuration and does not reboot the machine. Its reboot-related statements come from the documented lifecycle contract; the process-replacement assertion concerns a manager restart, not a machine reboot.

Operate an unattended account deliberately

Keep the account’s enabled units reviewable and remove services that are no longer needed. A service that remains available after logout can retain memory, open files, credentials, and network listeners. Ordinary user authority limits some actions but still permits everything granted to that identity. Use suitable permissions and supported resource controls for the actual workload.

Make configuration and credentials available through mechanisms that work without an interactive shell. A successful launch from a terminal does not prove unattended startup has the same environment. Verify working directories, input paths, and access after restarting the manager, then perform a real boot test when boot-time availability is a deployment requirement.

When retiring the role, distinguish disabling a service, stopping current execution, disabling linger, and deleting account data. Other sessions or activation paths may still matter. Inspect the account’s actual manager and service states after the intended change. The useful guarantee is a documented unattended lifecycle with explicit activation and shutdown, not the indefinite survival of one PID.

Sources

systemd 252 loginctl, enabling and disabling per-account lingering and inspecting sessions, checked 2026-09-12.
systemd 252 user@.service, account manager and runtime-directory lifecycles, checked 2026-09-12.
systemd 252 logind.conf, user stop delay and the distinction between sessions and user services, checked 2026-09-12.
systemd 252 systemctl, enable/disable versus start/stop behavior, checked 2026-09-12.
systemd-stable v252.39 user lifecycle source, user_may_gc checks stop delay only after a last-session timestamp exists; source evidence only, checked 2026-10-19.

Services · XB Field Notes

Kubernetes Services give pods a stable home

One address in front of many

Selection drives which pods respond

Traffic goes to whatever matches

The stable name is a promise to callers

When it breaks

Check it quickly

Further reading

Sources

Linger extends a systemd user manager beyond login

When it breaks

Check it quickly

Operate an unattended account deliberately

Further reading

Sources