A DaemonSet puts one Pod on every eligible node

Mon, 29 Mar 2027 00:00:00 +0000

A Deployment targets a replica count; a DaemonSet targets coverage of eligible nodes. That is useful for node-level log collectors, network agents, and device plugins. A newly eligible node creates reconciliation work, not an immediate guarantee that its agent is running or ready. Selection rules, scheduling, image pulls, startup, and node health can all delay useful coverage.

The controller creates and reconciles Pods from a shared template. Its desired state is normally one managed Pod per eligible node, rather than an operator-maintained replica count. Desired coverage and observed readiness are separate measurements. Keep an independently defined intended node population so a mistaken selector cannot make excluded machines disappear from the audit.

One per node, by design

The controller evaluates node selectors and required affinity, creates a Pod targeted to each eligible node, and normally lets the scheduler bind it. Nodes joining or leaving change that desired population. DaemonSet Pods automatically tolerate the unschedulable taint, so cordoning a node alone does not exclude them. Registration does not make scheduling, image retrieval, or initialization instantaneous.

Priority can help an important daemon preempt lower-priority workloads, but it does not create capacity or satisfy incompatible placement constraints. Choose priority and resource requests according to actual operational importance and workload impact. A Pod can remain Pending or fail after scheduling; neither a high PriorityClass nor a toleration proves that it is ready or immune to node-pressure eviction.

Coverage over count

RollingUpdate replaces Pods in a controlled sequence; OnDelete waits for old Pods to be deleted before replacing them from the changed template. With maxSurge zero, maxUnavailable one permits an unavailable node during rollout, not uninterrupted per-node coverage. A nonzero maxSurge can temporarily place old and new Pods on one node and needs extra resources. Version skew during updates is expected, and blocked rollouts can prolong it. kubectl rollout status ds/agent reports controller rollout progress, not continuity of delivered logs.

Selectors and affinity can divide heterogeneous fleets into deliberate subsets. A GPU device plugin and an edge log forwarder need not share one DaemonSet. Check the intended membership of each subset, and distinguish a node that is intentionally excluded from one whose expected agent is missing.

Rides the node lifecycle

The controller adds documented tolerations for conditions including not-ready, unreachable, several pressure taints, and unschedulable. Custom control-plane or hardware taints may still require explicit tolerations. A toleration permits placement past its matching taint; it does not select that node or overcome every other constraint. Label changes can alter membership, so observe the actual reconciliation rather than assuming an atomic handover between daemon fleets.

Ordinary kubectl drain refuses DaemonSet-managed Pods unless –ignore-daemonsets is supplied; that option skips them rather than deleting them. Cordoning and draining are therefore different from stopping these node services. Configure graceful termination for actual Pod deletion, shutdown, or other supported maintenance paths, and test the intended node-lifecycle procedure.

When it breaks

A new tainted pool can be silently excluded while desiredNumberScheduled still equals numberReady. Those fields describe the DaemonSet controller’s population, not every machine the operator intended to cover, and they are not per-pool counters. Compare node identities, selectors, taints, owned Pods, readiness, and update revision. A mismatch identifies a gap; events and Pod conditions are needed to distinguish missing tolerance, insufficient resources, image failure, or application startup failure.

A bad daemon update can remove observability on several nodes. Choose disruption and surge budgets, test a selector-scoped canary, and verify collection continuity and resource use before widening the rollout. Keep a usable previous revision and confirm rollback actually converges; a command returning successfully does not recover missing logs or guarantee that an old image remains available.

DaemonSet coverage audit: mechanism and evidence boundaries.

Check it quickly

Use Bash, Python 3, an authorized context/namespace/DaemonSet, and INTENDED_NODES: an independently maintained JSON map of intended node names to UIDs. Do not derive that roster from this DaemonSet’s selector. The script reads JSON rather than table columns:

(
set -euo pipefail
: "${CONTEXT:?Set the authorized kubectl context}"
: "${NAMESPACE:?Set the DaemonSet namespace}"
: "${DAEMONSET:?Set the DaemonSet name}"
: "${INTENDED_NODES:?Set the independent node-name to UID JSON file}"
umask 077
tmp=$(mktemp -d)
trap 'rm -rf -- "$tmp"' EXIT
k=(kubectl --context "$CONTEXT" --request-timeout=15s)
"${k[@]}" -n "$NAMESPACE" get ds "$DAEMONSET" -o json > "$tmp/before.json"
"${k[@]}" get nodes -o json > "$tmp/nodes.json"
"${k[@]}" -n "$NAMESPACE" get pods -o json > "$tmp/pods.json"
"${k[@]}" -n "$NAMESPACE" get ds "$DAEMONSET" -o json > "$tmp/after.json"
python3 - "$tmp" "$INTENDED_NODES" "$NAMESPACE" "$DAEMONSET" <<'PY'
import collections, json, pathlib, sys
root, roster, namespace, name = sys.argv[1:]
def require(ok, message):
 if not ok:
 raise SystemExit(message)
def unique(pairs):
 result = {}
 for key, value in pairs:
 require(key not in result, "Duplicate JSON key")
 result[key] = value
 return result
def load(path):
 return json.loads(pathlib.Path(path).read_text(), object_pairs_hook=unique)
def ready(obj):
 return any(c.get("type") == "Ready" and c.get("status") == "True"
 for c in obj.get("status", {}).get("conditions", []))
a, b = (load(pathlib.Path(root) / (x + ".json")) for x in ("before", "after"))
for ds in (a, b):
 require(ds["kind"] == "DaemonSet" and ds["apiVersion"] == "apps/v1",
 "Wrong resource type")
 require(ds["metadata"]["namespace"] == namespace and
 ds["metadata"]["name"] == name, "Wrong DaemonSet identity")
 require(ds["metadata"].get("uid") and not ds["metadata"].get("deletionTimestamp"),
 "Missing UID or deleting DaemonSet")
require(all(a["metadata"][k] == b["metadata"][k] for k in ("uid", "generation")),
 "DaemonSet changed during collection; retry")
uid = a["metadata"]["uid"]
expected = load(roster)
require(isinstance(expected, dict) and expected and
 all(isinstance(k, str) and k and isinstance(v, str) and v
 for k, v in expected.items()), "Use a nonempty node-name to UID map")
node_items = load(pathlib.Path(root) / "nodes.json")["items"]
nodes = {n["metadata"]["name"]: n for n in node_items}
require(len(nodes) == len(node_items), "Duplicate node names")
wrong_nodes = sorted(n for n in expected if n not in nodes or
 nodes[n]["metadata"]["uid"] != expected[n])
unready_nodes = sorted(n for n in expected if n in nodes and not ready(nodes[n]))
rows, seen = [], set()
for pod in load(pathlib.Path(root) / "pods.json")["items"]:
 m = pod["metadata"]
 require(m["namespace"] == namespace, "Wrong Pod namespace")
 if not any(o.get("controller") is True and o.get("uid") == uid and
 o.get("kind") == "DaemonSet" and o.get("apiVersion") == "apps/v1"
 and o.get("name") == name for o in m.get("ownerReferences", [])):
 continue
 require(m.get("uid") and m["uid"] not in seen, "Missing or duplicate Pod UID")
 seen.add(m["uid"])
 node = pod["spec"].get("nodeName", "")
 phase = pod.get("status", {}).get("phase", "Unknown")
 terminating = bool(m.get("deletionTimestamp"))
 rows.append(dict(pod=m["name"], uid=m["uid"], node=node, phase=phase,
 ready=ready(pod) and phase == "Running" and not terminating,
 terminating=terminating,
 revision=m.get("labels", {}).get("controller-revision-hash")))
counts = collections.Counter(r["node"] for r in rows if r["node"])
ready_nodes = {r["node"] for r in rows if r["ready"] and r["node"]}
issues = dict(node_identity_mismatch=wrong_nodes, node_not_ready=unready_nodes,
 missing_ready=sorted(set(expected) - ready_nodes),
 unexpected_owned_nodes=sorted(set(counts) - set(expected)),
 multiple_pods=sorted(n for n, count in counts.items() if count > 1),
 unassigned=[r["uid"] for r in rows if not r["node"]],
 unready_pods=[r["uid"] for r in rows if not r["ready"]])
print(json.dumps(dict(namespace=namespace, daemonset=name, uid=uid,
 intended=expected, pods=rows, issues=issues), indent=2))
sys.exit(1 if any(issues.values()) else 0)
PY
)

Zero means this collected snapshot has one nonterminating Running/Ready owned Pod per intended, Ready node with matching UID. Nonzero requires investigation: surge duplicates can be legitimate; unassigned Pods remain unresolved. Pod UIDs and revisions are reported, but target revision, placement eligibility and log delivery are not proved. Separate API reads are not atomic; recheck during change. Node UIDs do not prove current physical execution. Only syntax and synthetic file controls were exercised here. No cluster, rollout or drain ran; technical and runtime validation remain held.

Sources

Kubernetes node-pressure eviction, DaemonSet priority does not confer eviction immunity, checked 2026-10-18.
Kubernetes DaemonSet API, desired and ready status plus surge and unavailable budgets, checked 2026-10-18.
Kubernetes node drain, drain skips DaemonSet Pods when ignore-daemonsets is selected, checked 2026-10-18.
Kubernetes DaemonSet rolling updates, update strategies, disruption budget and surge configuration, checked 2026-10-18.
Kubernetes DaemonSet, documents daemonset behavior, checked 2026-06-01.
Kubernetes taints and tolerations, documents reaching tainted nodes, checked 2026-09-05.