Kernel · XB Field Notes

Linux overcommit budgets virtual-memory commitments

Sat, 18 Dec 2027 00:00:00 +0000

Linux overcommit policy controls admission of certain virtual-memory commitments. It is separate from allocating resident physical pages, choosing an OOM victim, and enforcing a container’s memory limit. A successful allocation can precede physical demand, but that does not make all allocations free or guarantee that later accesses will succeed under every resource constraint.

Virtual commitment differs from residence

An allocator such as malloc may obtain memory through brk or mmap and may reuse an existing arena without making a new system call. Anonymous pages are commonly populated lazily. Address-space metadata, page tables, and actual page faults still consume resources; the cost depends on what the application requests and touches.

Committed_AS is an accounting quantity, not the process resident-set sum and not a measurement of current pressure. Mapping type matters: private writable mappings can require commitment even before every page becomes resident, while ordinary shared or read-only file-backed mappings have different accounting. A large virtual-size figure cannot by itself identify the application responsible for a physical-memory shortage.

Three admission policies

Mode 0 is the default heuristic policy. It rejects obvious overcommits while allowing allocations that would exceed a simple resident-memory budget. Its decision is not a promise that every accepted byte has dedicated RAM. Mode 1 skips the overcommit admission check, which can support sparse workloads, but address-space limits, mapping limits, and other allocation failures still apply.

Mode 2 enforces stricter commitment accounting. The nominal CommitLimit is derived from swap plus a configured amount of RAM: ratio-based calculation excludes hugetlb-reserved memory, while a nonzero overcommit_kbytes supplies an absolute allowance instead of the ratio. Kernel reserves and the calling context can further affect admission. The displayed difference from Committed_AS is therefore useful context rather than a universal maximum guaranteed allocation.

A chargeable allocation may fail with ENOMEM when it cannot fit that budget. This can expose capacity problems earlier, including commitments needed by fork and later permission changes. Fork accounting follows applicable mappings and copy-on-write obligations; it is not simply twice the parent’s resident set. Some applications deliberately reserve sparse address space, so changing policy can change their startup behavior.

Strict accounting is not OOM immunity

Mode 2 reduces one class of overcommit risk, but does not override cgroup memory limits, NUMA constraints, other kernel allocations, or all later memory demands. A task can still encounter allocation failure or an OOM event. Do not promise that choosing mode 2 protects a particular worker from being killed.

OOM victim selection is a separate policy using the relevant scope and eligibility rules, not random punishment and not a decision made by the CPU scheduler. The killed process need not be the task that most recently requested memory. Inspect the event’s host or cgroup context before attributing responsibility from the victim’s name alone.

Containers share the host kernel and should not be assumed to own an independent vm.overcommit_memory policy. Their memory controls add another boundary. Virtual machines have their own guest kernel accounting, while host backing and ballooning introduce additional constraints. Record which layer each metric belongs to before comparing budgets.

When it breaks

A workload can successfully reserve memory and then encounter reclaim, swap activity, or OOM when demand grows. Neither mode 0 nor mode 1 guarantees that the failure will be slow, that swap is available, or that a particular process dies. Compare demand and latency with actual reclaim and memory-pressure evidence.

Conversely, mode 2 can reject a request despite apparently free RAM because the commitment budget is already occupied. That is one possible cause of ENOMEM, not a diagnosis from the error alone. Process resource limits, excessive mapping counts, and application configuration deserve separate checks.

Large nominal headroom does not prove the ratio is too conservative, and small headroom does not automatically justify increasing it. Budget changes can admit demand that later competes for real resources. Plan from workload requirements and recovery margin, preserve configuration context, and test changes outside production before rollout.

Overcommit controls commitment admission; accepted requests still face later physical and cgroup constraints.

Check it quickly

Save this as overcommit_probe.py and run python3 overcommit_probe.py. It reads only the named proc files, prints the observed mode and accounting fields, and performs synthetic parser checks. It does not reserve memory, trigger OOM, or suggest automatic tuning.

from pathlib import Path
import re


def read_value(path):
 try:
 return Path(path).read_text(encoding="utf-8").strip()
 except OSError as error:
 print("SKIP", path, type(error).__name__)
 return None


def meminfo_kb(text):
 return {name: int(value) for name, value in re.findall(
 r"^(MemTotal|SwapTotal|CommitLimit|Committed_AS):\s+(\d+)\s+kB\s*$",
 text, re.M)}


def report(mode, info):
 names = {"0": "heuristic", "1": "always overcommit", "2": "strict accounting"}
 lines = ["mode: " + names.get(mode, "unknown or unreadable")]
 limit, committed = info.get("CommitLimit"), info.get("Committed_AS")
 if limit is not None and committed is not None:
 lines.append("nominal commit headroom: %s kB" % (limit - committed))
 else:
 lines.append("commit headroom unavailable")
 if mode == "2":
 lines.append("Commit accounting can reject chargeable requests; other limits still apply")
 else:
 lines.append("CommitLimit is not the enforced mode-2 ceiling in this mode")
 lines.append("This snapshot is not available RAM, a cgroup budget, or OOM immunity")
 return lines


fixture = meminfo_kb("CommitLimit: 100 kB\nCommitted_AS: 0 kB\n")
assert "nominal commit headroom: 100 kB" in report("2", fixture)
assert meminfo_kb("CommitLimit: bad kB\nCommitted_AS: 7 MB\n") == {}
assert "commit headroom unavailable" in report("2", {})
assert report("9", {})[0] == "mode: unknown or unreadable"
for mode in ("0", "1", "2"):
 assert "OOM immunity" in report(mode, fixture)[-1]

mode = read_value("/proc/sys/vm/overcommit_memory")
ratio = read_value("/proc/sys/vm/overcommit_ratio")
kbytes = read_value("/proc/sys/vm/overcommit_kbytes")
text = read_value("/proc/meminfo")
if mode is None or text is None:
 print("SKIP live overcommit report; synthetic checks passed")
else:
 info = meminfo_kb(text)
 print("overcommit_memory=%s overcommit_ratio=%s overcommit_kbytes=%s" % (mode, ratio, kbytes))
 print("Observed kB fields:", info)
 for line in report(mode, info):
 print(line)

Expect a live snapshot or an explicit SKIP message. Zero committed memory remains a valid value. The revised example passed with Python 3.12.14 on Linux; synthetic checks cover malformed input, absent fields, and all three modes. Reading these counters does not test real allocation rejection or recovery.

Sources

Kernel vm sysctl documentation, defines the overcommit_memory modes and defaults, checked 2026-06-01.
proc manual page, documents /proc/meminfo CommitLimit and Committed_AS accounting, checked 2026-06-01.
Kernel overcommit accounting rules, checks policy modes and charged mappings, checked 2026-10-15.
proc_meminfo: CommitLimit and Committed_AS, checks commitment and huge-page counters with units, checked 2026-10-15.
Kernel cgroup v2 memory controls, checks local memory limits and OOM boundaries, checked 2026-10-15.

sysctl tunes the running kernel

Sun, 22 Nov 2026 00:00:00 +0000

On Linux, sysctl exposes selected kernel parameters through /proc/sys. It can inspect the running state, and an authorized write can change that state without rebooting. A configuration file is a separate instruction to a loader. Confusing these two layers produces a familiar incident: the desired value is on disk, but the kernel is doing something else.

This article covers Linux with procps-ng and systemd. BSD and macOS also have commands named sysctl, but their interfaces and persistence mechanisms are different. A key count collected on macOS does not establish anything about Linux’s procfs tree.

Read the interface you actually have

For ordinary names, kernel.ostype corresponds to /proc/sys/kernel/ostype; procps accepts slash separators too. Names containing literal dots need the documented separator rules, rather than a blind string replacement. Available keys depend on kernel configuration, loaded modules and the execution environment. The table is not a fixed schema across machines or even across one machine’s lifetime.

Normal procps output uses key = value; -n prints only the value. A successful read establishes an observation in the caller’s environment, not a durable policy. Some values change while you inspect them. Read only the keys needed for the diagnosis: even reads are not universally passive. The manual excludes vm.stat_refresh from ordinary enumeration because reading it flushes per-CPU statistics.

Persistence belongs to a particular loader

A runtime assignment does not create a persistent configuration file. Conversely, saving a drop-in does not apply it immediately. Identify which component applies settings at boot and on reload before reasoning about precedence.

For procps --system, same-named files in higher-priority directories mask lower-priority copies. The selected drop-ins are applied in filename order, regardless of directory; /etc/sysctl.conf is then read last. Systemd’s sysctl.d loader also combines directory priority with filename ordering, but does not directly read /etc/sysctl.conf. Distribution-provided links can still make that file participate through a drop-in. Inspect the installation rather than assuming the pathname alone settles the question.

This has two distinct override rules. A local file can replace a vendor file with the same basename, or a differently named file can override individual assignments by sorting later. Neither rule stops another service from writing the parameter after the loader finishes. Successful boot-time application and the value observed hours later are different facts.

Error policies are not interchangeable

Systemd’s sysctl.d documentation treats missing variables and permission failures as debug-level, nonfatal cases. Other assignment failures make its service fail after processing the remaining settings. A leading minus on an assignment suppresses failure for that assignment. These are systemd loader rules, not a promise about every invocation of procps sysctl.

Procps -e specifically ignores unknown-key errors; it does not mean every malformed value or denied write succeeds. Its configuration format also supports a leading minus to ignore an assignment’s error. Do not infer a portable exit status by combining options and error policies from different implementations.

When it breaks

A module-specific key may not exist when the boot loader runs. Check module availability and loader logs before concluding that a later read disproves the configuration syntax. Another writer may also overwrite a successfully applied value. Record the loader, file precedence, application time and subsequent effective value separately.

Containers add another boundary: mounts, permissions and namespaces affect visibility and write scope. Determine scope from the particular parameter’s documentation; a prefix such as net.* is not sufficient proof that a proposed change is isolated. Do not grant privileges or remount procfs merely to make a troubleshooting command work.

Loader selection and ordering precede a runtime value, which a later writer can change; the source check only reads.

Check it quickly

This proposed read-only exercise requires Linux, Python 3, procps-ng 4.0.4 and readable procfs; its exact live invocation remains unrun. It compares one ordinary key through both interfaces, then checks a nonexistent key. It does not enumerate unknown handlers, load configuration, write settings or test boot persistence.

import os
import subprocess
from pathlib import Path

def require_result(result, status, stdout, stderr):
 if (type(result.returncode) is not int or type(result.stdout) is not bytes
 or type(result.stderr) is not bytes
 or (result.returncode, result.stdout, result.stderr) != (status, stdout, stderr)):
 raise RuntimeError(f"unexpected sysctl result: {result!r}")

def read_sysctl(*args):
 return subprocess.run(
 ["sysctl", *args], capture_output=True, check=False, timeout=5,
 env={**os.environ, "LC_ALL": "C"})

require_result(read_sysctl("--version"), 0,
 b"sysctl from procps-ng 4.0.4\n", b"")
require_result(read_sysctl("kernel.ostype"), 0, b"kernel.ostype = Linux\n", b"")
require_result(read_sysctl("-n", "kernel.ostype"), 0, b"Linux\n", b"")
if Path("/proc/sys/kernel/ostype").read_bytes() != b"Linux\n":
 raise RuntimeError("unexpected procfs kernel type")
require_result(read_sysctl("-n", "kernel.xbcatm_audit_missing_key"), 1, b"",
 b"sysctl: cannot stat /proc/sys/kernel/xbcatm_audit_missing_key: "
 b"No such file or directory\n")
print("matching kernel type: Linux")
print("exact missing-key diagnostic accepted")

Expect the matching-value and exact-diagnostic messages only after all checks succeed. The C-locale negative control requires status 1, empty stdout and the complete missing-path diagnostic, including its newline. Permission errors, missing tools, timeouts, partial output and unexpected success fail instead. Another procps version or diagnostic requires separate review, not a looser substring match. Pure response tests do not validate procfs or a live command. sysctl --system and sysctl -p apply settings; they are not configuration-preview commands and do not belong in this read-only check.

Sources

procps sysctl(8), runtime interface, output options, unknown-key handling and system-file precedence, checked 2026-10-15.
procps sysctl.conf(5), assignment syntax and loader distinction, checked 2026-10-15.
systemd 252 sysctl.d documentation source, separators, missing modules and error policy, checked 2026-10-15.
systemd 252 configuration precedence source, basename masking and lexicographic ordering, checked 2026-10-15.
procps-ng 4.0.4 sysctl source, exact missing-path read error and failure status, checked 2026-10-19.

Kernel · XB Field Notes

Linux overcommit budgets virtual-memory commitments

Virtual commitment differs from residence

Three admission policies

Strict accounting is not OOM immunity

When it breaks

Check it quickly

Further reading

Sources

sysctl tunes the running kernel

Read the interface you actually have

Persistence belongs to a particular loader

Error policies are not interchangeable

When it breaks

Check it quickly

Sources