Concurrency · XB Field Notes

Advisory lock lifetime belongs to its connection or transaction

Wed, 24 Nov 2027 00:00:00 +0000

An advisory lock lets an application coordinate work using an identifier that need not correspond to a table row. For example, workers can agree that only one connection may refresh a particular report at a time. PostgreSQL manages ownership and contention, while the application defines what the identifier means and which work requires it.

When it breaks

The key is either one bigint or two integer values; there is no advisory-lock overload accepting a text name. Those two key spaces are distinct. If an application hashes names, it must define stable encoding and handle possible collisions; different computations can also give the same resource different keys. A documented integer namespace is often easier to reason about than an undocumented hash convention.

The lock is cooperative. A client that ignores it can still update the supposed protected rows. It does not replace a uniqueness constraint, transaction, or authorization check. Session locks survive rollback and require matching unlocks, or session termination. Transaction locks end with their transaction. In autocommit, a transaction lock obtained by one statement has already been released before a later statement starts, leaving that later work unprotected.

Session acquisition counts survive rollback; transaction ownership ends with its transaction. Values are manual expectations, not observed current contention.

Check it quickly

The exact two-session example was not rerun here. Retained reviews describe source-checked expectations without raw multi-session output; a standalone backend cannot establish inter-session contention.

Use two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, connected to one disposable PostgreSQL 18 database. Use the same role with schema creation rights, an unused advisory_demo schema, autocommit outside explicit transactions, and no timeout that interrupts the schedule. Reserve the example key pairs for this exercise and execute blocks in order. A creates a counter:

CREATE SCHEMA advisory_demo;
CREATE TABLE advisory_demo.counter (id integer PRIMARY KEY,n integer);
INSERT INTO advisory_demo.counter VALUES (1,0);

A tests an explicitly typed text key. The expected SQLSTATE is 42883, indicating no matching function:

\set ON_ERROR_STOP off
SELECT pg_advisory_lock('job-name'::text);
\echo text_key_state :SQLSTATE
\set ON_ERROR_STOP on

In A, acquire the same session lock twice inside a transaction, then roll back:

BEGIN;
SELECT pg_advisory_lock(8610,2);
SELECT pg_advisory_lock(8610,2);
ROLLBACK;

B cannot acquire that lock, but can update the counter without participating in the advisory protocol:

SELECT pg_try_advisory_lock(8610,2);
UPDATE advisory_demo.counter SET n=n+1 WHERE id=1 RETURNING n;

Expect false, followed by one. Both observations matter: rollback did not release A’s session lock, and the advisory lock did not prevent B’s ordinary write. In A, release one acquisition:

SELECT pg_advisory_unlock(8610,2);

The unlock returns true, but B still cannot acquire the lock:

SELECT pg_try_advisory_lock(8610,2);

A must release its second acquisition too:

SELECT pg_advisory_unlock(8610,2);

B can now obtain the session lock, and explicitly releases its own acquisition:

SELECT pg_try_advisory_lock(8610,2);
SELECT pg_advisory_unlock(8610,2);

Both calls return true. Successful repeated acquisitions are counted, so a single successful unlock does not prove that a session no longer owns the resource. In A, begin a transaction and acquire a different key using the transaction-scoped function:

BEGIN;
SELECT pg_advisory_xact_lock(8610,3);

B’s nonblocking attempt returns false while A’s transaction remains open:

SELECT pg_try_advisory_xact_lock(8610,3);

A rolls back:

ROLLBACK;

B can now obtain the transaction lock in autocommit:

SELECT pg_try_advisory_xact_lock(8610,3);

Immediately afterward, A can obtain it too:

SELECT pg_try_advisory_xact_lock(8610,3);

These two true results are sequential, not simultaneous ownership. Each statement’s implicit transaction ended and released the lock before the next statement began. Neither connection now protects a future critical section. Finally, A checks that the counter is one and removes the fixture:

SELECT n FROM advisory_demo.counter WHERE id=1;
DROP SCHEMA advisory_demo CASCADE;

Compare every printed result and SQLSTATE manually; the blocks contain no automatic assertions. After interruption, roll back fixture transactions and release successful session acquisitions, or close the dedicated test sessions, before removing the fixture. An unsuccessful try-lock call adds no ownership to release. Only a successful acquisition contributes to the locks held by that session or transaction.

Match ownership to the operation

For database work that fits one transaction, acquire the transaction lock inside an explicit transaction, then read, decide, and write before ending it. A nonblocking acquisition returning false means the client did not obtain ownership; it must skip or reschedule the protected work, not continue as though it succeeded. Blocking variants can wait, so their timeout and retry policies belong to the operation budget.

Use a session lock only when its longer lifetime is required, and keep acquisition, protected work, and release on the same physical connection. Returning that connection to a pool can leave a lock owned by a backend that serves unrelated callers; transaction pooling is especially unsuitable for session ownership assumptions. A terminated backend releases session locks automatically, although detecting a lost network connection can take time. This mechanism also supplies no durable record that a job completed: a subsequent worker must inspect committed state before deciding whether repeating the work is appropriate.

Sources

PostgreSQL 18 advisory lock functions, accepted key types and blocking or nonblocking functions, checked 2026-10-15.
PostgreSQL 18 explicit locking, advisory ownership, repeated acquisition, and transaction semantics, checked 2026-10-15.

Transactions make atomic commits

Wed, 12 May 2027 00:00:00 +0000

A transaction lets related database changes commit together or be discarded together. Debiting an account, crediting another, and recording a ledger entry belong to the same decision when the operation requires all three. PostgreSQL provides that boundary through BEGIN, COMMIT, and ROLLBACK. The application must still place the correct statements inside it: atomic execution does not discover a forgotten credit or invent a missing business check.

The failed second transfer must return to the committed balances and ledger count. A missing COMMIT reply is a separate uncertainty requiring outcome reconciliation. This conceptual diagram adds no runtime evidence.

Decide what belongs to one unit

Without an explicit transaction block, ordinary psql autocommit runs each statement in its own transaction. A successful debit can therefore commit before a later ledger insert fails. Explicitly grouping the writes changes the failure boundary. Other transactions do not see the group’s uncommitted intermediate table changes, and rollback discards its transactional changes rather than relying on compensating updates issued afterward.

Atomicity is distinct from isolation and durability. A committed unit can still embody an incorrect concurrent decision if its isolation or locking protocol is insufficient. The persistence guarantees of an acknowledged commit also depend on the database’s durability configuration. None of those questions is answered solely by finding a BEGIN in the code.

Every PostgreSQL statement already executes in a transaction, including reads. An explicit read transaction can have a purpose, such as selecting a repeatable snapshot or using transaction-local settings. The relevant decision is the required boundary, not a rule that all read transactions are useless overhead.

When it breaks

In PostgreSQL, an unhandled SQL error normally leaves an explicit transaction block aborted. Further ordinary statements fail until the client rolls back, or rolls back to an applicable earlier savepoint. Disabling a client’s stop-on-error behavior does not restore the server transaction. Returning that connection to a pool without cleaning up can make the next request inherit the failure state.

A server-reported rejection and a lost commit acknowledgement are different evidence. If a connection fails while the client awaits COMMIT, the client may not know whether the database committed before communication was lost. Blindly repeating a payment can duplicate it. A durable operation identifier and outcome reconciliation address that ambiguity; ordinary rollback semantics do not prove what happened on an unreachable server.

Check it quickly

Run the following in a fresh psql -X -v ON_ERROR_STOP=1 session on PostgreSQL 18 using a disposable database. Temporary tables keep the account identifiers local to the session. Error stopping is disabled only for two deliberate failures so their SQLSTATE values can be observed, then enabled again.

\set ON_ERROR_STOP on
CREATE TEMP TABLE accounts (id integer PRIMARY KEY,balance integer CHECK(balance>=0));
CREATE TEMP TABLE ledger (id integer PRIMARY KEY,amount integer CHECK(amount>0));
INSERT INTO accounts VALUES (1,100),(2,100);
BEGIN;
UPDATE accounts SET balance=balance-20 WHERE id=1;
UPDATE accounts SET balance=balance+20 WHERE id=2;
INSERT INTO ledger VALUES (1,20);
COMMIT;
SELECT id,balance FROM accounts ORDER BY id;
BEGIN;
UPDATE accounts SET balance=balance-30 WHERE id=1;
\set ON_ERROR_STOP off
INSERT INTO ledger VALUES (1,30);
\echo duplicate_error :SQLSTATE
SELECT balance FROM accounts WHERE id=1;
\echo aborted_error :SQLSTATE
\set ON_ERROR_STOP on
ROLLBACK;
SELECT id,balance FROM accounts ORDER BY id;
SELECT count(*) AS ledger_rows FROM ledger;
DO $$ BEGIN
 IF (SELECT array_agg(balance ORDER BY id) FROM accounts) IS DISTINCT FROM ARRAY[80,120]
 OR (SELECT count(*) FROM ledger) <> 1
 OR (SELECT amount FROM ledger WHERE id=1) IS DISTINCT FROM 20 THEN
 RAISE EXCEPTION 'partial transfer survived';
 END IF;
END $$;
DROP TABLE ledger,accounts;

The first transaction commits balances of 80 and 120 and one ledger row. The second transaction debits another thirty, then attempts to reuse ledger identifier one. The retained PostgreSQL 18.6 report records 23505 for that uniqueness violation; this review did not rerun the client workflow. The following ordinary read reports 25P02, showing that the transaction is aborted even though the client continues sending commands.

After rollback, the balances are still 80 and 120, with one ledger row. The second debit did not survive. The assertion checks the relevant balances and ledger state; the two printed SQLSTATE values still require inspection. Seeing an error alone does not prove rollback worked. The final drop removes the temporary fixture explicitly; closing the session would also remove its temporary tables.

This is a controlled atomicity experiment, not a complete money-transfer service. A real operation must verify that intended accounts exist, affected-row counts match expectations, amounts obey the business contract, and concurrent decisions are protected. Otherwise a transaction can atomically commit the wrong set of changes.

Keep recovery inside the contract

Savepoints allow selected work to be undone while preserving earlier transactional changes. They are useful when that partial outcome is intentional; swallowing a required ledger failure and committing the debit would violate this operation’s contract. Exceptions caught inside database procedural blocks can also establish a smaller rollback scope, so examine the actual handling path.

Sequence advancement and external actions such as sending mail are not undone by rolling back table writes. Keep irreversible external effects out of an attempt that may be retried, or coordinate them through a durable protocol. Short, explicit transactions make the commit decision easier to reason about, while checked results and a clear recovery path establish that the decision still means what the application intends.

Sources

PostgreSQL 18: Transactions, atomic groups and rollback boundaries, checked 2026-09-10.
PostgreSQL 18: Transaction isolation, visibility and nontransactional sequence behavior, checked 2026-09-10.
PostgreSQL 18: SAVEPOINT, intentional partial rollback within a transaction, checked 2026-09-10.

A transaction can fail and leave you to retry

Mon, 01 Feb 2027 00:00:00 +0000

A competing update does not automatically make a PostgreSQL transaction fail. The outcome depends on isolation, statement behavior, and the state another transaction commits. Treating every collision as the same retryable event can conceal invalid input or repeat a decision made from stale data. The useful boundary is a complete application operation whose reads and writes can be attempted again.

When it breaks

Under Read Committed, an ordinary query gets a snapshot for that statement. An update can wait for another updater and then recheck its condition against the updated row. Two increments can therefore both succeed. An application that reads a number separately and later writes a previously calculated absolute value has a different contract: Read Committed does not automatically recompute that client calculation.

Repeatable Read retains a transaction snapshot, established by its first nontransaction-control statement. If a row changed after that snapshot, attempting to update it can raise 40001 immediately. Serializable can also reject dependency patterns involving different rows. Neither level promises that failures occur only at commit, nor does stronger isolation imply a universally earlier failure. Check statement execution and commit, using the driver’s structured error information.

Record the first SQLSTATE alongside the isolation level and operation identifier. A later 25P02 is a consequence of an earlier error, so logging only the final failed query can hide the actual reason the transaction stopped.

A savepoint can recover from some statement errors without discarding earlier work, but it is not a substitute for retrying a serialization failure from a fresh transaction snapshot. Reusing the same earlier reads can reproduce the invalid decision. Keep external side effects outside an automatic retry loop unless their own deduplication contract makes replay safe.

The chosen schedule compares snapshot boundaries with ordered writes. It does not measure waiting, and a savepoint is not a fresh transaction snapshot.

Check it quickly

Use PostgreSQL 18 and two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, connected to the same disposable database. Use the same role with schema creation rights and an unused conflict_demo schema, with no timeout interrupting the schedule. Run blocks in order; both sessions use autocommit outside explicit transactions. In A, create the isolated fixture:

CREATE SCHEMA conflict_demo;
CREATE TABLE conflict_demo.counter (id integer PRIMARY KEY,n integer);
INSERT INTO conflict_demo.counter VALUES (1,0);

A begins at Read Committed and reads zero:

BEGIN ISOLATION LEVEL READ COMMITTED;
SELECT n FROM conflict_demo.counter WHERE id=1;

B commits an increment and returns one:

UPDATE conflict_demo.counter SET n=n+1 WHERE id=1 RETURNING n;

A now increments the current row and returns two, then commits successfully:

UPDATE conflict_demo.counter SET n=n+1 WHERE id=1 RETURNING n;
COMMIT;

This is a successful concurrent history. A’s earlier read did not freeze the value used by its later arithmetic update. In B, reset the counter after A commits:

UPDATE conflict_demo.counter SET n=0 WHERE id=1;

In A, begin Repeatable Read and establish the snapshot by reading zero:

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT n FROM conflict_demo.counter WHERE id=1;

B again commits an increment and returns one:

UPDATE conflict_demo.counter SET n=n+1 WHERE id=1 RETURNING n;

A attempts the same update. Temporarily allow the expected errors so the script can show both states and perform cleanup:

\set ON_ERROR_STOP off
UPDATE conflict_demo.counter SET n=n+1 WHERE id=1;
\echo update_state :SQLSTATE
SELECT n FROM conflict_demo.counter WHERE id=1;
\echo transaction_state :SQLSTATE
\set ON_ERROR_STOP on
ROLLBACK;

The unchanged two-session schedule was source-reviewed, not rerun here. Its expected update reports 40001; the subsequent read reports 25P02, meaning the transaction is already failed. In this documented schedule the failure occurs at UPDATE, before any commit attempt. Restoring the client’s stop-on-error setting does not repair a failed server transaction; rollback does. In A, retry the complete operation after B has finished:

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT n FROM conflict_demo.counter WHERE id=1;
UPDATE conflict_demo.counter SET n=n+1 WHERE id=1 RETURNING n;
COMMIT;

The fresh read returns one and the update returns two. Finally, B verifies the committed result and removes the fixture:

SELECT n FROM conflict_demo.counter WHERE id=1;
DROP SCHEMA conflict_demo CASCADE;

Manually check every returned value and SQLSTATE; the blocks print results without assertions. If interrupted, roll back both fixture transactions before removing this schema. The final two includes one committed increment from each participant. The failed attempt contributed nothing. If a result differs, check that both terminals selected the same database, used the intended isolation, and followed the reset and commit boundaries before interpreting it as a server anomaly.

Define the retry boundary

A 40001 response calls for a new transaction and new decisions, including reads used to choose keys or amounts. A 40P01 deadlock may also justify retrying the complete operation. Bound attempts and elapsed time; replay can be expensive and may still fail under sustained contention. Roll back before returning a failed connection to a pool, and preserve the final error when the budget expires.

Do not classify every 23505 uniqueness violation as transient. A duplicate permanent identifier remains invalid after waiting. Some application-generated keys depend on earlier reads, so a particular uniqueness failure can justify recomputation, but that decision needs operation-specific evidence. Likewise, losing the network acknowledgement of commit leaves an unknown outcome: it is different from receiving a server-declared serialization failure. Resolve that ambiguity with an operation identifier and durable result before replaying a payment or other externally visible effect.

Sources

PostgreSQL 18 transaction isolation, statement snapshots and concurrent update behavior, checked 2026-10-15.
PostgreSQL 18 serialization failure handling, complete transaction retries and error classification, checked 2026-10-15.

File locks protect a shared protocol and a specific object

Sat, 05 Dec 2026 00:00:00 +0000

A file lock coordinates participants that agree on what to lock, when to acquire it, and which operations belong inside the critical section. Linux offers several lock interfaces with different ownership and release rules. Choosing a filename and calling something named lock is insufficient: the protocol must identify the underlying object, the lock family, and the lifetime of every reference involved.

Local Linux semantics; participants must share the lock family and stable object. Ordinary writes can bypass advisory locks. Recreating the lock file can split ownership. Exclusion does not provide durability.

When it breaks

On the local Linux filesystem used here, flock provides advisory exclusion. Another process that requests an incompatible lock waits or fails, while an ordinary write that never requests the lock can proceed. The lock therefore does not grant content privacy or automatically constrain every program capable of opening the file.

Locking a data file becomes subtle when updates replace its pathname with another inode. The existing lock remains associated with the old open file description. A later opener reaches the replacement and can acquire a different lock. Both writers may believe they have exclusive access to the same logical document while actually locking different objects.

Traditional POSIX record locks have another surprising lifetime rule: they belong to the process, and closing any descriptor in that process referring to the file releases its record locks on that file. A library that independently opens and closes the same inode can therefore invalidate a lock held elsewhere in the process. This differs from flock ownership through an open file description. An explicit unlock through any duplicate also releases that flock; keeping another duplicate open does not override an explicit unlock.

Check it quickly

Save this as locking.py and run python3 locking.py in a disposable Linux environment. The recorded run used Linux 6.12.76, Python 3.11.2, and a local tmpfs. Each competing lock request runs in a separate process and uses nonblocking acquisition, so an expected conflict produces a result instead of hanging:

from pathlib import Path
from tempfile import TemporaryDirectory
import fcntl,os,subprocess,sys

probe_code=r'''
import fcntl,sys
with open(sys.argv[1],'r+') as handle:
 try:
 if sys.argv[2]=='flock':fcntl.flock(handle,fcntl.LOCK_EX|fcntl.LOCK_NB)
 else:fcntl.lockf(handle,fcntl.LOCK_EX|fcntl.LOCK_NB,1,int(sys.argv[3]),0)
 except BlockingIOError:print('blocked')
 else:print('acquired')
'''
def probe(path,kind='flock',start=0):
 result=subprocess.run([sys.executable,'-c',probe_code,str(path),kind,str(start)],capture_output=True,text=True,timeout=5)
 assert result.returncode==0,result.stderr
 return result.stdout.strip()
with TemporaryDirectory(prefix='xbcatm-locks-') as directory:
 root=Path(directory);target=root/'data';target.write_text('original')
 handle=target.open('r+');fcntl.flock(handle,fcntl.LOCK_EX)
 assert probe(target)=='blocked'
 subprocess.run([sys.executable,'-c',"from pathlib import Path;import sys;Path(sys.argv[1]).write_text('uncooperative')",str(target)],capture_output=True,text=True,check=True)
 handle.seek(0);assert handle.read()=='uncooperative'
 duplicate=os.dup(handle.fileno());handle.close()
 try:assert probe(target)=='blocked'
 finally:os.close(duplicate)
 assert probe(target)=='acquired'
 print('flock blocks cooperating contenders but not ordinary writes; dup retains the same lock')
 with target.open('r+') as handle:
 fcntl.flock(handle,fcntl.LOCK_EX)
 replacement=root/'replacement';replacement.write_text('new inode');replacement.replace(target)
 assert os.fstat(handle.fileno()).st_ino!=target.stat().st_ino
 assert probe(target)=='acquired'
 lockfile=root/'stable.lock';lockfile.touch()
 with lockfile.open('r+') as handle:
 fcntl.flock(handle,fcntl.LOCK_EX)
 replacement.write_text('next inode');replacement.replace(target)
 assert probe(lockfile)=='blocked'
 assert probe(lockfile)=='acquired'
 print('replacing locked data changes the inode; a separate stable lock file protects that replacement protocol')
 first=os.open(target,os.O_RDWR);second=os.open(target,os.O_RDWR)
 try:
 fcntl.lockf(first,fcntl.LOCK_EX|fcntl.LOCK_NB,1,0,0)
 assert probe(target,'record',0)=='blocked' and probe(target,'record',1)=='acquired'
 os.close(second);second=None
 assert probe(target,'record',0)=='acquired'
 print('POSIX record locks permit disjoint bytes; closing another descriptor to the inode releases process-owned locks')
 finally:
 if second is not None:os.close(second)
 os.close(first)

First, the parent holds an exclusive flock, and a cooperating child reports blocked. Another child writes without requesting any lock, and the parent reads the changed content. These two outcomes establish both the useful exclusion and its advisory boundary using the same file and permissions.

The parent then duplicates its descriptor and closes the original wrapper. A contender remains blocked because the duplicate still references the locked open file description. Closing that final duplicate releases the lock, and the next contender succeeds. Counting wrapper objects or relying on the first close would describe the lifetime incorrectly.

Next, the parent locks the data file and atomically replaces its pathname with a newly created file. The descriptor’s inode differs from the inode now found by pathname. A child opens that pathname and acquires its own lock. Atomic replacement keeps the name available, but it does not move the old lock onto the replacement inode.

The following case uses a separate stable.lock file. Writers agree to lock that stable object before replacing the data pathname. The contender remains blocked across replacement and succeeds after release. This works only while participants keep the lock file’s identity stable; deleting and recreating that file can reproduce the original split-object failure.

Finally, lockf exercises traditional POSIX record locking. Locking byte zero blocks a child requesting that byte but allows a child requesting byte one. Closing the parent’s second, independently opened descriptor releases the byte-zero lock, even though the descriptor used to acquire it remains open. The successful final probe makes this release rule observable.

Make the protocol explicit

Use the same lock family and object across all participants. On local Linux filesystems, traditional record locks and flock are normally separate mechanisms, so mixing them does not establish mutual exclusion. NFS and SMB can have different emulation and interaction rules; verify the deployed filesystem and mount behavior before applying local results there.

Choose a range that covers the entire invariant being updated. Separate record ranges only permit useful concurrency when operations really are independent; a shared header or index can require broader coordination. Keep unrelated slow work outside the critical section, and use a consistent acquisition order when an operation needs multiple locks.

Define cleanup for normal return, exceptions, and descriptor transfer. Process termination releases kernel-held locks according to their ownership rules, but duplicated or inherited references can extend some lifetimes. Locks also do not make writes durable or repair partially completed updates. Pair exclusion with an update and recovery protocol appropriate to the file’s data format.

Sources

flock(2), advisory locks, open file descriptions, and remote-filesystem caveats, checked 2026-09-12.
fcntl_locking(2), POSIX record ranges and process-owned lock release rules, checked 2026-09-12.
rename(2), pathname replacement and the persistence of open references, checked 2026-09-12.

A bounded retry gives a transaction another chance

Mon, 09 Nov 2026 00:00:00 +0000

A retry can recover a transaction rejected because of concurrent work. It cannot guarantee success, and it does not automatically make external effects idempotent. The retry unit should include every database read and application decision that determined the writes. Reusing an amount calculated from the first attempt’s snapshot can defeat the purpose of starting a fresh transaction.

Retry all reads and decisions after a confirmed concurrency rejection, within one budget. Stop on permanent errors or exhaustion. Reconcile unknown commit outcomes separately; rollback cannot undo external side effects.

When it breaks

PostgreSQL identifies serialization failures with 40001; deadlocks use 40P01. Either can justify another complete attempt after rollback, within a finite budget. A duplicate identifier producing 23505 may be permanent. Retrying all exceptions indiscriminately wastes capacity and hides bad input. Some uniqueness failures depend on earlier key-selection reads, but their policy requires knowledge of that particular operation.

Errors can appear during statements or commit. A server-declared rejection differs from losing the connection while awaiting commit confirmation: the latter may leave a committed operation whose acknowledgement never reached the caller. Replaying it blindly can duplicate work. A payment or message sent outside the database transaction also survives database rollback; a retry loop alone cannot undo or deduplicate it.

Check it quickly

Use Python 3 on Linux or macOS and PostgreSQL 18’s psql in your executable path. Set PGDATABASE, PGHOST, and any required connection settings to a disposable database where you can create a schema. Arrange authentication beforehand; the example disables interactive password prompts. Save the following program as retry-demo.py and run python3 retry-demo.py. Use one exclusive disposable database: an existing fixture schema makes setup fail. Setup precedes the cleanup block, so a later setup failure can leave newly created objects; inspect that failed run before retrying or cleaning up.

import os
import random
import re
import selectors
import subprocess
import time

PSQL = ['psql', '-X', '-w', '-qAt', '-v', 'ON_ERROR_STOP=1', '-v', 'VERBOSITY=sqlstate']
ENV = dict(os.environ, PGCONNECT_TIMEOUT='2')


def run(sql):
 return subprocess.run(PSQL, input=sql, text=True, capture_output=True,
 check=True, timeout=5, env=ENV).stdout.strip()


def read_value(worker):
 deadline = time.monotonic() + 5
 data = bytearray()
 with selectors.DefaultSelector() as ready:
 ready.register(worker.stdout, selectors.EVENT_READ)
 while b'\n' not in data:
 remaining = deadline - time.monotonic()
 if remaining <= 0 or not ready.select(remaining):
 raise TimeoutError('Initial read was not confirmed')
 chunk = os.read(worker.stdout.fileno(), 4096)
 if not chunk:
 raise RuntimeError('Connection ended before initial read')
 data.extend(chunk)
 return int(data.strip())


def attempt(mode, number):
 worker = subprocess.Popen(PSQL, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
 stderr=subprocess.PIPE, text=True, env=ENV)
 try:
 worker.stdin.write("SET statement_timeout='1s';\n"
 'BEGIN ISOLATION LEVEL REPEATABLE READ;\n'
 'SELECT n FROM retry_demo.counter WHERE id=1;\n')
 worker.stdin.flush()
 observed = read_value(worker)
 # A real second connection commits after this attempt's snapshot.
 if mode == 'always' or (mode == 'once' and number == 1):
 run('UPDATE retry_demo.counter SET n=n+1 WHERE id=1;')
 statement = ('INSERT INTO retry_demo.counter VALUES (1,99);' if mode == 'permanent'
 else f'UPDATE retry_demo.counter SET n={observed + 1} WHERE id=1;')
 finish = r'''
\set ON_ERROR_STOP off
__STATEMENT__
\if :ERROR
 \echo ATTEMPT :SQLSTATE
 ROLLBACK;
\else
 COMMIT;
 \if :ERROR
 \echo ATTEMPT :SQLSTATE
 \else
 \echo ATTEMPT 00000
 \endif
\endif
'''.replace('__STATEMENT__', statement)
 output, _ = worker.communicate(finish, timeout=5)
 match = re.fullmatch(r'ATTEMPT ([0-9A-Z]{5})', output.strip())
 if worker.returncode != 0 or match is None:
 raise RuntimeError('No confirmed transaction outcome; do not replay blindly')
 return match[1], observed
 finally:
 if worker.poll() is None:
 worker.kill()
 worker.communicate()


def increment(mode):
 observations = []
 for number in range(1, 4):
 state, observed = attempt(mode, number)
 observations.append(observed)
 if state not in {'40001', '40P01'} or number == 3:
 return {'attempts': number, 'state': state, 'reads': observations}
 time.sleep(random.uniform(0, 0.01 * 2 ** (number - 1)))


run('CREATE SCHEMA retry_demo; '
 'CREATE TABLE retry_demo.counter (id integer PRIMARY KEY,n integer); '
 'INSERT INTO retry_demo.counter VALUES (1,0);')
try:
 cases = [
 ('once', {'attempts': 2, 'state': '00000', 'reads': [0, 1]}, 2),
 ('permanent', {'attempts': 1, 'state': '23505', 'reads': [0]}, 0),
 ('always', {'attempts': 3, 'state': '40001', 'reads': [0, 1, 2]}, 3),
 ]
 for mode, expected, expected_value in cases:
 run('UPDATE retry_demo.counter SET n=0 WHERE id=1;')
 result = increment(mode)
 value = int(run('SELECT n FROM retry_demo.counter WHERE id=1;'))
 assert result == expected and value == expected_value, (mode, result, value)
 print(mode, result, 'stored', value)
finally:
 run('DROP SCHEMA retry_demo CASCADE;')

The program uses two actual database connections. Each attempt starts Repeatable Read, reads the counter, and calculates a new absolute value. A competing connection can commit between that read and update, forcing the original snapshot’s update to fail. This deliberately exposes whether the retry reads again rather than merely resending the original calculated value.

For the once case, expect two attempts, final state 00000, observations zero then one, and stored value two. One increment belongs to the competing connection and one to the successful retry. The rejected first attempt contributes nothing. If the second attempt reused the old calculated value, it would not preserve that result.

For permanent, expect one attempt, state 23505, observation zero, and stored value zero. The existing primary key makes the requested insert invalid. The classifier stops immediately instead of spending the whole retry allowance on unchanged input. No successful application increment occurred.

For always, expect three attempts, state 40001, observations zero, one, and two, and stored value three. All three stored increments belong to the competing connection; the application never committed its operation. This is deliberate exhaustion, not eventual success. The assertions check both the control-flow report and durable data, because the database value alone would not identify which participant succeeded.

Keep error handling attached to the decision

The example asks psql for its structured SQLSTATE and emits a dedicated result marker. It does not search localized error-message wording. Expected statement errors are followed by rollback; commit errors are checked separately. Missing confirmation, process failure, and transport timeouts escape the retry classifier instead of being guessed into a concurrency category. The initial read and process completion have separate time bounds. These three cases do not generate a deadlock or inject a lost commit reply, and the harness does not implement a shared deadline or caller cancellation.

In an application, use the database driver’s transaction and structured error APIs rather than copying this process harness. Include connection acquisition, every attempt, and backoff in the request’s total deadline. Record attempts and the final error, and stop when the caller cancels. Jitter spreads retries but does not repair permanent contention or guarantee fairness.

Give externally visible operations stable identifiers and a durable way to discover their outcome. Reconcile an uncertain commit before reissuing a non-idempotent action. Keep irreversible effects outside a retryable callback unless their delivery protocol explicitly tolerates repetition. These boundaries are what make recovery safe; a small loop and a sleep only implement the scheduling part.

Sources

PostgreSQL 18 serialization failure handling, retry classifications and recomputing the whole transaction, checked 2026-09-10.
PostgreSQL 18 transaction isolation, snapshot conflicts used by the experiment, checked 2026-09-10.
PostgreSQL 18 psql, SQLSTATE, ERROR, conditional execution, and client error handling, checked 2026-09-10.

Locks and leases tame concurrent writers

Sun, 04 Oct 2026 00:00:00 +0000

Locks and leases coordinate access to shared resources, but their failure behavior depends on the implementation. Here the concern is exclusive writers; locks can also have shared modes. A lease adds a bounded validity period. Neither a familiar name nor a timeout alone proves that a delayed writer cannot damage data after another writer takes over.

Expiry permits handover without canceling old effects. A separate resource detects obsolete tokens only after seeing newer ones. The article’s co-located, one-clock model additionally checks expiry atomically with each write.

A lock has a defined lifetime

A lock need not survive forever after a holder crashes. PostgreSQL session-level advisory locks are released when the session ends, and transaction-level advisory locks are released when the transaction ends. Detecting a broken remote session can take time, but that is different from a lock with no cleanup mechanism. A timeout while waiting for a lock also differs from expiration of ownership.

Inside one process, a mutex may be the right mechanism, but deadlocks and stalled threads remain possible. Across processes, operating-system or database locks may fit better than an application lease. Choose from the resource, ownership scope, and recovery semantics instead of applying a rule that all distributed work must use leases.

Advisory locks protect only cooperating participants. Code that ignores the lock can still touch an otherwise accessible resource. Database-enforced data locks and an external worker’s advisory ownership have different enforcement points. Identify which component actually prevents the conflicting operation rather than relying on every caller’s good intentions.

A lease allows later handover

A coordinator can grant ownership until a deadline, accept timely renewal, and permit another acquisition after expiry. This helps recover from a vanished holder without waiting for an explicit release. However, expiration does not stop a paused process, retract an already sent request, or cancel a write already executing at another service.

Checking ownership before a write leaves a gap between check and effect. A fencing protocol makes the resource participate, rejecting obsolete acquisition generations. With a separate resource that only remembers the largest token it has seen, an old token becomes stale there after a newer token is observed; that mechanism alone does not detect every lease expiry immediately.

A unique acquisition identifier also belongs on renewal and release requests. Owner names can be reused: a delayed release from an earlier acquisition by worker B must not release B’s later lease. Redis’s compare-value release pattern illustrates acquisition-specific ownership, but a random ownership value is not automatically an ordered fencing token.

Renewal has uncertainty

Choose renewal margins from scheduling delays, network latency, pauses, and the clock assumptions of the protocol. One-third of a TTL is a possible operational choice, not a correctness theorem. A ten-second lease does not guarantee complete service recovery within ten seconds; coordinator availability, retries, scheduling, and resource recovery can add delay.

A renewal timeout means its outcome may be unknown. The holder should stop initiating work once safe authority cannot be established, but attempted cancellation does not prove that in-flight effects stopped. Reacquisition must obtain a new identity, and the resource must enforce the required stale-request rule.

When it breaks

An expired task can resume after another holder starts and overwrite newer work if storage accepts its old request. Client-side checks are particularly vulnerable to a pause immediately after checking. Reproduce delayed effects at the enforcement point, not merely a client noticing that its timer expired.

Clock comparisons need an explicit model. Wall-clock jumps, rate drift, and differing clock origins can invalidate deadline calculations. A fast local clock does not universally extend a lease; with a shared absolute deadline it can expire it early. Monotonic elapsed-time measurement avoids wall-clock steps, but does not remove every protocol timing assumption.

Silent renewal-loop failure, excessively short TTLs, missing releases, and delayed duplicate releases affect different transitions. Test them separately. Preserve a token per acquisition rather than treating a reused owner string as sufficient authorization, and remember that application cancellation and lease invalidation are distinct operations.

Check it quickly

Save this as lease.py and run python3 lease.py. The sequential model combines coordinator and resource in one object with one manual nondecreasing clock. It checks ownership, expiry, and generation atomically with each modeled write. It is intentionally stronger than a separate resource that only tracks tokens already received.

class LeaseLostError(RuntimeError):
 pass


class LeaseStore:
 """Serial, co-located coordinator and resource; one manual clock."""
 def __init__(self):
 self.holder, self.expiry, self.fence, self.data = None, 0, 0, None
 self.clock = 0

 def observe(self, now):
 if now < self.clock:
 raise ValueError("clock moved backwards")
 self.clock = now

 def acquire(self, owner, ttl, now):
 self.observe(now)
 if ttl <= 0:
 raise ValueError("TTL must be positive")
 if self.holder is None or now >= self.expiry:
 self.holder, self.expiry = owner, now + ttl
 self.fence += 1
 return self.fence
 return None

 def valid(self, owner, token, now):
 return self.holder == owner and token == self.fence and now < self.expiry

 def renew(self, owner, token, ttl, now):
 self.observe(now)
 if ttl <= 0:
 raise ValueError("TTL must be positive")
 if not self.valid(owner, token, now):
 return False
 self.expiry = now + ttl
 return True

 def release(self, owner, token, now):
 self.observe(now)
 if not self.valid(owner, token, now):
 return False
 self.holder = None
 return True

 def write(self, owner, token, value, now):
 self.observe(now)
 if not self.valid(owner, token, now):
 raise LeaseLostError("expired, wrong owner, or stale acquisition")
 self.data = value


def rejected(store, owner, token, now):
 before = store.data
 try:
 store.write(owner, token, "forbidden", now)
 except LeaseLostError:
 assert store.data == before
 else:
 raise AssertionError("invalid write accepted")


store = LeaseStore()
a = store.acquire("A", 5, 0)
assert a == 1
store.write("A", a, "a-1", 2)
assert store.renew("A", a, 5, 4) and store.expiry == 9
store.write("A", a, "a-2", 8)
rejected(store, "A", a, 9) # Equality is expired.
assert not store.renew("A", a, 5, 9)
b = store.acquire("B", 5, 10)
assert b == 2
store.write("B", b, "b-1", 11)
rejected(store, "B", a, 12)
rejected(store, "A", a, 12)
assert store.acquire("C", 5, 12) is None
assert not store.release("A", a, 12)
assert store.release("B", b, 12)
new_b = store.acquire("B", 5, 13)
assert new_b == 3
assert not store.release("B", b, 13)
assert not store.renew("B", b, 5, 13)
store.write("B", new_b, "new-b", 14)
assert store.release("B", new_b, 14)
c = store.acquire("C", 5, 14)
assert c == 4
store.write("C", c, "c-1", 15)
assert store.data == "c-1"
for operation in (lambda: store.acquire("D", 0, 15),
 lambda: store.renew("C", c, -1, 15),
 lambda: store.write("C", c, "backwards", 14)):
 try:
 operation()
 except ValueError:
 pass
 else:
 raise AssertionError("invalid TTL or backwards time accepted")
print("PASS: renewal, expiry, fencing, same-owner reacquisition, and input guards")

Expect the PASS line. Renewal moves expiry from five to nine; equality is expired. Wrong owners and stale tokens cannot change data. B then reacquires under a new token, and delayed release and renewal using B’s old token fail. Invalid TTLs and backwards time also fail.

Both revised examples passed with Python 3.12.14 on Linux. This establishes the serial model’s transitions, not network consensus, durable fencing, real clock behavior, PostgreSQL locking, Redis availability, or production concurrency.

Sources

Distributed Systems: leases and fencing (Martin Kleppmann), why locks leak and leases help, checked 2026-10-02.
PostgreSQL: advisory locks, application-level locks, checked 2026-10-02.
Redis: SET and locks, building a lock with a time-bound key, checked 2026-10-02.

Deadlocks happen when two writers wait on each other

Tue, 11 Aug 2026 00:00:00 +0000

A deadlock requires a cycle of incompatible waits. If one transaction holds a row while another waits, the holder can still finish and release it. If both hold a resource the other needs, neither can complete that path. PostgreSQL detects the cycle and aborts one transaction, allowing the other to continue. The application must distinguish that rejection from an ordinary slow wait.

A deadlock needs a wait cycle. Either participant may lose all its writes. Retry only the losing complete operation after rollback; consistent resource ordering and a bounded budget remain necessary.

When it breaks

Row updates acquire locks even without explicit locking statements, so two innocent-looking updates in opposite orders can deadlock. More participants, foreign-key checks, triggers, advisory locks, or lock upgrades can make the cycle less obvious. Sorting just one list of row identifiers is not proof that every resource in a larger operation follows one order.

PostgreSQL reports deadlock detection as 40P01. Applications must not rely on the victim being the youngest transaction or the one that did least work. A statement or lock timeout may fire before deadlock detection, producing a different error. Inspect the actual SQLSTATE and server diagnostics instead of classifying every pause as a deadlock or every retry as a successful repair.

Check it quickly

Use two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, on the same exclusively used disposable PostgreSQL 18 database, with the same role and an unused schema name. Stop if setup fails; cleanup is only for this run’s fixture. A creates two rows and shows the detector interval:

CREATE SCHEMA deadlock_demo;
CREATE TABLE deadlock_demo.items (id integer PRIMARY KEY,n integer);
INSERT INTO deadlock_demo.items VALUES (1,0),(2,0);
SHOW deadlock_timeout;

The retained historical report recorded a one-second interval; this review did not rerun the two-session experiment. The statements below have a ten-second budget, leaving room for detection; ensure no shorter lock timeout interferes. If your detector interval is longer, adjust the demonstration’s statement budget accordingly. A records its increment and updates row one, without committing:

\set delta 10
SET statement_timeout='10s';
BEGIN;
UPDATE deadlock_demo.items SET n=n+:delta WHERE id=1;

B records a different increment and updates row two:

\set delta 100
SET statement_timeout='10s';
BEGIN;
UPDATE deadlock_demo.items SET n=n+:delta WHERE id=2;

In A, submit the next block. It waits for B’s row. Do not wait for this block to finish before switching terminals:

\set ON_ERROR_STOP off
UPDATE deadlock_demo.items SET n=n+:delta WHERE id=2;
\echo second_result :SQLSTATE
\set ON_ERROR_STOP on

While A is waiting, submit B’s second update:

\set ON_ERROR_STOP off
UPDATE deadlock_demo.items SET n=n+:delta WHERE id=1;
\echo second_result :SQLSTATE
\set ON_ERROR_STOP on

The cycle now exists. One terminal prints second_result 40P01, while the other eventually prints second_result 00000. Either terminal can lose. Temporarily allowing expected errors makes both outcomes visible; it does not change which transaction the server rejects. In the losing terminal, clear the failed transaction:

ROLLBACK;

In the successful terminal, commit its complete operation:

COMMIT;

In either terminal, inspect both rows:

SELECT id,n FROM deadlock_demo.items ORDER BY id;

If A won, both values are ten; if B won, both are one hundred. Mixed values would contradict the expected atomic result. The losing transaction’s first update was rolled back along with its second attempt, so counting its first statement as completed work would overstate what was committed.

In the losing terminal, retry the whole operation. Its delta variable still identifies the original increment. This time, explicitly lock both rows in identifier order before updating:

BEGIN;
SELECT id FROM deadlock_demo.items ORDER BY id FOR UPDATE;
UPDATE deadlock_demo.items SET n=n+:delta;
COMMIT;

The locking read returns identifiers one and two. With the winner already committed, the retry completes. In either terminal, verify both increments are present exactly once and remove the fixture:

SELECT id,n FROM deadlock_demo.items ORDER BY id;
DO $$ BEGIN
 IF (SELECT array_agg(n ORDER BY id) FROM deadlock_demo.items) IS DISTINCT FROM ARRAY[110,110] THEN
 RAISE EXCEPTION 'deadlock recovery lost or duplicated work';
 END IF;
END $$;
DROP SCHEMA deadlock_demo CASCADE;

Both values must be one hundred ten. The assertion detects lost or duplicated work, not merely absence of an error message. Keep the two original sessions throughout; opening a third terminal for the retry would lose the client variable identifying which operation needs recovery.

Prevent recurrence across the complete path

Consistent ordering removes the particular two-row cycle shown here when all participants follow it and identifiers remain stable. Apply the ordering to every relevant resource and request the needed lock mode deliberately. Audit trigger and constraint paths too; a local ordering convention cannot exclude cycles introduced elsewhere.

Keep transactions short and avoid holding database locks while waiting for a person or remote service. Retry server-declared deadlocks only after rollback, repeating the reads and decisions as well as writes. Use a bounded budget and jitter under contention. Recurring deadlocks deserve investigation of acquisition order and workload, because repeated aborts still consume time and do not guarantee eventual completion.

Sources

PostgreSQL 18 explicit locking, deadlock cycles, unpredictable victims, and lock ordering, checked 2026-09-10.
PostgreSQL 18 serialization failure handling, full transaction retries after concurrency failures, checked 2026-09-10.

MVCC selects row versions through visibility rules

Sat, 08 Aug 2026 00:00:00 +0000

Multi-version concurrency control lets PostgreSQL keep different row versions and decide which ones a query may see. An ordinary reader can continue seeing an older committed version while another transaction writes a newer one. This reduces conflicts between reading and writing, but does not give every transaction one permanently frozen view. Snapshot lifetime depends on isolation, and a transaction can see its own earlier changes even when other transactions cannot.

A’s Repeatable Read snapshot retains an earlier view of B’s work while admitting A’s own later insert. A fresh read after commit sees current rows. The source’s historical concurrency test remains unrerun; this visual adds no runtime evidence.

Visibility follows the snapshot

Under Read Committed, an ordinary read normally gets a new snapshot for each statement. Under PostgreSQL Repeatable Read, the transaction reuses the snapshot established by its first non-transaction-control statement. Merely sending BEGIN does not establish the useful reading boundary. A transaction that begins earlier but first reads later can therefore see commits that occurred between those events.

Updating a row creates a new row version. Visibility rules determine whether an older snapshot reads the earlier version while a newer snapshot reads the update. The physical presence of a tuple is not sufficient to make it visible, and a SQL query does not simply return the newest bytes that happen to be stored on a page.

The transaction’s own changes are an important exception to a simplistic historical-picture metaphor. It can read records it inserted after its snapshot was established. Consequently, its visible data can combine an earlier view of other transactions with its own new work. That behavior is necessary for useful transactions, but it is not a photograph of one externally committed moment.

When it breaks

A plain read’s ordinary table lock does not conflict with normal row updates, yet locking has not disappeared. Two writers can contend for the same row. SELECT FOR UPDATE explicitly requests row locks, and an ACCESS EXCLUSIVE table lock conflicts with ordinary readers. MVCC does not promise that every command containing the word SELECT will proceed without waiting.

An old snapshot can also keep obsolete row versions needed by a reader. Vacuum cannot simply remove a version that remains visible to a relevant snapshot. Space pressure depends on update volume, retained versions, and cleanup progress, not just the number of open transactions. Starting another vacuum does not make an active snapshot forget its visibility requirements.

Check it quickly

The unchanged historical two-session test was not rerun in this review. A single-user backend cannot establish concurrent snapshot behavior.

Open two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, in the same disposable PostgreSQL 18 database. Use the same role and normal autocommit outside explicit blocks. The schema name must be unused. Keep the following order so B commits its changes while A’s reading transaction remains open.

In B, create one committed original row.

CREATE SCHEMA mvcc_visibility_demo;
CREATE TABLE mvcc_visibility_demo.items (id integer PRIMARY KEY,note text);
INSERT INTO mvcc_visibility_demo.items VALUES (1,'old');

In A, establish a Repeatable Read snapshot by reading it.

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;

In B, update the original row, insert a second row, and inspect the committed result.

UPDATE mvcc_visibility_demo.items SET note='new' WHERE id=1;
INSERT INTO mvcc_visibility_demo.items VALUES (2,'other');
SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;

In A, read again, insert its own third row, read again, and commit.

SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;
INSERT INTO mvcc_visibility_demo.items VALUES (3,'mine');
SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;
COMMIT;

In A, read with a fresh statement after committing. Then clean up in B.

SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;

DROP SCHEMA mvcc_visibility_demo CASCADE;

On PostgreSQL 18.6, A’s first read returned (1,old). B completed without waiting for that ordinary reader and saw (1,new) plus (2,other). A still saw only (1,old) when it read again. After A’s own insertion, it saw (1,old) and (3,mine), while B’s second row remained outside A’s snapshot.

Once A committed, its next read returned all three current rows: (1,new), (2,other), and (3,mine). This sequence checks visibility of both another transaction’s writes and one’s own writes. A single reader with no concurrent update would not establish that distinction.

End the snapshot when its purpose ends

Choose a transaction snapshot for work that requires that reading contract, and finish it when the operation is complete. Investigate long-lived snapshots alongside write volume and vacuum progress rather than assuming version storage cleans itself immediately. Repeatable visibility also does not prevent every cross-row business anomaly; an operation may still require Serializable isolation, explicit coordination, or constraints. MVCC supplies row-version visibility, while the application’s complete concurrency contract determines whether its decisions remain correct.

Sources

PostgreSQL 18: MVCC introduction, row versions and reading concurrency, checked 2026-09-10.
PostgreSQL 18: Transaction isolation, snapshot lifetime and visibility of own writes, checked 2026-09-10.
PostgreSQL 18: Explicit locking, reader and writer lock conflicts, checked 2026-09-10.
PostgreSQL 18: Routine vacuuming, retaining visible versions and reclaiming obsolete rows, checked 2026-09-10.

Concurrency · XB Field Notes

Advisory lock lifetime belongs to its connection or transaction

When it breaks

Check it quickly

Match ownership to the operation

Sources

Transactions make atomic commits

Decide what belongs to one unit

When it breaks

Check it quickly

Keep recovery inside the contract

Sources

A transaction can fail and leave you to retry

When it breaks

Check it quickly

Define the retry boundary

Sources

File locks protect a shared protocol and a specific object

When it breaks

Check it quickly

Make the protocol explicit

Sources

A bounded retry gives a transaction another chance

When it breaks

Check it quickly

Keep error handling attached to the decision

Sources

Locks and leases tame concurrent writers

A lock has a defined lifetime

A lease allows later handover

Renewal has uncertainty

When it breaks

Check it quickly

Further reading

Sources

Deadlocks happen when two writers wait on each other

When it breaks

Check it quickly

Prevent recurrence across the complete path

Sources

MVCC selects row versions through visibility rules

Visibility follows the snapshot

When it breaks

Check it quickly

End the snapshot when its purpose ends

Sources