Explain · XB Field Notes

A slower query needs both a plan and a wait check

Mon, 31 Jan 2028 00:00:00 +0000

A formerly fast query can become slow with the same data, statistics, and access plan. Another transaction may hold a lock, the storage cache may be colder, or the application may wait before PostgreSQL even receives the statement. Rechecking the plan is useful, but it belongs inside a broader comparison of where the elapsed time went. Begin by distinguishing time spent doing work from time spent waiting to do it.

Keep a comparable baseline

Record the statement shape, representative parameter values, returned row count, relevant session settings, and whether the application uses a prepared statement. Different parameters can produce very different populations. A manual query with a literal may not reproduce a prepared statement’s generic plan. A different role or search path can also change which objects and permissions apply.

Inspect current activity while the slowdown is happening. pg_stat_activity can show an active query with a lock wait; active does not mean continuously consuming CPU. pg_blocking_pids identifies sessions currently blocking that backend. Visibility into other roles’ activity requires appropriate privileges, so missing details are not evidence that nothing is running. Use fresh autocommit observations rather than repeatedly reading a cached monitoring snapshot inside one long transaction.

When it breaks

Blindly running ANALYZE treats every slowdown as a stale-statistics problem. That can add maintenance work while leaving the actual blocker untouched. Likewise, building an index on a one-row table cannot release another transaction’s row lock. Accurate estimated rows and a tiny sequential scan can coexist with arbitrarily long application waits.

EXPLAIN ANALYZE is an execution, including for writes, and can itself block. A broad update copied from production is not a harmless inspection command. First inspect waiting and lock ownership; measure a safe fixture or a statement whose effects are understood. Do not terminate an unfamiliar transaction merely because its identifier appears in the blocker list.

A lock-free plan sample still does not equal end-to-end latency. EXPLAIN ANALYZE does not deliver ordinary result rows to the client; without SERIALIZE it also omits their output conversion. TIMING OFF reduces per-node timing overhead while retaining total execution timing. Large result transfer, application decoding, and pool queues therefore need observations outside this plan.

The multi-session story is attributed history with no recovered raw log. Printed observations need manual checks; timeout is a separate failure outcome.

Check it quickly

Open three fresh psql sessions, A, B, and C, using the same role and disposable PostgreSQL 18 database with normal autocommit enabled. The named schema must not already exist, and no other session in the cluster may use the fixture application name. Use a role allowed to create and remove the schema, with activity tracking enabled and no shorter lock timeout. Stop on unexpected errors in each session. Keep the session roles separate; pasting every block into one connection cannot demonstrate blocking between transactions.

In A, create the fixture, then begin a transaction that changes its only row without committing.

\set ON_ERROR_STOP on
CREATE SCHEMA query_recheck_demo;
CREATE TABLE query_recheck_demo.items (id integer PRIMARY KEY,value integer);
INSERT INTO query_recheck_demo.items VALUES (1,0);
ANALYZE query_recheck_demo.items;

BEGIN;
UPDATE query_recheck_demo.items SET value=10 WHERE id=1;
SELECT pg_backend_pid() AS blocker_pid;

In B, start the measured update. It waits until A ends its transaction or the sixty-second statement timeout expires.

SET application_name='xbcatm_recheck_waiter';
SET statement_timeout='60s';
EXPLAIN (ANALYZE, BUFFERS, TIMING OFF)
UPDATE query_recheck_demo.items SET value=value+1 WHERE id=1;

While B is waiting, run this in C, before the timeout. The test application name identifies B without exposing its complete query text.

SELECT pid,state,wait_event_type,wait_event,pg_blocking_pids(pid) AS blockers
FROM pg_stat_activity WHERE application_name='xbcatm_recheck_waiter';

In A, undo the uncommitted change and release its lock. Then collect B’s completed plan.

ROLLBACK;

After B finishes successfully, use C to verify the stored value and remove the fixture.

SELECT value FROM query_recheck_demo.items WHERE id=1;
DROP SCHEMA query_recheck_demo CASCADE;

The original PostgreSQL 18.6 article report, not rerun here and without a recovered raw multi-session log in the retained validation files, described B as active with wait_event_type=Lock, wait_event=transactionid, and A’s process identifier in blockers. The wait-event detail can depend on the exact contention; the blocking relationship is the central evidence. After A rolled back, B incremented the original zero to one. The report says the final query returned one and cleanup removed the schema. Manually check that C identifies exactly B, that its blockers contain A’s reported pid, and that the final value is one; the SQL prints these observations without asserting them.

B’s scan estimated and read one row. Its top Update node returned zero rows because there was no RETURNING clause, not because it changed nothing. Execution time includes the induced wait and depends on how quickly A releases it. If the timeout occurs first, roll back A and use C’s final block to remove the fixture before restarting. The canceled update did not complete its increment. Errors or interruption can leave A’s transaction and the permanent schema; end that transaction before removing this fixture schema.

Follow the observed cause

For a lock wait, investigate transaction lifetime and ownership before changing access paths. For excessive processed rows, compare selectivity and current statistics. For sorts or hashes that spill, inspect their memory and temporary I/O. Finally, compare database execution with application latency to expose connection or client delays. A new plan is meaningful only when it explains the measured regression and preserves the required result.

Sources

PostgreSQL 18: Monitoring database activity, active sessions and wait observations, checked 2026-10-15.
PostgreSQL 18: Using EXPLAIN, row counts and loops and plan interpretation, checked 2026-10-15.
PostgreSQL information functions, blocking-session identification with pg_blocking_pids, checked 2026-10-15.

EXPLAIN ANALYZE runs the query to show real cost

Thu, 16 Dec 2027 00:00:00 +0000

EXPLAIN describes a PostgreSQL execution plan; EXPLAIN ANALYZE also executes the statement and measures its work. The useful comparison is between estimated rows and observed rows at the same node. Neither number is a verdict by itself: an estimate is a model, while a measurement describes one execution under particular data, cache, concurrency, and instrumentation conditions.

ANALYZE really executes. The article example separates restored table changes from consumed sequence values; it was not rerun for this illustration.

Read comparable quantities

Planner costs are relative units, not milliseconds. The two cost values describe startup and completion, and the preferred plan can change when LIMIT makes early rows especially valuable. Actual timings use milliseconds. Comparing a cost of 100 with an elapsed time of 100 is therefore meaningless even when the printed values happen to match.

For a node executed repeatedly, actual rows and node timings are averages per loop. Multiplying by loops helps expose repeated work, but summing every node’s time double-counts work included in parent nodes. Parallel execution makes naive totals still less useful. Read the tree, its loop counts, filters, and row flow before assigning blame to the visually largest number.

BUFFERS separates shared, local, and temporary block activity. Shared hits mean PostgreSQL found blocks in its shared buffer cache. A shared read does not prove a physical disk access: the operating system may satisfy it from another cache. Temporary tables use local buffers, while sort or hash spills can use temporary files. Parent counters include children, so adding all rows of counters inflates the apparent I/O.

Investigate the first meaningful discrepancy

A selective predicate estimated at ten rows but returning ten thousand deserves investigation before rewriting a join above it. Stale statistics, skew, correlated columns, parameter values, and unsupported selectivity models can all contribute. ANALYZE refreshes statistics, but it cannot guarantee exact estimates or repair every plan. Inspect representative data and consider targeted statistics changes only after identifying the missing information.

An accurate row estimate does not prove the query is efficient. Missing access paths, expensive expressions, spills, lock waits, and workload contention can remain. Conversely, a sequential scan can be appropriate for a small table or a query returning much of it. Measure the application’s actual query shape and parameters instead of treating every index-free plan as a defect.

When it breaks

The dangerous mistake is treating ANALYZE as a dry run. An UPDATE changes rows, triggers execute, locks are acquired, and a SELECT can call a function with side effects. Wrapping the statement in a transaction and rolling back restores transactional table changes, but sequence advances and external effects are not generally reversible. Use a disposable dataset when the statement’s effects are uncertain.

Timing also misleads when the first execution warms caches or instrumentation dominates a cheap operation. TIMING OFF retains actual row counts and total execution time while avoiding per-node clock reads. EXPLAIN discards ordinary result delivery, so its duration does not include sending the application’s full response over the network. Repeated representative measurements and application latency answer different parts of the performance question.

Check it quickly

Use psql against a disposable PostgreSQL 18 database. Save this block as explain.sql and run psql -X -v ON_ERROR_STOP=1 -f explain.sql. Connection options must name that database. Everything created here is temporary, and the final rollback removes it.

\set ON_ERROR_STOP on
BEGIN;
SET LOCAL statement_timeout = '5s';
CREATE TEMP TABLE orders (id integer, status text);
INSERT INTO orders SELECT g, CASE WHEN g <= 100 THEN 'paid' ELSE 'new' END
FROM generate_series(1, 10000) AS g;
ANALYZE orders;
EXPLAIN (COSTS ON) SELECT * FROM orders WHERE status = 'paid';
EXPLAIN (ANALYZE, BUFFERS, TIMING OFF)
SELECT * FROM orders WHERE status = 'paid';
CREATE TEMP SEQUENCE measured_calls;
SAVEPOINT before_measurement;
EXPLAIN (ANALYZE, BUFFERS, TIMING OFF)
UPDATE orders SET id = id + 10000 WHERE id <= 3;
EXPLAIN (ANALYZE, TIMING OFF) SELECT nextval('measured_calls');
ROLLBACK TO SAVEPOINT before_measurement;
SELECT count(*) AS updated_rows_after_rollback FROM orders WHERE id > 10000;
SELECT nextval('measured_calls') AS sequence_after_rollback;
DO $$ BEGIN
 IF (SELECT count(*) FROM orders WHERE status = 'paid') <> 100 THEN
 RAISE EXCEPTION 'unexpected selected row count';
 END IF;
 IF EXISTS (SELECT FROM orders WHERE id > 10000) THEN
 RAISE EXCEPTION 'UPDATE was not rolled back';
 END IF;
 IF (SELECT last_value FROM measured_calls) <> 2 THEN
 RAISE EXCEPTION 'sequence advance unexpectedly rolled back';
 END IF;
END $$;
ROLLBACK;

The fixture contains exactly 100 paid orders. The measured SELECT reports 100 actual rows and 9,900 removed by its filter. The UPDATE’s scan reports three rows even though its top node returns zero without RETURNING; zero there does not mean nothing changed. After the savepoint rollback, the changed-row count is zero, but the next sequence value is two because the measured SELECT already consumed one.

The retained review record reports passing SQL assertions from PostgreSQL 18.6 single-user checks; psql invocation was untested. Buffer counts, timings, formatting, and chosen plans can vary; the deterministic results are the row counts, rollback behavior, and sequence advance. The five-second statement timeout bounds individual statements in this exercise. It does not make arbitrary production writes safe to inspect.

Sources

PostgreSQL EXPLAIN, execution effects and instrumentation options, checked 2026-09-10.
PostgreSQL using EXPLAIN, costs, loops and buffer interpretation, checked 2026-09-10.
PostgreSQL planner statistics, estimates and statistics limits, checked 2026-09-10.
PostgreSQL sequence functions, sequence advances outside rollback, checked 2026-09-10.

EXPLAIN shows how Postgres will do the work

Thu, 08 Jul 2027 00:00:00 +0000

EXPLAIN prints the planner’s proposed execution plan; EXPLAIN ANALYZE executes the statement and adds measured rows and times. A slow query needs both a hypothesis and a measurement. Save the query parameters, PostgreSQL version, and original plan before changing an index or rewriting a join. Those details make the next plan comparable and prevent an improvement on one input from hiding a regression on another.

Compare estimated and actual rows, but treat a gap as evidence to interpret, not the diagnosis itself. Loops and early stopping can explain apparently different counts. Accurate estimates also coexist with expensive work, lock waits, or spills. Change one variable at a time and preserve the before-and-after results so the proposed explanation can be tested.

Read the input dependencies, then follow rows upward. A commonly chosen ordered index changes the work while the ten ordered IDs must stay the same; this is a conceptual comparison, not a fresh run.

Read the shape top-down or leaves-first

Text output prints a parent before its indented children. Read downwards to understand which inputs a node consumes, then follow rows upwards to understand the result. Execution can be demand-driven: a LIMIT may stop requesting rows early, while a sort often needs its entire input before returning anything. Indentation describes dependencies, not a universal chronological execution order.

Costs use planner units, not milliseconds. A node’s total cost includes its children, so adding every displayed cost double-counts work. Startup cost estimates the work before the first row; total cost estimates completing the node. Actual times and rows are averages per execution when loops exceeds one. Multiply by loops when judging repeated work, and remember that parent timing also includes descendant work.

Recognize scans, joins, sorts, aggregates, and limits, then inspect their inputs. A nested loop with a selective inner index can be appropriate; a hash join can spill to multiple batches. The node name does not establish whether its work is excessive for this result.

Follow the work and amplification

The largest row count is a clue, not proof of the bottleneck. A cheap sequential scan can outperform many random index reads; a small node repeated thousands of times can dominate a query. Find the first substantial estimated-versus-actual row mismatch, then trace its effect on joins, sorting, and memory use. Check buffers and temporary reads or writes before deciding whether storage access, repeated execution, or a spill explains the delay.

The planner chooses among explored alternatives using a cost model; statistics are one input. Inspect pg_stats for skew or stale estimates, and consider correlations between columns. Extended statistics can describe some same-table relationships, but do not fix every estimate or generally supply cross-table join selectivity. Verify that the proposed statistics actually change the relevant estimate before crediting them.

Bind rows to cost with ANALYZE

EXPLAIN ANALYZE runs the real query. Large row-estimation errors can come from stale statistics, skew, or correlations between columns. Refreshing statistics may help, but does not guarantee a fix. Extended statistics can describe some correlations; a representative test must still confirm the chosen plan and result. An index is justified by the access pattern and measured cost, not just by seeing many filtered rows.

Use an isolated database for write experiments. Rolling back table changes does not undo sequence allocation or external effects from functions and triggers. Even a SELECT can call a function with side effects. Set a statement timeout, understand the statement, and use plain EXPLAIN first when execution cost is uncertain. Buffer reads also do not prove physical disk I/O: the operating system may satisfy them from its cache.

When it breaks

Tuning the wrong statement wastes effort. Rank workload costs, for example with pg_stat_statements, then connect candidates to actual slow requests. A statement with high cumulative time may simply run frequently; that ranking alone does not explain a particular request’s tail latency. Include call counts, representative parameters, and request tracing when choosing the target.

Another failure is parameter sensitivity: one tenant is tiny and another holds most rows, while a prepared statement’s generic plan treats both alike. Compare representative parameters through the same prepared-statement path used by the application. PostgreSQL core has no general SQL hint syntax; investigate statistics, query shape, and generic versus custom plans before changing planner settings globally.

Check it quickly

In a writable scratch database, create data owned only by this session:

BEGIN;
SET LOCAL statement_timeout = '10s';
CREATE TEMP TABLE explain_demo AS
SELECT n AS id, n % 100 AS tenant_id FROM generate_series(1, 20000) AS g(n);
ANALYZE explain_demo;
EXPLAIN (ANALYZE, BUFFERS)
SELECT id FROM explain_demo WHERE tenant_id = 17 ORDER BY id LIMIT 10;
SELECT id FROM explain_demo WHERE tenant_id = 17 ORDER BY id LIMIT 10;
CREATE INDEX explain_demo_lookup ON explain_demo (tenant_id, id);
EXPLAIN (ANALYZE, BUFFERS)
SELECT id FROM explain_demo WHERE tenant_id = 17 ORDER BY id LIMIT 10;
SELECT id FROM explain_demo WHERE tenant_id = 17 ORDER BY id LIMIT 10;
ROLLBACK;

Both queries must return IDs 17 through 917 at intervals of 100, in that order. Compare scan type, sort presence, rows filtered, and buffers; the second plan will commonly use the ordered index. Do not require an exact time or node shape across versions. Repeat on representative data before applying the index to a real workload; rollback removes this experiment.

Sources

PostgreSQL EXPLAIN documentation, EXPLAIN options and execution side effects, checked 2026-10-15.
PostgreSQL using EXPLAIN, plan trees, row estimates, loops and measured costs, checked 2026-10-15.
PostgreSQL 18: Extended statistics, same-table correlations and ANALYZE collection, checked 2026-10-15.
PostgreSQL 18: CREATE STATISTICS, statistics kinds and the table-join selectivity limitation, checked 2026-10-15.
PostgreSQL 18: pg_stat_statements, call counts and cumulative execution time, checked 2026-10-15.