A Redis stream stores entries under ordered IDs so readers can revisit retained history. Unlike Pub/Sub, the reader need not be online when the producer appends. That difference provides replay, not unconditional durability: deleted entries, retention limits, and persistence settings still determine what remains available after delay or failure.
Separate entry order from event time
XADD with * generates an ID containing a millisecond component and a sequence component. IDs increase within one stream, including when Redis must compensate for a backward clock movement. Explicit IDs must also advance beyond the stream’s last generated ID. Compare their numeric components, not arbitrary string order.
The ID orders insertion into that stream. It does not establish a global order across independent streams or prove when an external event happened. Store the business timestamp separately when needed. Two ordinary XADD calls with identical fields still create separate entries: automatic IDs identify records, not duplicate business operations.
Save the last processed position
XRANGE includes both boundaries by default. Starting it at the last ID already processed returns that entry again; an opening parenthesis makes a range boundary exclusive. XREAD instead returns entries strictly greater than the supplied ID. With COUNT, save the final returned ID and continue from that position.
Ordinary XREAD does not maintain a server-side offset for each reader or remove delivered entries. Independent readers can read the same range, and repeating a request repeats retained data. The application owns its checkpoint. Saving it before a side effect risks skipping work after failure; saving it afterward risks repeating completed work. Coordinate checkpointing with idempotent processing where that distinction matters.
A dollar ID means start after the stream’s current end when Redis handles this call. It is useful for entering a live feed, not for resuming a saved position. Reusing $ after every response can skip entries appended between calls. After receiving data, continue with the actual last ID. Consumer groups add separate delivery and acknowledgement state; XACK is not an offset commit for ordinary XREAD.
When it breaks
A reader can fall behind retention. XTRIM MAXLEN removes the oldest, lowest-ID entries and keeps newer ones; it does not trim the newest end. With the default reference policy, exact MAXLEN enforces the count; approximate trimming may retain extra entries. ACKED can retain unacknowledged entries beyond that count. redis-py’s xtrim defaults to approximate trimming, so the exercise explicitly disables it.
XDEL removes selected records. Neither deletion nor a short retention limit becomes safe merely because a consumer has not acknowledged an entry. Default KEEPREF trimming can leave pending references without payloads. Redis 8.2 added explicit reference policies, including ACKED, but retention remains an application decision. A saved ID cannot reconstruct deleted data.
Streams are Redis data, subject to RDB/AOF configuration, replication behavior, key deletion, and possible eviction. A successful append alone does not establish lossless failover or exactly-once processing. Size retention for expected consumer outages and catch-up capacity, monitor lag, and test recovery using the actual persistence configuration. This experiment deliberately disables persistence and makes no restart-survival claim.
Check it quickly
The article reports macOS, Python 3.14.4, Redis 8.10.1 and redis-py 8.1.0; no matching run log was located among reviewed evidence, and nothing was rerun here. Require POSIX, a C compiler, make, curl, tar and Python 3.10+. Keep assertions enabled: -O or PYTHONOPTIMIZE skips checks and required calls.
redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0
Run with "$redis_lab/venv/bin/python" in the same shell. Check setup succeeds; downloads need network. Redis uses a private /tmp Unix socket, with TCP, scheduled snapshots and AOF disabled. Cleanup attempts to stop its process and remove runtime files; the build/venv remains. Printed summaries precede cleanup. Default Redis is not contacted.
from concurrent.futures import ThreadPoolExecutor
from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
import time
import redis
from redis.backoff import NoBackoff
from redis.retry import Retry
@contextmanager
def instance():
with TemporaryDirectory(prefix="redis-lab-", dir="/tmp") as directory:
endpoint = str(Path(directory) / "redis.sock")
process = subprocess.Popen([
os.environ["REDIS_SERVER"], "--port", "0",
"--unixsocket", endpoint, "--unixsocketperm", "700",
"--save", "", "--appendonly", "no", "--dir", directory,
"--maxmemory-policy", "noeviction",
], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
socket_timeout=2, retry=Retry(NoBackoff(), 0))
try:
deadline = time.monotonic() + 5
while True:
assert process.poll() is None, "Redis exited during startup"
try:
if client.ping():
break
except redis.ConnectionError:
if time.monotonic() >= deadline:
raise
time.sleep(0.02)
yield client
finally:
client.close()
if process.poll() is None:
process.terminate()
try:
_, errors = process.communicate(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
process.communicate()
raise
assert process.returncode == 0, errors.decode()
def ids(entries):
return [entry[0] for entry in entries]
with instance() as client:
for entry_id in ("1-0", "1-1", "2-0"):
assert client.xadd("events", {"value": entry_id}, id=entry_id) == entry_id.encode()
try:
client.xadd("events", {"value": "duplicate"}, id="1-1")
except redis.ResponseError as error:
assert "equal or smaller" in str(error)
else:
raise AssertionError("XADD must reject an ID below the stream's last ID")
generated = [client.xadd("auto", {"value": "same"}) for _ in range(2)]
assert tuple(map(int, generated[0].split(b"-"))) < tuple(map(int, generated[1].split(b"-")))
assert client.xlen("auto") == 2
print("explicit IDs advance; duplicate/lower ID rejected; equal payloads get distinct auto IDs")
assert ids(client.xrange("events", "1-1", "+")) == [b"1-1", b"2-0"]
assert ids(client.xrange("events", "(1-1", "+")) == [b"2-0"]
first = client.xread({"events": "0-0"}, count=2)
second = client.xread({"events": "0-0"}, count=2)
assert first == second and ids(first[0][1]) == [b"1-0", b"1-1"]
saved = first[0][1][-1][0]
remaining = client.xread({"events": saved})
assert ids(remaining[0][1]) == [b"2-0"]
saved = remaining[0][1][-1][0]
print("XRANGE includes its start; XREAD excludes the saved ID; independent reads replay")
client.xadd("events", {"value": "between-calls"}, id="3-0")
assert client.xread({"events": "$"}, block=100) == []
assert ids(client.xread({"events": saved})[0][1]) == [b"3-0"]
def read_live():
with client.client() as connection:
connection.client_setname("stream-lab-reader")
return connection.xread({"events": "$"}, block=1000)
with ThreadPoolExecutor(max_workers=1) as worker:
future = worker.submit(read_live)
deadline = time.monotonic() + 1
while not any(
entry["name"] == "stream-lab-reader" and "b" in entry["flags"]
for entry in client.client_list()
):
assert time.monotonic() < deadline, "reader never blocked"
time.sleep(0.01)
client.xadd("events", {"value": "live"}, id="4-0")
assert ids(future.result(timeout=2)[0][1]) == [b"4-0"]
print("$ skips existing entries; a saved ID catches up; observed blocking receives a new entry")
assert client.xdel("events", "2-0") == 1
assert client.xrange("events", "2-0", "2-0") == []
assert client.xtrim("events", maxlen=2, approximate=False) == 2
assert ids(client.xrange("events")) == [b"3-0", b"4-0"]
assert ids(client.xread({"events": "0-0"})[0][1]) == [b"3-0", b"4-0"]
print("XDEL removes a record; exact MAXLEN keeps the newest two; old IDs cannot restore deleted data")
Explicit IDs produce a deterministic three-entry log. An older ID fails, while two automatic appends with the same payload produce increasing distinct IDs. The boundary assertions show XRANGE including 1-1 and XREAD continuing strictly after it. Two independent reads from zero return identical data without consuming it.
An entry added before a $ call is skipped and that call times out; reading from the saved ID finds it. For live delivery, the producer waits until CLIENT LIST reports the named reader as blocked before appending 4-0. The assertion checks receipt, but the one-second read can expire before the append on a slow host.
Finally, XDEL removes 2-0 and exact MAXLEN keeps only 3-0 and 4-0. Reading from zero now returns those retained records alone. These results demonstrate ID order, cursor boundaries, live waiting, and retention loss. They do not validate crash durability, consumer-group recovery, cross-stream ordering, or production throughput.
Sources
- Redis XADD, entry IDs and monotonic ordering, checked 2026-10-15.
- Redis XREAD, exclusive saved positions, blocking, and the dollar ID, checked 2026-10-15.
- Redis XRANGE, inclusive and exclusive range bounds, checked 2026-10-15.
- Redis XTRIM, retention limits and pending-reference policies, checked 2026-10-15.
- Redis XDEL, explicit removal of retained entries, checked 2026-10-15.
- Redis persistence, RDB and AOF durability tradeoffs, checked 2026-10-15.
- Redis 8.10.1 build instructions, core-only compilation and allocator selection, checked 2026-10-15.
- Python assert, optimization removes assertion evaluation, checked 2026-10-18.
