A webhook lets a provider notify a registered receiver when something changes. It reverses who initiates the HTTP request, but that alone does not guarantee delivery, immediacy, ordering, or exactly-once processing. The operational contract comes from the provider and the receiver’s handling of acknowledgment and state.

Delivery policy belongs to the provider

Providers differ. GitHub does not automatically redeliver failed webhook deliveries; a receiver needs an appropriate redelivery or reconciliation process. Stripe retries live-mode deliveries for up to three days with exponential backoff, while sandbox retry behavior differs. Neither policy means an unavailable endpoint can remain offline indefinitely without missing work. Record the actual retry window and how an operator discovers exhausted deliveries.

An event is not necessarily a complete current object. Some payloads describe a historical snapshot or contain identifiers requiring another API read. Events can arrive out of order, and two different event IDs can concern the same business transition. Decide whether to apply an event, fetch current state, or reconcile a version before making irreversible business changes. Delivery deduplication and business-operation idempotency solve related but different problems.

Authenticate, retain, then acknowledge

Verify the provider’s signature according to its exact format, using the original request bytes when required. Re-encoding parsed JSON can change whitespace or ordering and break a genuine signature. TLS protects the transport to the endpoint; ordinary server-authenticated HTTPS does not prove which provider sent the request. Keep signing secrets out of logs, compare signatures safely, and use the provider’s rotation procedure.

Timestamp protection is provider-specific. Stripe signs a delivery timestamp and its libraries enforce a tolerance. GitHub’s documented HMAC body signature does not add the same signed timestamp, so one cannot demand an equivalent header universally. A retry can have a fresh delivery signature while representing the same old event. Validate freshness according to the provider, and deduplicate according to the event’s stable identity.

Before returning success for new work, retain it durably in a queue or transactional inbox. Acknowledging first and enqueueing afterward leaves a crash window in which the sender stops retrying but the receiver loses the event. Keep the request path short, but do not remove authentication or durable handoff from it. A valid duplicate already retained can receive the provider’s normal success response without repeating the effect.

When it breaks

An in-memory seen set disappears on restart and races across workers. Recording an ID before its effect without a recoverable pending state can lose work; recording it afterward can repeat the effect after a crash. Use a uniqueness constraint and transaction where possible, then make downstream effects retry-safe with their own operation identifiers. A database transaction cannot atomically cover an unrelated external shipment or payment request by itself.

A rotation applied to only some verifier instances rejects genuine requests intermittently. Permit overlapping valid keys only for the intended transition, monitor rejection reasons, and remove retired keys according to policy. Also bound request sizes and queue growth. A fast 2xx hides trouble if retained events accumulate without progress; delivery health and completed business work need separate observations.

Invented signing bytes and receiver validation order precede process-local deduplication; durable production handoff and provider retry policies are explicitly separate.
Webhook signed delivery: mechanism and evidence boundaries.

Check it quickly

Save this as webhook_verify_demo.py and run python3 webhook_verify_demo.py. It tests an explicitly invented HMAC format with public fixture keys and a fixed clock. It uses only the standard library and in-memory state; it does not contact a payment service or implement a production receiver.

import hashlib
import hmac
import re

SECRET = b"public-demo-key-not-a-real-credential"
OLD_SECRET = b"public-retired-demo-key"
TOLERANCE_SECONDS = 300
seen_ids = set()


def sign(event_id, timestamp, payload, key=SECRET):
    message = f"{event_id}.{timestamp}.".encode("ascii") + payload
    return hmac.new(key, message, hashlib.sha256).hexdigest()


def accept(event_id, timestamp, payload, signature, now):
    # This is an invented wire format, not a Stripe or GitHub verifier.
    if not re.fullmatch(r"evt_[0-9]+", event_id):
        return "rejected"
    if abs(now - timestamp) > TOLERANCE_SECONDS:
        return "rejected"
    if not re.fullmatch(r"[0-9a-f]{64}", signature):
        return "rejected"
    expected = sign(event_id, timestamp, payload)
    if not hmac.compare_digest(expected, signature):
        return "rejected"
    if event_id in seen_ids:
        return "duplicate"
    seen_ids.add(event_id)  # Process-local demonstration, NOT durable acceptance.
    return "accepted"


now = 1_800_000_000
payload = b'{"type":"order.paid","order":"42"}'
fresh = sign("evt_1", now, payload)
assert accept("evt_1", now, payload, fresh, now) == "accepted"
assert accept("evt_1", now, payload, fresh, now) == "duplicate"
# Authenticate even a claimed duplicate before calling it harmless.
assert accept("evt_1", now, payload, "0" * 64, now) == "rejected"
# Change only the body: id and timestamp still match the signed message.
second = sign("evt_2", now, payload)
assert accept("evt_2", now, payload + b" ", second, now) == "rejected"
# Change only the id, leaving body and timestamp intact.
assert accept("evt_3", now, payload, second, now) == "rejected"
for timestamp in (now - 301, now + 301):
    assert accept("evt_4", timestamp, payload,
                  sign("evt_4", timestamp, payload), now) == "rejected"
assert accept("evt_5", now, payload,
              sign("evt_5", now, payload, OLD_SECRET), now) == "rejected"
assert accept("evt_6", now, payload, "not-a-digest", now) == "rejected"
assert seen_ids == {"evt_1"}
assert accept("evt_2", now, payload, second, now) == "accepted"
assert seen_ids == {"evt_1", "evt_2"}
print("PASS: authenticated duplicate, independent body/id tampering, stale/future, retired key, malformed digest")

On 2026-10-19, both complete bilingual fences ran unchanged under Python 3.12.14 with assertions enabled: exit 0, the displayed PASS line and empty stderr. Keep assertions enabled, without -O or PYTHONOPTIMIZE; otherwise the accept calls disappear while PASS still prints. The exact fence covers authenticated repeats, forged duplicates, independent body/ID tampering, ±301 seconds, a retired key and a malformed digest. Separate supplemental tests accepted the inclusive ±300 boundary and rejected seven malformed event IDs; those extra tests are not inside this fence. Five deliberate mutations per language, disabling signature, body, ID, freshness or dedup protection, each failed an assertion before PASS.

Only authenticated new events enter the set; rejected damage does not prevent a later valid delivery. These are local, fixed-clock results with public fixture keys. They establish neither durable acceptance nor cross-process uniqueness, HTTP responses, queue recovery, provider compatibility or end-to-end exactly-once effects. Test those properties at the chosen provider and storage boundary before deployment.

Further reading

Sources