Migrations are versioned schema changes

Sun, 24 Jan 2027 00:00:00 +0000

The schema of a database is not a one-time design; it evolves with the software that uses it. Instead of editing the schema ad hoc, migrations apply a named, ordered set of changes that any environment can replay.

Schema as code

Treat the schema as source code: each migration is one small, named, ordered step that moves the database from one state to the next, and the sequence itself lives in the repository. A fresh environment replays the steps from an empty database to the current version, which is exactly how tests, CI, and new servers all converge on the same shape without ad-hoc ALTER TABLE edits on a live box. The schema stops being an undocumented fact of a single instance and becomes a reviewable artifact with history, blame, and a pull request.

The order is the contract

With the same starting schema, compatible engine, required extensions, and deterministic steps, ordered replay can reproduce the intended schema. Order alone does not neutralize environmental differences or data-dependent changes. That predictability is the contract: it is what lets a test runner build a disposable database, a deploy spin up a staging clone, and an operator upgrade production all with the same script. The corollary is inviolable: never rewrite or reorder a migration that anyone has already applied, because the runner uses its recorded identity to decide what has run; that identity need not include a content checksum. Editing history can create divergent environments that require inspection and an explicit forward repair.

The danger is the irreversible

A committed migration that drops a column or table destroys stored data; recreating its structure alone cannot recover the values. Give it a plan: back up before it applies, split the drop into its own later migration so you can revert the schema change first, and only run it toward a state you can live with. The real discipline is ‘forward only, with a way back’ — never a migration that silently locks you out. Always practice a destructive migration against a staging schema before it touches production, so the irreversible step is boring by the time it runs for real.

When it breaks

An irreversible migration is the most expensive way this breaks. On PostgreSQL, dropping a column avoids a table rewrite but still needs an ACCESS EXCLUSIVE lock. Many type changes and volatile defaults require rewrites; qualifying binary-compatible changes can avoid them. Avoiding a rewrite does not remove the lock or its waiting time. A large rewrite may wait or run for a long time. PostgreSQL can roll back transactional DDL on failure; partially applied migrations instead depend on earlier commits, nontransactional steps, and the runner’s boundaries. A down migration does not undo data loss either: dropping the email column and re-adding it returns an empty column, not the deleted addresses. Split destructive steps into their own later migration, back up first, and prefer expand-then-contract — add, backfill, remove — so deploys stay boring.

The other two classic failures are concurrent runners and edited history. If two deploy replicas both execute the migrator with no lock, they can apply the same version twice or interleave versions. Production runners therefore use a single migrator or a lock such as a PostgreSQL advisory lock, and golang-migrate marks the database dirty after a failed run. Rewriting a migration that is already applied — or merging two branches that each added version 7 — forks history: environments can diverge. Check what the runner actually records: golang-migrate’s PostgreSQL ledger tracks version and dirty state, not a checksum for every file. Reconcile actual state before a forward repair. Tools such as strong_migrations can flag risky operations during migration execution; run those checks in development before deployment.

The version/name ledger makes the local chain repeatable, but cannot detect arbitrary code edits or restore deleted values. Historical execution passed only for the in-memory model; real locks, concurrent runners and backup restores were not tested. No example was rerun for this drawing.

Check it quickly

Save the following as migration_chain_check.py and run python3 migration_chain_check.py with Python 3.10 or newer. It uses only the standard library and models three migrations over an in-memory schema and an applied ledger, so no database is needed. Its identity records only version and name, not executable content; it detects renaming, not arbitrary code edits.

"""Simulate a versioned migration chain with an applied ledger (stdlib only)."""

# Each migration is (version, name, up, down). down=None means irreversible.
# The `applied` dict emulates a schema_migrations table: version -> a name-based identity (not a content digest).

def identity(version, name):
 return "%d:%s" % (version, name)


def make_migrations():
 db = {"users": []}

 def up1():
 db["users"] = []

 def down1():
 del db["users"]

 def up2():
 for user in db["users"]:
 user["email"] = None
 db["has_email_column"] = True

 def down2():
 # Irreversible in effect: dropping the column destroys its data.
 for user in db["users"]:
 del user["email"]
 del db["has_email_column"]

 def up3():
 db["index_on_email"] = True

 def down3():
 del db["index_on_email"]

 migrations = [
 (1, "create_users", up1, down1),
 (2, "add_email_column", up2, down2),
 (3, "add_email_index", up3, down3),
 ]
 return db, migrations


def apply_single(db, applied, migrations, version):
 by_version = {m[0]: m for m in migrations}
 assert version in by_version, "unknown migration version: %r" % (version,)
 wanted = by_version[version]
 if version in applied:
 if applied[version] != identity(version, wanted[1]):
 raise RuntimeError("version drift: v%d was edited after apply" % version)
 return "skipped" # idempotent re-run
 missing = [m[0] for m in migrations if m[0] < version and m[0] not in applied]
 if missing:
 raise RuntimeError("out of order: v%d needs missing %s first" % (version, missing))
 wanted[2]()
 applied[version] = identity(version, wanted[1])
 return "applied"


def migrate_up(db, applied, migrations):
 results = []
 for m in migrations:
 results.append((m[0], apply_single(db, applied, migrations, m[0])))
 return results


def migrate_down(db, applied, migrations, target):
 by_version = {m[0]: m for m in migrations}
 while applied and max(applied) > target:
 version = max(applied)
 migration = by_version[version]
 assert migration[3] is not None, "v%d is irreversible" % version
 migration[3]()
 del applied[version]
 return sorted(applied)


# 1. Fresh database replays the whole chain in order.
db, migrations = make_migrations()
applied = {}
assert migrate_up(db, applied, migrations) == [(1, "applied"), (2, "applied"), (3, "applied")]
assert sorted(applied) == [1, 2, 3] and db["has_email_column"] is True
print("fresh replay applies v1..v3 in order:", sorted(applied))

# 2. Re-running is idempotent: the applied ledger makes it a no-op.
assert migrate_up(db, applied, migrations) == [(1, "skipped"), (2, "skipped"), (3, "skipped")]
assert sorted(applied) == [1, 2, 3]
print("second run is a no-op; applied ledger still:", sorted(applied))

# 3. Out-of-order apply is rejected instead of silently forking history.
db2, migrations2 = make_migrations()
applied2 = {}
try:
 apply_single(db2, applied2, migrations2, 3)
except RuntimeError as error:
 assert "out of order" in str(error), error
 print("out-of-order v3 rejected:", error)
else:
 raise AssertionError("applying v3 before v1..v2 must fail")
assert applied2 == {} and "index_on_email" not in db2

# 4. Rollback runs downs in reverse and only to the requested target.
assert migrate_down(db, applied, migrations, 2) == [1, 2]
assert "index_on_email" not in db and sorted(applied) == [1, 2]
assert migrate_up(db, applied, migrations) == [(1, "skipped"), (2, "skipped"), (3, "applied")]
print("rollback to v2 drops the index; re-up restores it:", sorted(applied))

# 5. A down migration destroys data: emails do not come back on re-up.
db["users"].append({"id": 1, "email": "a@example.com"})
assert migrate_down(db, applied, migrations, 1) == [1]
assert db["users"] == [{"id": 1}] # email column (and its values) is gone
assert migrate_up(db, applied, migrations)[1] == (2, "applied")
assert db["users"] == [{"id": 1, "email": None}] # re-up gives NULL, not the old address
print("down v2 drops stored emails; re-up yields None instead of the original")

# 6. Editing an already-applied migration is version drift, not a fix.
drifted = [(v, n + "_renamed" if v == 2 else n, u, d) for v, n, u, d in migrations]
try:
 migrate_up(db, applied, drifted)
except RuntimeError as error:
 assert "drift" in str(error), error
 print("edited history detected:", error)
else:
 raise AssertionError("renaming applied v2 must fail")

print("PASS: ordered up, idempotent ledger, rejected jump, ordered rollback, lossy down, drift check")

Six scenarios run in order: fresh replay applies v1 through v3, a second run is a no-op thanks to recorded identities, jumping to v3 is rejected, rollback to v2 removes only the index, rolling back v2 proves the down step is lossy as the address returns None, and renaming an applied migration trips the drift check. The final line prints PASS only if every assertion holds.

This fixture checks ordering, idempotency, rollback, and drift logic — not real database locks, concurrent migrator processes, or backup restores. The revised exact example passed on Python 3.12.14 on Linux. Before production, rehearse the real migration on a staging copy and confirm the down path.

Sources

golang-migrate/migrate, versioned up/down migrations with dirty-state tracking, checked 2026-10-15.
PostgreSQL: ALTER TABLE, locking and rewrite behavior of table changes, checked 2026-10-15.
strong_migrations, catches dangerous migrations such as unindexed or blocking operations, checked 2026-10-15.

Migrations · XB Field Notes