Refined from the original #530 framing after a dry-run uncovered that the
"bootstrap vs migration chain" diff produces mostly noise — most of the
~200 lines of difference are expected (migrations add new tables and
columns over time). initializeDatabase() isn't a parallel path that
diverges from migrations; it's invoked by migration 001 itself, so every
normal install/upgrade runs both.
The genuine drift hazard surfaced during the dry-run: a DB with the
modern bootstrap tables but an empty `migrations` table (which happens
when a backup was restored that lost the migrations table, or someone
invoked initializeDatabase() outside the runner, or the DB was moved
between systems without copying the migrations row) fails to upgrade.
Failure mode:
1. detectExistingSchema sees the bootstrap tables + empty migrations,
treats it as an "existing deployment".
2. Runs the legacy chain first.
3. legacy/008 renames email_templates.subject → subject_en.
4. core/029 (later in the chain) inserts email templates referencing
the pre-rename `subject` column.
5. Postgres rejects: column "subject" doesn't exist; subject_en is
NOT NULL with no default.
Fresh installs avoid this because they only run core/* (and core/059
handles the rename AFTER core/029 has inserted). Real legacy upgrades
avoid it because their migrations table already records legacy/008–028
as applied historically.
Fix in detectExistingSchema:
- Detect the modern bootstrap fingerprint (photo_categories + cms_pages
both present, which initializeDatabase produces as part of the
consolidated post-004-era bootstrap).
- When matched, enumerate every file in migrations/legacy/ and mark
each as applied. This puts the recovery state on the same code path
fresh installs use — only core migrations run, in core order.
- Real legacy upgrades that already have entries in the migrations
table hit no-op markings (markMigrationAsApplied skips duplicates),
so their behaviour is unchanged.
New CI workflow (`.github/workflows/schema-drift.yml`):
- Boots fresh postgres.
- Seeds via `node -e \"require('./src/database/db').initializeDatabase()\"`
— reproduces the recovery state in one line.
- Runs `npm run migrate:safe`.
- Asserts: precondition (bootstrap fingerprint + empty migrations
table), migrate:safe exits 0, final schema has ≥40 tables (soft floor,
not exact pin so future migrations don't force workflow edits),
legacy migrations marked applied (confirms the fingerprint check
actually fired vs. the chain silently bailing).
- Triggers only on PRs that touch backend/migrations/**,
src/database/db.js, knexfile.js, or this workflow.
Manually verified end-to-end before this commit:
Before fix: migrate:safe dies at core/029 with NOT NULL violation
on email_templates.subject_en (17/48 tables present).
After fix: 82 migrations applied + 27 marked applied = 109 total,
final state has all 48 tables matching fresh-install.
Issue body in #530 has been updated to match this refined scope.
Refs: #530, #484, #519
Paul Nothaft
8f0108ce23