8f0108ce23
Refined from the original #530 framing after a dry-run uncovered that the "bootstrap vs migration chain" diff produces mostly noise — most of the ~200 lines of difference are expected (migrations add new tables and columns over time). initializeDatabase() isn't a parallel path that diverges from migrations; it's invoked by migration 001 itself, so every normal install/upgrade runs both. The genuine drift hazard surfaced during the dry-run: a DB with the modern bootstrap tables but an empty `migrations` table (which happens when a backup was restored that lost the migrations table, or someone invoked initializeDatabase() outside the runner, or the DB was moved between systems without copying the migrations row) fails to upgrade. Failure mode: 1. detectExistingSchema sees the bootstrap tables + empty migrations, treats it as an "existing deployment". 2. Runs the legacy chain first. 3. legacy/008 renames email_templates.subject → subject_en. 4. core/029 (later in the chain) inserts email templates referencing the pre-rename `subject` column. 5. Postgres rejects: column "subject" doesn't exist; subject_en is NOT NULL with no default. Fresh installs avoid this because they only run core/* (and core/059 handles the rename AFTER core/029 has inserted). Real legacy upgrades avoid it because their migrations table already records legacy/008–028 as applied historically. Fix in detectExistingSchema: - Detect the modern bootstrap fingerprint (photo_categories + cms_pages both present, which initializeDatabase produces as part of the consolidated post-004-era bootstrap). - When matched, enumerate every file in migrations/legacy/ and mark each as applied. This puts the recovery state on the same code path fresh installs use — only core migrations run, in core order. - Real legacy upgrades that already have entries in the migrations table hit no-op markings (markMigrationAsApplied skips duplicates), so their behaviour is unchanged. New CI workflow (`.github/workflows/schema-drift.yml`): - Boots fresh postgres. - Seeds via `node -e \"require('./src/database/db').initializeDatabase()\"` — reproduces the recovery state in one line. - Runs `npm run migrate:safe`. - Asserts: precondition (bootstrap fingerprint + empty migrations table), migrate:safe exits 0, final schema has ≥40 tables (soft floor, not exact pin so future migrations don't force workflow edits), legacy migrations marked applied (confirms the fingerprint check actually fired vs. the chain silently bailing). - Triggers only on PRs that touch backend/migrations/**, src/database/db.js, knexfile.js, or this workflow. Manually verified end-to-end before this commit: Before fix: migrate:safe dies at core/029 with NOT NULL violation on email_templates.subject_en (17/48 tables present). After fix: 82 migrations applied + 27 marked applied = 109 total, final state has all 48 tables matching fresh-install. Issue body in #530 has been updated to match this refined scope. Refs: #530, #484, #519
Database Migrations
This directory contains database migrations for the PicPeak photo sharing platform.
Directory Structure
/core
Essential migrations that are always run for new deployments. These include:
init.js- Initial database schema creation- Backup service tables (029-035)
- Gallery feedback tables (033)
- Pre-generated watermarks (061)
/legacy
Migrations needed only when upgrading from older versions. New deployments can skip these as the core schema already includes all necessary tables and columns.
For New Deployments
If you're deploying this application for the first time:
- The
initializeDatabase()function insrc/database/db.jswill create all necessary tables - Only migrations in the
/coredirectory will be run - This ensures a clean, optimized database schema
For Existing Deployments
If you're upgrading from an older version:
- All migrations (both core and legacy) will be run in sequence
- The migration system tracks which migrations have been applied
- Only new migrations will be executed
Running Migrations
# Development
npm run migrate
# Production
npm run migrate:prod
Note on Duplicate Migration Numbers
The legacy directory contains renamed duplicates:
014_add_host_name_to_events_duplicate.js(was duplicate of 014)027_add_rate_limit_settings_duplicate.js(was duplicate of 027)
These have been renamed to avoid conflicts while preserving the migration history.