Files
picpeak/backend/migrations
Paul Nothaft 8f0108ce23 feat(install): skip legacy chain when modern bootstrap fingerprint detected (#530)
Refined from the original #530 framing after a dry-run uncovered that the
"bootstrap vs migration chain" diff produces mostly noise — most of the
~200 lines of difference are expected (migrations add new tables and
columns over time). initializeDatabase() isn't a parallel path that
diverges from migrations; it's invoked by migration 001 itself, so every
normal install/upgrade runs both.

The genuine drift hazard surfaced during the dry-run: a DB with the
modern bootstrap tables but an empty `migrations` table (which happens
when a backup was restored that lost the migrations table, or someone
invoked initializeDatabase() outside the runner, or the DB was moved
between systems without copying the migrations row) fails to upgrade.

Failure mode:
  1. detectExistingSchema sees the bootstrap tables + empty migrations,
     treats it as an "existing deployment".
  2. Runs the legacy chain first.
  3. legacy/008 renames email_templates.subject → subject_en.
  4. core/029 (later in the chain) inserts email templates referencing
     the pre-rename `subject` column.
  5. Postgres rejects: column "subject" doesn't exist; subject_en is
     NOT NULL with no default.

Fresh installs avoid this because they only run core/* (and core/059
handles the rename AFTER core/029 has inserted). Real legacy upgrades
avoid it because their migrations table already records legacy/008–028
as applied historically.

Fix in detectExistingSchema:
  - Detect the modern bootstrap fingerprint (photo_categories + cms_pages
    both present, which initializeDatabase produces as part of the
    consolidated post-004-era bootstrap).
  - When matched, enumerate every file in migrations/legacy/ and mark
    each as applied. This puts the recovery state on the same code path
    fresh installs use — only core migrations run, in core order.
  - Real legacy upgrades that already have entries in the migrations
    table hit no-op markings (markMigrationAsApplied skips duplicates),
    so their behaviour is unchanged.

New CI workflow (`.github/workflows/schema-drift.yml`):
  - Boots fresh postgres.
  - Seeds via `node -e \"require('./src/database/db').initializeDatabase()\"`
    — reproduces the recovery state in one line.
  - Runs `npm run migrate:safe`.
  - Asserts: precondition (bootstrap fingerprint + empty migrations
    table), migrate:safe exits 0, final schema has ≥40 tables (soft floor,
    not exact pin so future migrations don't force workflow edits),
    legacy migrations marked applied (confirms the fingerprint check
    actually fired vs. the chain silently bailing).
  - Triggers only on PRs that touch backend/migrations/**,
    src/database/db.js, knexfile.js, or this workflow.

Manually verified end-to-end before this commit:
  Before fix:  migrate:safe dies at core/029 with NOT NULL violation
               on email_templates.subject_en (17/48 tables present).
  After fix:   82 migrations applied + 27 marked applied = 109 total,
               final state has all 48 tables matching fresh-install.

Issue body in #530 has been updated to match this refined scope.

Refs: #530, #484, #519
2026-05-19 22:48:54 +02:00
..

Database Migrations

This directory contains database migrations for the PicPeak photo sharing platform.

Directory Structure

/core

Essential migrations that are always run for new deployments. These include:

  • init.js - Initial database schema creation
  • Backup service tables (029-035)
  • Gallery feedback tables (033)
  • Pre-generated watermarks (061)

/legacy

Migrations needed only when upgrading from older versions. New deployments can skip these as the core schema already includes all necessary tables and columns.

For New Deployments

If you're deploying this application for the first time:

  1. The initializeDatabase() function in src/database/db.js will create all necessary tables
  2. Only migrations in the /core directory will be run
  3. This ensures a clean, optimized database schema

For Existing Deployments

If you're upgrading from an older version:

  1. All migrations (both core and legacy) will be run in sequence
  2. The migration system tracks which migrations have been applied
  3. Only new migrations will be executed

Running Migrations

# Development
npm run migrate

# Production
npm run migrate:prod

Note on Duplicate Migration Numbers

The legacy directory contains renamed duplicates:

  • 014_add_host_name_to_events_duplicate.js (was duplicate of 014)
  • 027_add_rate_limit_settings_duplicate.js (was duplicate of 027)

These have been renamed to avoid conflicts while preserving the migration history.