spawnToFile and spawnFromFile passed an unopened WriteStream/ReadStream
directly as a stdio entry to child_process.spawn. Older Node versions
auto-extracted .fd; Node 22 throws synchronously:
The argument 'stdio' is invalid.
Received WriteStream { fd: null, path: '/backup/database/...sql', ... }
Bug bit Ralf's install once today's `bugfix/crm-backup` image landed —
Node 22 came with that image, and Stage A's inline-dump path is the
first caller of spawnToFile on this install. Latent on the previous
image (Node 20); fatal on this one. restoreService's pre-restore
safety snapshot uses the same helper and would have hit it next time
a restore ran.
Cure: stdio: ['ignore', 'pipe', 'pipe'] (and ['pipe', 'pipe', 'pipe']
for spawnFromFile) + manual pipe of child.stdout/stdin through the
file stream. Works on every Node version. Also wires the WriteStream's
'error' event to the promise via settleReject so a future EACCES /
ENOSPC reaches the caller's try/catch instead of becoming a process-
fatal unhandled error event — closing the same "Stage A guard
bypassed" hole noted in the spawned follow-up task.
Side benefit: outStream.end() now awaits flush before resolving, so
fast pg_dump runs can no longer produce a truncated dump.
databaseBackupService.backup() did `const [runId] = await db(...).insert({...})`
without a .returning() — works on SQLite (knex returns [lastInsertId]) but
throws "(intermediate value) is not iterable" on Postgres (knex returns
a non-iterable shape).
Bug was latent until Stage A of the backup-hardening plan wired this
method into the "Run Backup Now" inline-dump path. Before Stage A only
the scheduled cron + the dedicated admin-DB-backup page called it, and
Ralf's install had never exercised either — so the inline-dump default
landing in production was the first time the destructure ran on his PG.
Cure: same explicit .returning('id') + dual-shape coalesce pattern that
backupService.js uses for its own backup_runs insert (line 949).
Two more sibling files have the same anti-pattern (userManagementService,
customerAccountsService — invitation flows) and will bite under the
same conditions; spawned a follow-up task to fix them in a separate PR.
Stage C of the three-stage backup-hardening plan (Stage A: inline
DB dump + fail-loud landed in 7fdf01a; Stage B: config-driven walker
in 302fc6b). Answers the "what would I lose if I clicked Run Backup
Now right now?" question that Stage B made possible to answer.
Backend:
- new backupCoverageService.js: per-path coverage classification,
drift detection (top-level subdirs not in backup_paths and not
in the backups/tmp allow-list), DB-dump mode + staleness block
- new GET /api/admin/system-health/backup-coverage route, same
auth + settings.view permission as /backup-integrity
- 7 integration scenarios pinning the classifier behaviour
Frontend:
- new BackupCoverageCard with auto-fetch (cheap; no recursion)
- new Coverage tab on BackupManagement next to Integrity
- en + de i18n; other locales fall back to en keys until a native
speaker reviews
Verification:
- 26/26 backup integration tests pass (Stage A 5 + Stage B 7 +
Stage C 7 + adminBackupIntegrity 4 + businessDocs 3)
- frontend build clean
- 4 pre-existing integration failures confirmed unrelated
Stage B of the three-stage backup-hardening plan (Stage A:
inline-DB-dump + fail-loud guard already landed). The file-backup
walker used to hard-code its subdirectory list inside
`getFilesToBackupInternal`, which is the same footgun that hid the
`business-docs` gap for ~6 months — a new feature drops artefacts
under STORAGE_PATH and the maintainer has to remember to edit the
walker.
Now driven by a `backup_paths` table:
- Migration 108 creates the table and seeds the 7 canonical
defaults (events/active, events/archived, thumbnails, previews,
heroes, uploads, business-docs). Seed data lives on the
migration as `DEFAULT_PATHS` so the boot self-heal can re-use it.
- `_backupPathsBoot.js` mirrors `_emailTemplateBoot.js`: on every
boot it diffs the canonical list against the current rows and
`INSERT ... ON CONFLICT DO NOTHING`s the missing ones. Keeps
admin edits intact, picks up new defaults shipped after the
install (Knex won't re-run migration 108). Wired into server.js
just before `startBackupService()`.
- Walker now calls `resolveBackupPaths(config)` which:
* reads `backup_paths WHERE include_in_default=true ORDER BY
display_order`
* falls back to a hard-coded `LEGACY_BACKUP_PATHS` if the
table is missing OR empty (defense in depth — never silently
scans nothing)
* gates each row by its `feature_flag` column (matches how
`backup_include_archived` already worked; data-driven now)
- Backward compatible: `getFilesToBackup(true|false)` still works
for legacy callers and the existing businessDocs test. New
callers should pass the full config object so feature gates
other than `backup_include_archived` evaluate correctly.
Tests:
- new: `backupService.configurableWalker.test.js` — 7 cases
covering canonical seed, toggling include_in_default, runtime
INSERT picked up without restart, feature_flag gating both on
and off, empty-table → LEGACY fallback, boolean backward compat
- all 15 backup-walker integration tests pass
(configurableWalker 7 + inlineDbDump 5 + businessDocs 3)
- frontend build clean
- 4 pre-existing integration failures (webhookDelivery, storage
backend, adminPhotos.reference, imageProcessor.storage) confirmed
unrelated via `git stash` baseline run
Stage C (CRM feature coverage audit + diagnostic UI) follows
in a separate commit.
The previous file-backup workflow only LOOKED UP an existing
database dump via getDatabaseBackupInfo() and silently shipped a
files-only manifest when none was found. Admins clicking "Run
Backup Now" (or relying on the schedule) got an apparent success
that omitted every customer / quote / invoice / contract / payment-
log row. The data-loss footgun was discovered 2026-05-29 when an
admin who'd been "backing up" for weeks via the UI lost the entire
CRM after a routine docker compose down -v — every produced
manifest had database: { backup_file: null, size: 0, tables: {} }.
New helper `ensureDatabaseDumpForBackup(config)` encapsulates:
1. Inline pg_dump (or SQLite copy) before the file scan, via
databaseBackupService.backup(). Result lands in
database_backup_runs and is picked up by the existing
getDatabaseBackupInfo lookup that writes the manifest.
2. Fail-loud guard: if no usable dump file is reachable (path
missing, 0 bytes, or never existed), throw — the existing
catch in runBackupInternal marks the backup_runs row failed
with the error_message and emails the admin if configured.
No more silent files-only manifests.
3. Opt-out: `backup_database_inline_dump = false` skips the
inline dump for admins who already run their own scheduled
`backup_database_schedule`. The fail-loud guard still
applies, so an opted-out install with no recent dump still
aborts loudly instead of producing a partial backup. Default
ON is encoded as "skip only when explicitly false" — undefined
(existing installs upgrading) falls through to the safe-
default ON branch.
The helper returns the verified `databaseInfo` so the manifest-build
step at runBackupInternal:917 reuses it instead of calling
getDatabaseBackupInfo a second time. S3/future destinations that
override `result.databaseInfo` are still respected (the existing
`result.databaseInfo ||` fallback shape stays put).
Test suite covers: default-on happy path, dump-throws-aborts-run,
opt-out + recent dump + proceeds, opt-out + no-dump + fail-loud,
opt-out + 0-byte dump + fail-loud. Mocks
databaseBackupService.backup so the tests don't depend on pg_dump
or sqlite3 CLI binaries being installed.
Stage A of three-stage backup hardening plan. Stage B (config-
driven walker) and Stage C (audit + diagnostic UI) follow in
separate commits.
The previous file-backup workflow only LOOKED UP an existing database
dump via getDatabaseBackupInfo() and silently shipped a files-only
manifest when none was found. Admins clicking "Run Backup Now" (or
relying on the schedule) got an apparent success that omitted every
customer / quote / invoice / contract / payment-log row. The
data-loss footgun was discovered 2026-05-29 when an admin who'd been
"backing up" for weeks via the UI lost the entire CRM after a routine
docker compose down -v — every produced manifest had database:
{ backup_file: null, size: 0, tables: {} }.
Changes to runBackupInternal:
1. Inline pg_dump (or SQLite copy) before the file scan, via
databaseBackupService.backup(). Result lands in
database_backup_runs and is picked up by the existing
getDatabaseBackupInfo lookup that writes the manifest.
2. Fail-loud guard after the dump step: if no usable dump file is
reachable (path missing, 0 bytes, or never existed), throw —
the existing catch block marks the backup_runs row failed with
the error_message and emails the admin if configured. No more
silent files-only manifests.
3. Opt-out: `backup_database_inline_dump = false` skips the inline
dump for admins who already run their own scheduled
`backup_database_schedule`. The fail-loud guard still applies,
so an opted-out install with no recent dump still aborts loudly
instead of producing a partial backup. Default ON is encoded
as "skip only when explicitly false" — undefined (existing
installs upgrading) falls through to the safe-default ON path.
Test suite covers: default-on happy path, dump-throws-aborts-run,
opt-out + recent dump + proceeds, opt-out + no-dump + fail-loud,
opt-out + 0-byte dump + fail-loud. Mocks
databaseBackupService.backup so the tests don't depend on pg_dump
or sqlite3 CLI binaries being installed.
Stage A of three-stage backup hardening plan. Stage B (config-driven
walker) and Stage C (audit + diagnostic UI) follow in separate
commits.
The fix shipped in 3ab3756 added /backup to the boot-time chown list.
That broke installs that don't bind-mount ./backup:/backup — the
single greedy `chown -R /a /b /c /backup` returned non-zero on any
individual failure, exiting the script and putting the backend into
a restart loop.
Reverting to the upstream-stable version. The original EACCES at
backup time is better fixed by admins pointing the backup destination
at a writable path via the admin UI (e.g. /app/storage/backups,
which the script already chowns) rather than baking a /backup
assumption into every install's boot path.
The docker-compose `./backup:/backup` mount was the only bind mount
not included in wait-for-db.sh's startup chown step. On a fresh
install (or any time the mount point is recreated), it stays
owned by root, and the nodejs (UID 1001) process running the
backup service gets EACCES when trying to mkdir under /backup.
Added /backup to both the chown list (root branch) and the
writable-check list (compose `user:` override branch), each guarded
by `[ -d /backup ]` so installs that don't use the bind mount —
native deployments, k8s with a different backup destination, etc. —
still boot cleanly.
Existing installs hit by this need a one-time host-side
sudo chown -R 1001:1001 <host-mount-for-/backup>
because the on-disk ownership won't fix itself; the script only
chowns at startup, and the directory was already created with
the wrong ownership by Docker's mount-point auto-creation. From
this commit onward, fresh installs are correct from the first
boot.
Line 205 of databaseBackup.test.js reassigned `fs.unlink` directly
(`fs.unlink = jest.fn(...)`), which permanently mutated the global
fs.promises module. Every test running after this in the same jest
worker process inherited the no-op stub, including
integration/storageBackend.test.js — whose LocalFsStorage.delete()
silently became a no-op, making the subsequent exists() assertion
flip from false to true.
Confirmed by adding a diagnostic patch to LocalFsStorage.delete:
post-await fsp.unlink, fs.existsSync(abs) returned true. unlink had
resolved without throwing but the file was still there → the unlink
was a mock.
Fix: jest.spyOn(fs, 'unlink').mockResolvedValue(undefined) + a
matching mockRestore() at the end of the test. Behaviour is
identical inside this test; the original fs.unlink is restored
when the test finishes, so subsequent tests get real fs.unlink
again.
Pre-existing issue — has been latent on upstream/beta forever.
Only surfaces consistently when CI load shifts jest's worker
allocation such that databaseBackup and storageBackend land in
the same worker process. This PR's extra integration test files
made that allocation deterministic locally and frequent enough on
CI to fail reliably.
The 5-minute session-sweep interval at sessionTimeout.js:17 fired at
module-load time without .unref(), so every jest worker that
transitively required this module (server.js → middleware → most
of the route layer) kept the event loop alive forever. The worker
then got force-killed on shutdown, surfacing as the longstanding
"worker failed to exit gracefully" warning at the end of every CI
run on upstream/beta.
Under enough I/O / memory pressure on a CI runner, the force-kill
could land MID-test rather than after the suite finished, taking
out whatever else was running on that worker — most visibly
integration/storageBackend.test.js on PR #555's runs.
.unref() makes the timer not keep the loop alive on its own.
Production behaviour is unchanged: the timer still fires every
5 min as long as anything else is holding the loop open (the HTTP
server, always).
CI's SQLite returned `[N]` (plain int) from `.insert().returning('id')`
while local SQLite returned `[{ id: N }]` (object form). The brittle
`const [{ id }] = ...` destructure crashed on the int shape. Switched
to the unwrap pattern used by the existing crmDb test harness so the
suite runs on both PG and every SQLite/knex combo the project supports.
Diagnostic for the bug fixed in a9280ea — confirms every *_path
column on quotes / contracts / invoices points at a file that
actually exists on disk and (where a *_sha256 column is set) the
file's bytes still hash to the expected value. Read-only;
on-demand only; no scheduler.
Per the design decisions locked in this PR's design call:
D1 — on-demand only for v1; scheduling deferred until we have
runtime data on large installs
D2 — not auto-triggered after restore; surface a "verify
integrity now" CTA on the restore-completed screen instead
D3 — wet-upload contracts hash-verified same as system-rendered
(signed_pdf_sha256 is computed at upload time, no special
case needed in the verifier)
Coverage (single source of truth in backupIntegrityService.CHECKS):
quotes.pdf_path existence
contracts.pdf_path + pdf_sha256 existence + hash
contracts.signed_pdf_path + signed_pdf_sha256 existence + hash
contracts.signed_customer_signature_path existence (PNG/JPG, no hash)
contracts.signed_admin_signature_path existence (PNG/JPG, no hash)
invoices.pdf_path existence
invoices.imported_pdf_path existence (admin-uploaded scans)
Report shape buckets each row into verifiedOk / missing /
hashMismatches / existsButNoHash so callers can distinguish hash-
verified from existence-only — the latter is weaker evidence in
a legal dispute and the UI should reflect that.
Route GET /api/admin/system-health/backup-integrity accepts an
optional ?scope= CSV filter (quote | contract | contract-signature
| invoice). Unknown scope tokens are rejected with a 400 +
BACKUP_INTEGRITY_UNKNOWN_SCOPE code rather than silently scanning
everything.
Frontend half (BackupIntegrityCard on a System Health page) is
deferred until backlog #11 (System Health page) is scaffolded.
The endpoint is independently useful via curl in the meantime.
backupService.getFilesToBackupInternal() enumerated a fixed list of
storage subdirectories (events/active, events/archived, thumbnails,
previews, heroes, uploads) and silently omitted the entire
business-docs/ tree. Every CRM PDF artefact and signature image fell
outside the in-app scheduled backup — restoring the DB without the
PDFs would have left every *_path column on quotes/contracts/invoices
as a broken FK and lost forensic evidence (the customer signature
PNG/JPG drawn on the public signing page is referenced by
contracts.signed_customer_signature_path; the rendered contract PDF
is referenced by signed_pdf_path with a stored signed_pdf_sha256
that would have nothing to verify against; wet-uploaded contracts
and admin-imported historical invoices are irrecoverable by design
since no renderer can reproduce them).
Single new scanDirectory call after the existing uploads scan,
covering:
- business-docs/quote/<year>/*.pdf
- business-docs/contract/<year>/*.pdf
- business-docs/contract/signatures/<contract_id>/*.{png,jpg}
- business-docs/invoice/<year>/*.pdf
- business-docs/invoice-imports/<year>/*.pdf
- and incidentally business-docs/dev-test/ (managed by adminDev.js,
bounded to 7 newest files, harmless to back up)
Verified that no migration is needed: hasFileChanged returns
!existing || checksum mismatch, so the first backup after this lands
flags every business-docs/** file as new and copies it. Restore path
in restoreService.performFilesRestore uses fs.mkdir({ recursive:
true }) on path.dirname(targetPath), so business-docs subdirectories
are recreated automatically from manifest entries — no restore-side
code change required.
Integration test pins the contract so a future refactor cannot
silently drop business-docs again.
The shell-script backup at scripts/backup.sh already covered all of
this via blanket `tar -czf storage`; only the in-app service was
affected.
Closes#567.
The sidebar already had a "vX.Y.Z available" indicator (#566 made it a
link to that release's page) but there was no way to read the actual
changelog inline or to grab a copy-paste upgrade command. This adds
the modal the issue spec'd, layered on top of the existing
updateCheckService / environmentService backend infrastructure that
already shipped.
## Backend
- `updateCheckService.fetchAvailableVersions` now returns full release
objects (tag, name, body, publishedAt, htmlUrl) instead of just
version strings — body data is what the changelog modal renders.
`checkForUpdates` extracts the version strings for its existing
consumers; no API change visible to callers.
- New `getReleasesSince(currentVersion, channel)` returns the list of
releases strictly newer than current, filtered to the user's
channel. Reuses the same 1-hour cache as `checkForUpdates` so the
modal opening doesn't trigger an extra GitHub round-trip.
- New `GET /admin/system/updates/changelog` route in `adminSystem.js`,
same auth + UPDATE_CHECK_ENABLED gating as the existing
/updates and /updates/instructions endpoints.
- 4 unit tests (axios mocked) pin: strictly-newer filtering,
channel-scoped, empty array on GitHub fetch failure, empty array
when already on latest.
## Frontend
- New `UpdateAvailableModal.tsx` — opens from the sidebar chip. Two
sections:
1. **How to upgrade** — fetches /updates/instructions for the
environment-detected copy-paste command (Docker compose / git /
standalone). Copy-to-clipboard button per step.
2. **Release notes** — fetches /updates/changelog for every
version between current and latest in the user's channel.
Latest is auto-expanded; older releases are collapsed by
default (click to expand). Each release also has a "View on
GitHub" link to the canonical release page.
- Renders release body markdown through the existing safe
MarkdownContent component (marked + DOMPurify allowlist).
- New `updateDismissal.ts` helper — single localStorage key holds the
last-dismissed version. Chip stays hidden until a STRICTLY newer
version appears, using the same compare semantics as the backend
(stable > beta, higher beta > lower beta, semantic numeric on
major.minor.patch). 9 unit tests pin the rules.
- `VersionInfo.tsx` — chip is now a button that opens the modal
instead of an external link (the #566 link-to-release behaviour is
preserved on the modal's per-release "View on GitHub" affordance).
Dismissal triggers an immediate re-render so the chip disappears
without waiting for the next route change.
No new dependencies — uses `marked` + `DOMPurify` that were already
present in the bundle for the contract block renderer.
Reviewer feedback on #555: nextQuoteNumber inside createQuote's
db.transaction was called without passing the outer trx, so
claimNextSequence opened its own connection — Postgres tolerated this
via the pool, SQLite (1-connection default) deadlocked on every quote
creation.
Audited the same pattern across invoiceService + contractService and
found five more matching call sites:
- createInvoice (single-row path after installment auto-route)
- spawnInstallmentInvoices (per-sibling claim inside the loop)
- createStorno
- createContract
- createFromQuote
All now thread trx through to nextXxxNumber → claimNextSequence so
the claim joins the caller's transaction on both engines.
convertToInvoiceOnly's Path B (standalone-contract) is the lone
remaining nextInvoiceNumber() call without trx — that path isn't
wrapped in a transaction at all (separate concern: sequence-number
leak on insert failure, tracked separately).
The CRM template seeders (crmEmailTemplates / contractEmailTemplates /
eventReminderTemplates) were idempotent and ready, but only
contractEmailTemplates was actually called (lazily, by contractService
sends). crmEmailTemplates had no caller anywhere — every install that
didn't pre-exist its templates failed every quote_sent / invoice_sent /
storno_issued / invoice_reminder_* send with "Email template '<key>'
not found". The queue processor retries 3 times then leaves the row
in status='pending', retry_count=3, silently dead with no admin
surface (see project_crm_backlog for the eventual System Health page).
Fix: wire all three seeders into server.js startServer() right before
startEmailQueueProcessor. The new _emailTemplateBoot.js orchestrates
all three and then, for any template_key it just inserted, resets
retry_count on stuck email_queue rows of that email_type so the
queue processor's next tick picks them back up. Recovery is targeted:
unrelated retry-exhausted rows (e.g. SMTP-timeout failures) are not
touched.
Integration test boots a fresh CRM DB, pre-seeds a stuck quote_sent
row plus an unrelated stuck row, runs the boot helper, and asserts:
templates landed, stuck quote_sent row was reset, unrelated row was
left alone.
Already-deployed installs heal automatically on the next backend
restart after this lands.
Wires customer_accounts.billing_email into the invoice, Storno, and
payment-reminder send paths. Previously the column existed on the
schema and the customer-detail page rendered an input for it, but no
send path read it — every outbound email landed on customer_accounts.email
regardless. That mismatch is the failure mode flagged in
feedback_data_driven_completeness: a UI field that promises behavior
the backend silently doesn't deliver.
Routing matrix:
- invoice / Storno / payment reminder
To: billing_email (fallback email when unset)
CC: email (when billing_email took the To slot) + per-doc cc_pdf_email
- quote / contract / event reminder / gallery share
To: email (unchanged — decision-maker address)
- payment-check / paid-notification
To: admin contact (unchanged — internal flow)
A new resolveBillingRecipients helper centralises the rules:
prefer billing_email, dedupe addresses case-insensitively, keep
per-doc cc_pdf_email as a supplemental CC. Lives in its own file
(_billingRecipients.js) to match the _renderContext.js convention.
Drops the isInt({ min: 0 }) constraint on lineItems.*.unitPriceMinor
in both the adminInvoices and adminQuotes POST/PUT validators so
admins can add Treuerabatt / Frühbucherrabatt rows as standalone
negative-priced lines (matches standard DE/CH invoice practice).
A service-layer guard rejects saves whose computed total goes below
zero (INVOICE_TOTAL_NEGATIVE / QUOTE_TOTAL_NEGATIVE, both 400) so a
mis-typed discount can't accidentally mint a credit-balance invoice
that would masquerade as a regular row in dashboards. Credit notes
still belong in the Storno path (createStorno), which is unchanged.
Quote-side integration coverage is omitted for now — createQuote's
cold-require path takes ~30s under the test harness; the invoice
test exercises the same validator + guard shape.
The pre-event reminder feature shipped with 5 seeded templates
(event_reminder_default + wedding/birthday/corporate/other) but the
CRM → Development "Send any CRM email to me" picker only listed the
quote/invoice/contract templates. Maintainer can now eyeball each
reminder category's body without staging a real event.
Backend:
- Extend TEMPLATES_KEYS in adminDev.js with all 5 reminder keys.
- Add event_date (today+2d), days_before (2), business_name (from
business_profile.legal_name) to the common payload so the
{{tokens}} in the reminder bodies resolve.
Frontend:
- Extend CrmEmailTemplateKey union.
- Add TEMPLATE_LABEL_KEYS entries.
- EN+DE i18n labels under crmDev.templates.label.event_reminder_*.
No PDF attachment — reminders are body-only emails (matches the
real flow).
Adds two pieces:
- __tests__/integration/helpers/crmDb.js — boots a temp-SQLite test
DB by invoking every migrations/core/*.up() directly. Bypasses
knex's Migrator because its exclusive write lock deadlocks
001_init's nested initializeDatabase() call. ~1 second cold start.
- __tests__/integration/crmSchema.test.js — 36 assertions on the
table + column layout after the consolidated CRM migration runs.
Pins:
- every CRM table present (quotes, contracts, invoices + the
eight supporting tables)
- deal_uuid columns on all three lineage tables (the column
DocumentLineageCard joins on — drop it anywhere and the card
silently returns partial data)
- back-pointer FKs (converted_contract_id, source_contract_id,
source_quote_id) — the exact columns that triggered the
Postgres FK-ordering bug fixed earlier in this PR
- Storno discriminator (kind, cancels_invoice_id, replaces_
invoice_id) per feedback_storno_filter_everywhere
- event time columns from migration 137
A full quote→contract→invoice lineage walk is deferred — quote
service's nextQuoteNumber() opens an inner transaction from inside
the createQuote outer transaction, which deadlocks SQLite's default
1-connection pool. Postgres dev DBs never see it. Either fix the
service to thread trx through, or run lineage tests against a real
Postgres in CI (mirror schema-drift.yml). Filed as separate work.
Two upstream tests regressed because the CRM PR added expected behavior
they didn't anticipate:
- galleryOgService.shareImage.test.js: formatEventDate is now async and
routes through utils/dateFormatter so the OG card respects the admin's
general_date_format setting (per feedback_respect_general_format_settings).
That adds a third db('app_settings') call on every buildOgMetadata path.
Mock the formatter module directly — the format itself is irrelevant
to the cover-vs-logo contract this file pins.
- customerAccountsService.test.js: createInvitation now allows a duplicate
email when the existing row is PASSIVE (password_hash IS NULL) — that's
the "promote passive customer to portal" path. The active-customer
rejection mock now has to set password_hash so the guard fires.
Both are test-only changes; no service code touched.
quotes.converted_contract_id and invoices.source_contract_id were
declared with inline FKs to contracts(id), but contracts is created
later in the same migration. SQLite accepted the forward reference;
Postgres rejected it ("relation \"contracts\" does not exist"), which
broke the Schema drift (#530) workflow and any fresh Postgres install.
Same pattern as events.hero_photo_id → photos.id in db.js: declare the
column without a constraint, then add the FK in a separate alterTable
after both sides exist. Wrapped in try/catch so re-runs against a DB
that already has the constraint are a no-op.
Verified locally against the #530 recovery scenario (initializeDatabase
then migrate:safe) and the fresh-install path: both converge cleanly,
both FKs land on the expected tables.
POST /v1/events was a strict subset of the admin create path: it did not
accept color_theme on the body, and it skipped the event_feedback_settings
insert that adminEvents.js does. Two visible bugs followed.
1. Editing an API-created event in the admin UI snapped the theme picker
to GALLERY_THEME_PRESETS.default (EventDetailsPage.tsx falls through to
the default preset when event.color_theme is falsy), and saving wrote
that default back. Inherited themes were silently clobbered.
2. The "Enable Guest Feedback by default" admin setting (#520) did not
apply to API-created events. With no event_feedback_settings row the
gallery UI reads feedback as off, regardless of
event_default_feedback_enabled.
Fix mirrors the admin path:
- color_theme accepted on the request body (optional, persisted as-is —
preset name or JSON-encoded ThemeConfig, same shape adminEvents
stores).
- feedback_enabled accepted on the request body; when omitted, falls
back to the event_default_feedback_enabled global setting (same
behaviour adminEvents.js:511-520 implements via readBooleanSetting).
- event_feedback_settings row inserted when feedback resolves to true,
using the same sub-flag defaults as the admin form (everything on
except require_name_email).
OpenAPI JSDoc updated so docs.picpeak.app picks up the new fields.
Tests cover all four scenarios — explicit color_theme persisted, JSON
theme persisted verbatim, explicit feedback_enabled creates the row,
omitted feedback_enabled honours the global setting, and a validator
regression for non-boolean feedback_enabled.
Refined from the original #530 framing after a dry-run uncovered that the
"bootstrap vs migration chain" diff produces mostly noise — most of the
~200 lines of difference are expected (migrations add new tables and
columns over time). initializeDatabase() isn't a parallel path that
diverges from migrations; it's invoked by migration 001 itself, so every
normal install/upgrade runs both.
The genuine drift hazard surfaced during the dry-run: a DB with the
modern bootstrap tables but an empty `migrations` table (which happens
when a backup was restored that lost the migrations table, or someone
invoked initializeDatabase() outside the runner, or the DB was moved
between systems without copying the migrations row) fails to upgrade.
Failure mode:
1. detectExistingSchema sees the bootstrap tables + empty migrations,
treats it as an "existing deployment".
2. Runs the legacy chain first.
3. legacy/008 renames email_templates.subject → subject_en.
4. core/029 (later in the chain) inserts email templates referencing
the pre-rename `subject` column.
5. Postgres rejects: column "subject" doesn't exist; subject_en is
NOT NULL with no default.
Fresh installs avoid this because they only run core/* (and core/059
handles the rename AFTER core/029 has inserted). Real legacy upgrades
avoid it because their migrations table already records legacy/008–028
as applied historically.
Fix in detectExistingSchema:
- Detect the modern bootstrap fingerprint (photo_categories + cms_pages
both present, which initializeDatabase produces as part of the
consolidated post-004-era bootstrap).
- When matched, enumerate every file in migrations/legacy/ and mark
each as applied. This puts the recovery state on the same code path
fresh installs use — only core migrations run, in core order.
- Real legacy upgrades that already have entries in the migrations
table hit no-op markings (markMigrationAsApplied skips duplicates),
so their behaviour is unchanged.
New CI workflow (`.github/workflows/schema-drift.yml`):
- Boots fresh postgres.
- Seeds via `node -e \"require('./src/database/db').initializeDatabase()\"`
— reproduces the recovery state in one line.
- Runs `npm run migrate:safe`.
- Asserts: precondition (bootstrap fingerprint + empty migrations
table), migrate:safe exits 0, final schema has ≥40 tables (soft floor,
not exact pin so future migrations don't force workflow edits),
legacy migrations marked applied (confirms the fingerprint check
actually fired vs. the chain silently bailing).
- Triggers only on PRs that touch backend/migrations/**,
src/database/db.js, knexfile.js, or this workflow.
Manually verified end-to-end before this commit:
Before fix: migrate:safe dies at core/029 with NOT NULL violation
on email_templates.subject_en (17/48 tables present).
After fix: 82 migrations applied + 27 marked applied = 109 total,
final state has all 48 tables matching fresh-install.
Issue body in #530 has been updated to match this refined scope.
Refs: #530, #484, #519
Folds all three follow-up items tracked in #525 into one commit:
1. Mirror PR #500's category scoping on adminPhotos.js. The admin
upload route at adminPhotos.js:231 still accepted any category_id
without event scoping — quietly less strict than the public v1
API after #500 landed. Same one-liner fix (event_id OR is_global)
with a matching 400 response shape so admin + v1 stay consistent.
2. Extract a shared slugify() in backend/src/utils/slug.js with the
NFD-strip-combining-marks fix from #502, and route 5 callers
through it:
- adminEvents.js (event-name slug)
- events.js (event-create slug)
- v1/events.js (replaces local slugify helper)
- adminArchives.js (archive→category slug)
For pure-ASCII input the output is byte-identical to each old
inline pipeline, so existing slugs in the DB keep round-tripping
cleanly via lookup. Accented inputs now transliterate (Família
→ familia) instead of dropping the diacritic (Família → f-mlia).
adminCategories.js stays with its own pipeline (underscores-as-
word-chars semantics differ from the events-style transform —
changing would silently shift wedding_party → wedding-party on
new inserts). xmpGenerator.sanitizeKeyword stays unchanged for
the same compat-cautious reason.
3. Cover the v1 upload happy path. Existing test only exercised the
400-out-of-scope branch. Add two happy-path cases that stub
sharp / generateThumbnail / storage.putFromFile and pin the
response shape (id, category_id, type, etc.) plus the collage-
slug → type='collage' flip. Temp file recreated in beforeEach
because the handler unlinks it on success.
Tests:
- New slug.test.js: 22 cases pinning ASCII parity with the legacy
pipeline (so the refactor is provably non-breaking for existing
data) and the corrected accent handling across de/es/fr/nl/pt
inputs, plus CJK and edge-case behaviour.
- events.category.test.js: 4 tests total (2 existing + 2 new happy
path).
- galleryOgService.shareImage.test.js: 11 (3 added in #521 + 8 pre-
existing) still pass.
37 tests pass across the three touched files.
Refs: #525, follows up #500 and #502
@Rekoo-PS reported that gallery URLs sent via the WhatsApp Business
API render an unbranded "PicPeak - Photo Sharing Platform" preview
even though manual link sends from the WhatsApp app pick up the
per-event rich preview correctly. Two root causes, two fixes:
1. WhatsApp Business and 3rd-party preview services (Twilio,
LinkPreview.net, etc.) don't always crawl with the recognisable
"WhatsApp/X.Y.Z" UA we matched in nginx + galleryOgService.
Extend the regex (both copies) to also catch WhatsAppBot, wa-bot,
LinkPreview, and Slack-ImgProxy.
2. Even with broader UA coverage, some senders cache metadata with
no UA at all and fetch the static SPA shell. That shell's
<title> was hard-coded to "PicPeak - Photo Sharing Platform" —
embarrassingly generic for any self-hosted brand. Switch to
Vite's %VITE_DEFAULT_TITLE% / %VITE_DEFAULT_DESCRIPTION% HTML
substitution so self-hosters can bake their brand into the
fallback at build time. Defaults stay "PicPeak" so the upstream
image doesn't change behaviour for anyone.
The per-event rich preview path (handleGalleryOgRequest, fired on
matched crawler UAs) is unchanged — this only improves the fallback
for unrecognised UAs and for the SPA-shell title that humans see in
their browser tab.
Adds a vite.config plugin to provide the defaults when env vars
aren't set, so unsubstituted "%VITE_..." literals never reach the
built HTML. Adds .env.example entries explaining the override.
Tests: extend galleryOgService.shareImage.test.js with an
isSocialCrawler suite that pins every documented UA (incl. the new
ones) plus three browser UAs (negative) and null/empty edge cases.
Verified locally: `vite build` with VITE_DEFAULT_TITLE="MyBrand"
produces <title>MyBrand</title> + og:title="MyBrand"; without the
env var falls back to "PicPeak".
Refs: #521
@Rekoo-PS asked for an admin-level switch so new events can have Guest
Feedback enabled out of the box instead of toggling it on every time.
Mirrors the existing event_default_require_password pattern (#317) —
same shape end-to-end, same set of five files.
- publicSettings.js: whitelist + expose event_default_feedback_enabled
(defaults to false to match the prior hard-coded form default; no
behaviour change for existing installs until an admin flips it).
- adminEvents.js: rename `feedback_enabled = false` destructure to
`feedback_enabled: feedbackEnabledInput` so we can distinguish
"omitted" from "explicit false", then resolve the default from the
setting only when the caller omitted it — identical to the
require_password handling a few lines above.
- Frontend EventSettings type + state + loader: new boolean,
default false.
- EventsTab: toggle UI right under "Require password by default".
- CreateEventPage: one-shot useEffect that seeds
feedback_settings.feedback_enabled from the public setting on first
load (mirrors the require_password seed effect right above it).
Sub-toggles (likes / ratings / comments) keep their hard-coded
true defaults so flipping the master setting immediately gives
sensible behaviour without a second admin setting to manage.
Refs: #520
Unit test for the v1 upload route's category lookup, requested in
the PR review. Mocks db (chainable, mirroring src/routes/__tests__/
adminAuth.test.js) plus apiTokenAuth/requireApiScope (pass-through)
and multer (stub req.file). Two cases:
1. The scoping clause: the andWhere callback applied to a knex
builder spy produces .where({event_id: <event.id>}).orWhere(
'is_global', true) — exactly the contract the reviewer asked
for, exercising the OR-clause rather than just asserting the
callback was passed.
2. Null lookup result yields 400 with "Unknown or out-of-scope
category_id <N>".
No v1 jest scaffolding existed before, but the project-wide harness
(backend/jest.config.js + jest.setup.js) already covers the new
file via testMatch '**/__tests__/**/*.test.js'. Happy-path tests
deferred — would require stubbing fs/sharp/imageProcessor/share
linkService and several more db chains, which the reviewer was
willing to accept as a separate follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR review pointed out the original lookup
db('photo_categories').where({ id: parsedCategoryId }).first()
accepted any category id — including one that belongs to a different
event. photo_categories carries both event_id (per-event) and is_global
(see backend/migrations/legacy/004_add_categories_and_cms.js); the v1
upload route should require either match.
Not a privilege issue (apiTokenAuth.js inherits the admin's powers, no
per-event scoping), but it lets a misconfigured uploader silently file
photos under a category the target event doesn't own — and the 201 echo
includes a category_id that makes no semantic sense.
Tighten to:
.where({ id: parsedCategoryId })
.andWhere(function () {
this.where({ event_id: event.id }).orWhere('is_global', true);
})
…and update the 400 message to "Unknown or out-of-scope category_id N".
OpenAPI description already documents the intended scope.
Tests deferred to a follow-up; v1 has no jest harness today, see PR
discussion.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The v1 photo upload endpoint previously ignored any caller-supplied
category and inserted photos with category_id=NULL. That meant
programmatic uploads via API tokens (e.g. a photobox sidecar) landed
in picpeak as uncategorized, forcing operators to bulk-assign category
in the admin UI after each event.
Mirror the adminPhotos.js category-handling logic on v1:
- Read optional `category_id` from the multipart form body.
- Reject unknown ids with 400 (with the id in the error) so callers
fail fast on misconfigured envs instead of silently uncategorized
uploads.
- Set photos.category_id on insert.
- Flip photos.type to 'collage' when the category's slug is
collage/collages, matching adminPhotos.
Backwards-compatible: omitting category_id keeps the prior behavior
(insert with NULL category, type='individual'). OpenAPI spec + 201
response body updated to include the new field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The fresh-install restart loop reported by @MrGabri (and confirmed by
@AloePacci with the user:0:0 workaround) had a clear root cause:
- Dockerfile pinned USER nodejs (UID 1001) before the entrypoint
ran, so the existing chown branch in init-production.sh:13 was
dead code.
- wait-for-db.sh (the actual entrypoint, not init-production.sh)
silently swallowed mkdir/EACCES on bind mounts with || true,
then a downstream migration error surfaced as the visible failure.
- Net effect on a typical Linux host where the bind-mount dir is
owned by UID 1000: container can't write, exits non-zero,
restarts forever with no clear error.
Switch to the standard Docker drop-privileges pattern:
1. Install su-exec, drop `USER nodejs` from the Dockerfile —
container now starts as root.
2. wait-for-db.sh: if running as root, chown /app/storage,
/app/data, /app/logs to nodejs and re-exec self via
su-exec nodejs:nodejs. App still ends up running as UID 1001.
3. Preflight check for non-root invocations (compose `user:`
overrides): verify the bind mounts are actually writable
before continuing. If not, exit 1 immediately with an
actionable error pointing at the docs — no more silent
restart loops.
Also:
- Delete backend/init-production.sh. It was an orphan — no caller
in the Dockerfile, compose, or anywhere else. Its chown logic
looked authoritative enough that @MrGabri ran it manually trying
to debug, which is what finally surfaced the EACCES.
- docker-compose.yml: drop user: + PUID/PGID env. The pattern-B
UID-matching workaround they implemented is obsolete now that
pattern A (root-then-drop) is in place.
- .env.example + README: drop PUID/PGID documentation.
- Add fresh-install smoke test workflow. Boots backend + postgres
against bind mounts owned by UID 1000 (the GitHub runner UID,
and the common-mismatch case on Linux hosts) and verifies:
+ container reaches healthy without restart-looping
+ chown happened (dirs now owned by 1001 inside the container)
+ node runs as nodejs, not root (su-exec drop worked)
+ /health returns status:ok
+ with --user 5005:5005 + unwritable mounts, preflight exits
loud with the expected error string
Verified locally end-to-end against a fresh Postgres + UID-501-owned
bind mount: backend reaches healthy in ~20s, chown applied, node
runs as nodejs, no restart loop. Docs in picpeak-docs cover the new
behavior + a Troubleshooting section for the install-path bugs
fixed in #484/#494/#511/#488.
Refs: #484
Luca
paul