pg_dump emits setval() statements for SERIAL/IDENTITY columns, but
they don't always land cleanly: --clean ordering, knex pool sequence
caching, rows inserted mid-restore (the pre-restore safety backup
writes a database_backup_runs row before DROP), etc. Net result on
Ralf's install after a successful restore:
- "A record with this value already exists" on every CRUD action
- duplicate key value violates unique constraint
"database_backup_runs_pkey" on the next Run Backup Now
Same root cause: every SERIAL column's sequence was pointing at or
below MAX(id), so the next INSERT collided.
Fix: append a DO block after the psql restore that walks pg_class +
pg_attribute and setval()s every public-schema sequence to
GREATEST(MAX(<col>), 1). Cheap (a few ms even on large schemas),
safe (read-only on row data), idempotent — re-running it just
re-asserts the same values.
Seventh latent PG-restore bug discovered on Ralf's install tonight.
Manual hand-fix worked; this commit makes the fix automatic for
every future restore.
PostgreSQL refuses DROP DATABASE while any session is connected:
ERROR: database "picpeak_prod" is being accessed by other users
DETAIL: There are 6 other sessions using the database.
The backend's own knex pool holds 5-25 active connections to the
target DB. So even after closing the request that initiated the
restore, the pool keeps the DB busy and the DROP statement fails.
Three-layered cure, all in the restore service's PG branch:
1. Call `db.destroy()` first to close the in-process knex pool so
we don't fight ourselves. Knex will lazily re-open on the next
query via db.js's retry logic, so this is safe to do mid-restore.
2. SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE
datname=<target> AND pid<>pg_backend_pid() — evicts any sessions
from other processes (other server replicas, leftover idle
transactions, things our own pool destroy missed).
3. DROP DATABASE IF EXISTS "<target>" WITH (FORCE) — PG13+ kills
remaining connections atomically with the DROP. Falls back to
plain DROP on older Postgres where WITH (FORCE) is a syntax error.
Surfaced as the FIFTH latent bug in the restore path tonight: the
DROP DATABASE statement always assumed a quiescent destination, but
the live backend keeps the destination busy at all times. Every
previous PG install of picpeak that ever tried Restore would have
hit this — meaning the disaster-recovery feature has shipped broken
for a long time without anyone exercising it end-to-end.
`psql` with no -d connects to a database whose name matches the
connecting user. On installs where the user's home DB doesn't exist
(common pattern: DB_USER=picpeak, DB_NAME=picpeak_prod, no `picpeak`
DB), the restore's DROP DATABASE / CREATE DATABASE statements failed
with:
FATAL: database "picpeak" does not exist
even though the target DB (picpeak_prod) was alive and connectable.
And of course you can't connect to the target DB itself for DROP —
PostgreSQL refuses while a connection is open to it.
Fix: explicitly connect to `postgres` (the maintenance DB every PG
cluster ships with) for the DROP/CREATE statements. Override via
DB_CHECK_DB env var if the `postgres` DB is restricted to superusers
on the cluster — matches the pattern wait-for-db.sh already exposes.
Also quote the database name in the SQL so installs whose DB has
unusual characters (numbers, hyphens) don't break the statement.
Surfaced during Ralf's end-to-end restore validation — yet another
"never been tested on a real PG install" latent bug exposed by the
Stage A inline-dump path actually being able to produce a restorable
manifest for the first time on his install.
Two changes that close the disaster-recovery loop the Stage A-B-C
backup-hardening plan opened:
1. Resolve 'local' source to backup_destination_path
The wizard passes options.source = 'local' (the SOURCE TYPE
string). The old code assigned that verbatim to localBackupPath
and every downstream path.join() ended up with junk like
'local/database/<file>.sql.gz'. Fixed by looking up
backup_destination_path from app_settings when source='local',
plus a layered candidate fallback in performDatabaseRestore so
absolute paths in manifests are honoured first.
2. Auto-rollback on ANY failure during restore
Previously rollback only fired when post-restore VERIFICATION
failed (inside the try block). Anything that threw earlier —
path bugs, pg_restore failure, file copy errors — left the
destination half-clobbered with no automatic recovery. Now the
catch block always invokes attemptRollback if a pre-restore
backup exists, and persists rollback status in
was_rollback_attempted + an enriched error_message so the admin
can tell at a glance whether the destination is safe to retry
on top of or needs manual inspection first.
Surfaced during Ralf's validation of the end-to-end backup +
restore cycle (`docker compose down -v` then restore from disk).
Every prior failed attempt left stray PDFs behind that the next
attempt had to navigate around — exactly the "every failure makes
the next worse" pattern this fix kills.
Two stacked bugs in the disaster-recovery path:
1. The wizard passes `options.source = 'local'` (the source TYPE
string) and the service assigned it verbatim to `localBackupPath`.
Every downstream `path.join(localBackupPath, ...)` ended up with
junk like `local/database/<file>.sql.gz` and `local/events/...`.
2. performDatabaseRestore reconstructed the dump path from the
manifest by basename-only:
path.join(backupPath, 'database', path.basename(dbBackupFile))
discarding the absolute path the manifest actually recorded.
Cure:
- At the entry point, if `options.source === 'local'`, look up
`backup_destination_path` from app_settings and use that as the
local root. Honour s3:// downloads via the existing branch.
- In performDatabaseRestore, try the manifest's absolute path
first, then `localRoot + manifest_value`, then the legacy
`localRoot + 'database' + basename` reconstruct as a final
fallback. First hit wins; error message lists every candidate
so future failures are diagnosable.
Surfaced during Ralf's end-to-end validation of the Stage A-B-C
backup-hardening plan — restored fresh after `down -v`, the wizard
failed silently with `Database backup file not found: local/database/...`
even though the dump existed at the path the manifest recorded.
With this fix, the same destruction-and-recovery sequence completes.
The Restore wizard's "Choose Backup to Restore" list was driven only
by the backup_runs table. After `docker compose down -v` (the disaster
this whole hardening effort is designed to recover from), the DB is
empty and the wizard shows "No backups found in selected source" —
exactly when it's needed most. The manifest JSONs are still on disk;
the wizard just can't see them.
Adds disk-first discovery:
- Walks backup_destination_path AND backup_manifest_path (manifests
can live in a sibling directory under the canonical
<root>/manifests/backup-manifest-<id>.json layout). Depth-limited
recursion (3 levels) so the scan doesn't enumerate the photo tree.
- Matches backup-manifest-*.json|yaml AND legacy bare manifest.json.
- Parses each manifest for real metadata (timestamp, size, file
count, database.backup_file presence) instead of showing the
admin opaque filenames.
- Layers in surviving backup_runs rows, deduping by manifest_id.
Applied to both GET /available-backups (legacy) and POST /list-backups
(the one the frontend actually calls). Same helper, two call sites.
Side benefit: each returned row now carries `databaseIncluded` — so a
future Restore UI iteration can show a "this backup has no DB dump"
warning before the admin picks a files-only backup. Exactly the
surface that would have caught Ralf's original four files-only
manifests if it had existed.
`BackupHistory.jsx` opened `/admin/backup/download/<id>` via window.open,
which goes to the React SPA's router — no matching route, so it
rendered the "Page Not Found" screen.
The actual download endpoint lives at `/api/admin/backup/download/:id`
on the backend (adminBackup.js:685). Cookie-based admin auth already
supports the implicit cookie sent by window.open, so the URL prefix
was the only thing missing.
Predates today's backup-hardening work — the bug has existed since
this download button shipped. Surfaced now because Ralf finally has a
completed backup to try downloading after the Stage A inline-dump
guard started working.
pg_dump rejects `--single-transaction` — it's a pg_restore / psql flag,
never a pg_dump one. Triggered as soon as the inline-dump path landed
on Ralf's install:
pg_dump: unrecognized option: single-transaction
pg_dump: hint: Try "pg_dump --help" for more information.
pg_dump already wraps the entire export in a single REPEATABLE READ
snapshot automatically (since Postgres 9.x), so the original intent —
consistent snapshot of the live DB — is preserved by removing the
flag. Same "latent until Stage A wired it in" pattern as the three
prior bugs this rollout has surfaced (PG insert destructure → bind-
mount EACCES → Node 22 stdio strict mode → this).
spawnToFile and spawnFromFile passed an unopened WriteStream/ReadStream
directly as a stdio entry to child_process.spawn. Older Node versions
auto-extracted .fd; Node 22 throws synchronously:
The argument 'stdio' is invalid.
Received WriteStream { fd: null, path: '/backup/database/...sql', ... }
Bug bit Ralf's install once today's `bugfix/crm-backup` image landed —
Node 22 came with that image, and Stage A's inline-dump path is the
first caller of spawnToFile on this install. Latent on the previous
image (Node 20); fatal on this one. restoreService's pre-restore
safety snapshot uses the same helper and would have hit it next time
a restore ran.
Cure: stdio: ['ignore', 'pipe', 'pipe'] (and ['pipe', 'pipe', 'pipe']
for spawnFromFile) + manual pipe of child.stdout/stdin through the
file stream. Works on every Node version. Also wires the WriteStream's
'error' event to the promise via settleReject so a future EACCES /
ENOSPC reaches the caller's try/catch instead of becoming a process-
fatal unhandled error event — closing the same "Stage A guard
bypassed" hole noted in the spawned follow-up task.
Side benefit: outStream.end() now awaits flush before resolving, so
fast pg_dump runs can no longer produce a truncated dump.
databaseBackupService.backup() did `const [runId] = await db(...).insert({...})`
without a .returning() — works on SQLite (knex returns [lastInsertId]) but
throws "(intermediate value) is not iterable" on Postgres (knex returns
a non-iterable shape).
Bug was latent until Stage A of the backup-hardening plan wired this
method into the "Run Backup Now" inline-dump path. Before Stage A only
the scheduled cron + the dedicated admin-DB-backup page called it, and
Ralf's install had never exercised either — so the inline-dump default
landing in production was the first time the destructure ran on his PG.
Cure: same explicit .returning('id') + dual-shape coalesce pattern that
backupService.js uses for its own backup_runs insert (line 949).
Two more sibling files have the same anti-pattern (userManagementService,
customerAccountsService — invitation flows) and will bite under the
same conditions; spawned a follow-up task to fix them in a separate PR.
#574 follow-up — @blazmaric flagged that once an admin user is
deactivated, the UI loses every affordance to manage that record.
The deactivate button hides (rightly — they're already deactivated)
but nothing replaces it, leaving the row stranded in the list with
no path to either restore access or permanently remove it.
## Backend
New on `userManagementService`:
- **`activateAdminUser(id, activatedById)`** — symmetric to
`deactivateAdminUser`. Flips `is_active` back to true, logs
`admin_user_activated` activity. Idempotent: already-active target
short-circuits without bumping `updated_at`. No "can't activate
yourself" guard needed (actor is by definition already active).
- **`deleteAdminUser(id, deletedById)`** — hard-deletes the row.
Same self-action and last-super-admin guards as deactivate.
Last-super-admin guard counts ACTIVE super admins excluding the
target — so an already-deactivated super_admin can still be
deleted when an active super_admin remains. FK ON DELETE rules
in core migrations handle the cascade: SET NULL on
`created_by_admin_id` everywhere (events, photos, quotes,
invoices, contracts, customer_accounts, …); CASCADE on the
user's own `api_tokens` + their pending admin / customer
invitations.
New routes on `adminUsers.js`:
- `POST /api/admin/users/:id/activate` — `users.delete` permission
(same tier as deactivate; reverting deactivation is the same
scope of action as performing it).
- `DELETE /api/admin/users/:id` — `users.delete`.
## Frontend
`UserManagementPage.tsx`:
- New mutation hooks: `activateUserMutation`, `deleteUserMutation`.
- The row's action cell now branches on `user.isActive`: active
users see Edit + Deactivate (unchanged); deactivated users see
Edit + Reactivate (`UserCheck` icon, green hover) + Delete
(`Trash2` icon, red hover).
- The shared `ConfirmDialog` handles all four action types
(deactivate / activate / delete / cancelInvitation) via per-type
title / message / confirmText / variant lookup.
`userManagement.service.ts`:
- New `activateUser(id)` and `deleteUser(id)` methods mirroring the
existing `deactivateUser` shape.
i18n keys are added with English fallbacks via `t(key, fallback)`
so the page works on every locale without a missing-translation
warning. Native translations can be filled in via a follow-up.
## Test plan
- [x] 8 new service tests pin: activate happy-path, idempotency on
already-active, NotFoundError on missing target, activity log
emitted, delete self-refusal, last-super-admin guard for both
active and already-deactivated super_admin targets, hard-delete
success, delete activity log.
- [x] Frontend type-check clean.
- [x] Frontend lint clean for the changed files.
- [x] Backend lint clean.
- [ ] Manual: deactivate a user → row now shows Reactivate + Delete
→ reactivate → user can log in again. Then deactivate again →
delete → row vanishes, pending tokens for that user invalidated.
Closes the UX gap blazmaric called out in
https://github.com/the-luap/picpeak/pull/579#issuecomment-... .
Closes#570.
PR #555 shipped the CRM module with strong service-layer coverage
but no HTTP-layer tests. This adds Supertest-based route coverage
across the externally-reachable public routes (P0) and an auth-gate
sweep of every CRM admin route (P1+P2).
## What's covered
### P0 — Public routes (49% of new tests)
The three public routes are the security-sensitive surface — any IP
with the raw token from a leaked email can hit them. Tests pin the
publicTokenGuards.loadActionToken contract end-to-end:
- **publicQuotes** (8 tests) — GET load + POST respond: 404 unknown,
400 malformed, 410 expired, 200 valid w/ sanitised payload (no
customer_account_id / created_by_admin_id leakage), 429 after 20
bad attempts (IP lockout), 400 invalid action.
- **publicContracts** (10 tests) — GET load + POST sign + POST
upload-signed-pdf + GET pdf: same guard outcomes per endpoint,
plus the pre-multer token check (malformed token rejected before
multer reads the body — prevents the disk-spam attack the
preMulterTokenGuard was added for).
- **publicPaymentCheck** (6 tests) — different shape (no
loadActionToken; service does its own validation): validator gate
on token shape, all 4 canonical actions pass through the
validator, negative amountMinor rejected.
The NULL-expires_at defensive branch in loadActionToken is
documented but not tested here — current schema declares
quote/contract_action_tokens.expires_at NOT NULL, so the branch is
unreachable at the route level. Worth a direct unit test on
loadActionToken if anyone wants to cover it.
### P1 + P2 — Admin routes (51% of new tests, 25 cases)
One consolidated `adminCrmAuth.test.js` file rather than nine
per-route files — the auth-gate contract is identical for every CRM
admin route, so a parametrised `describe.each` is more efficient
and lands the same coverage:
Per route (adminQuotes, adminContracts, adminInvoices, adminCalendar,
adminDeals, adminTaxReport, adminBusinessProfile):
- 401 without Authorization header (adminAuth gate)
- 401 with invalid JWT signature (adminAuth signature check)
- 2xx with super-admin token + CRM feature flags on (permission +
feature-flag gates both pass)
Plus 4 tests for the CRM additions in adminCustomers
(hour-entries / bill / trigger-monthly-bill) — those endpoints
are mixed in with pre-existing customer routes, so they get
explicit coverage rather than bulk via the parametrised sweep.
## Harness extensions to integration/helpers/crmDb.js
Three new helpers (one place for any future route test to find):
- `mintAdminToken(adminId, opts)` — JWT signed with the test
JWT_SECRET, shape matches what adminAuth expects.
- `createPublicToken(db, tableName, opts)` — insert a row into
quote/contract_action_tokens with controllable expires_at /
used_at / token. Note: Date values are explicitly ISO-stringified
before insert — bare Date objects round-tripped inconsistently
through knex+SQLite, sometimes via .toString() → literal
`"[object Object]"` which parsed back to NaN and silently defeated
the expiry guard. Caught it in test bring-up.
- `buildRouteApp(mount, router)` — minimal Express app (json + cookies)
with a catch-all error handler that mirrors middleware/errorHandler
(uses err.statusCode, not err.status — getting that wrong silently
maps every 4xx to 500 in tests).
- `assignAdminRole(db, adminId, roleName)` — promotes a seedMinimal
admin into super_admin (or any seeded role) for happy-path tests.
## Out of scope (follow-up)
Deeper integration tests for the document mint/send paths
(adminQuotes.send → PDF persisted + token minted + email queued;
adminInvoices.Storno → new row with shared deal_uuid + original
cancelled; adminContracts.countersign → integrity_hash computed)
are deferred. The service-layer behind those is already covered by
the existing __tests__/services/ suites — this PR pins the
HTTP-layer contract, which is what #570 actually asked for.
## Counts
- 4 new test files, 49 tests total
- ~860 LOC of test code + ~85 LOC of new harness in crmDb.js
- All tests pass in <2.5s (no real network, no real disk except the
per-test tmpdir, no email sending)
Stage C of the three-stage backup-hardening plan (Stage A: inline
DB dump + fail-loud landed in 7fdf01a; Stage B: config-driven walker
in 302fc6b). Answers the "what would I lose if I clicked Run Backup
Now right now?" question that Stage B made possible to answer.
Backend:
- new backupCoverageService.js: per-path coverage classification,
drift detection (top-level subdirs not in backup_paths and not
in the backups/tmp allow-list), DB-dump mode + staleness block
- new GET /api/admin/system-health/backup-coverage route, same
auth + settings.view permission as /backup-integrity
- 7 integration scenarios pinning the classifier behaviour
Frontend:
- new BackupCoverageCard with auto-fetch (cheap; no recursion)
- new Coverage tab on BackupManagement next to Integrity
- en + de i18n; other locales fall back to en keys until a native
speaker reviews
Verification:
- 26/26 backup integration tests pass (Stage A 5 + Stage B 7 +
Stage C 7 + adminBackupIntegrity 4 + businessDocs 3)
- frontend build clean
- 4 pre-existing integration failures confirmed unrelated
Stage B of the three-stage backup-hardening plan (Stage A:
inline-DB-dump + fail-loud guard already landed). The file-backup
walker used to hard-code its subdirectory list inside
`getFilesToBackupInternal`, which is the same footgun that hid the
`business-docs` gap for ~6 months — a new feature drops artefacts
under STORAGE_PATH and the maintainer has to remember to edit the
walker.
Now driven by a `backup_paths` table:
- Migration 108 creates the table and seeds the 7 canonical
defaults (events/active, events/archived, thumbnails, previews,
heroes, uploads, business-docs). Seed data lives on the
migration as `DEFAULT_PATHS` so the boot self-heal can re-use it.
- `_backupPathsBoot.js` mirrors `_emailTemplateBoot.js`: on every
boot it diffs the canonical list against the current rows and
`INSERT ... ON CONFLICT DO NOTHING`s the missing ones. Keeps
admin edits intact, picks up new defaults shipped after the
install (Knex won't re-run migration 108). Wired into server.js
just before `startBackupService()`.
- Walker now calls `resolveBackupPaths(config)` which:
* reads `backup_paths WHERE include_in_default=true ORDER BY
display_order`
* falls back to a hard-coded `LEGACY_BACKUP_PATHS` if the
table is missing OR empty (defense in depth — never silently
scans nothing)
* gates each row by its `feature_flag` column (matches how
`backup_include_archived` already worked; data-driven now)
- Backward compatible: `getFilesToBackup(true|false)` still works
for legacy callers and the existing businessDocs test. New
callers should pass the full config object so feature gates
other than `backup_include_archived` evaluate correctly.
Tests:
- new: `backupService.configurableWalker.test.js` — 7 cases
covering canonical seed, toggling include_in_default, runtime
INSERT picked up without restart, feature_flag gating both on
and off, empty-table → LEGACY fallback, boolean backward compat
- all 15 backup-walker integration tests pass
(configurableWalker 7 + inlineDbDump 5 + businessDocs 3)
- frontend build clean
- 4 pre-existing integration failures (webhookDelivery, storage
backend, adminPhotos.reference, imageProcessor.storage) confirmed
unrelated via `git stash` baseline run
Stage C (CRM feature coverage audit + diagnostic UI) follows
in a separate commit.
The previous file-backup workflow only LOOKED UP an existing
database dump via getDatabaseBackupInfo() and silently shipped a
files-only manifest when none was found. Admins clicking "Run
Backup Now" (or relying on the schedule) got an apparent success
that omitted every customer / quote / invoice / contract / payment-
log row. The data-loss footgun was discovered 2026-05-29 when an
admin who'd been "backing up" for weeks via the UI lost the entire
CRM after a routine docker compose down -v — every produced
manifest had database: { backup_file: null, size: 0, tables: {} }.
New helper `ensureDatabaseDumpForBackup(config)` encapsulates:
1. Inline pg_dump (or SQLite copy) before the file scan, via
databaseBackupService.backup(). Result lands in
database_backup_runs and is picked up by the existing
getDatabaseBackupInfo lookup that writes the manifest.
2. Fail-loud guard: if no usable dump file is reachable (path
missing, 0 bytes, or never existed), throw — the existing
catch in runBackupInternal marks the backup_runs row failed
with the error_message and emails the admin if configured.
No more silent files-only manifests.
3. Opt-out: `backup_database_inline_dump = false` skips the
inline dump for admins who already run their own scheduled
`backup_database_schedule`. The fail-loud guard still
applies, so an opted-out install with no recent dump still
aborts loudly instead of producing a partial backup. Default
ON is encoded as "skip only when explicitly false" — undefined
(existing installs upgrading) falls through to the safe-
default ON branch.
The helper returns the verified `databaseInfo` so the manifest-build
step at runBackupInternal:917 reuses it instead of calling
getDatabaseBackupInfo a second time. S3/future destinations that
override `result.databaseInfo` are still respected (the existing
`result.databaseInfo ||` fallback shape stays put).
Test suite covers: default-on happy path, dump-throws-aborts-run,
opt-out + recent dump + proceeds, opt-out + no-dump + fail-loud,
opt-out + 0-byte dump + fail-loud. Mocks
databaseBackupService.backup so the tests don't depend on pg_dump
or sqlite3 CLI binaries being installed.
Stage A of three-stage backup hardening plan. Stage B (config-
driven walker) and Stage C (audit + diagnostic UI) follow in
separate commits.
The previous file-backup workflow only LOOKED UP an existing database
dump via getDatabaseBackupInfo() and silently shipped a files-only
manifest when none was found. Admins clicking "Run Backup Now" (or
relying on the schedule) got an apparent success that omitted every
customer / quote / invoice / contract / payment-log row. The
data-loss footgun was discovered 2026-05-29 when an admin who'd been
"backing up" for weeks via the UI lost the entire CRM after a routine
docker compose down -v — every produced manifest had database:
{ backup_file: null, size: 0, tables: {} }.
Changes to runBackupInternal:
1. Inline pg_dump (or SQLite copy) before the file scan, via
databaseBackupService.backup(). Result lands in
database_backup_runs and is picked up by the existing
getDatabaseBackupInfo lookup that writes the manifest.
2. Fail-loud guard after the dump step: if no usable dump file is
reachable (path missing, 0 bytes, or never existed), throw —
the existing catch block marks the backup_runs row failed with
the error_message and emails the admin if configured. No more
silent files-only manifests.
3. Opt-out: `backup_database_inline_dump = false` skips the inline
dump for admins who already run their own scheduled
`backup_database_schedule`. The fail-loud guard still applies,
so an opted-out install with no recent dump still aborts loudly
instead of producing a partial backup. Default ON is encoded
as "skip only when explicitly false" — undefined (existing
installs upgrading) falls through to the safe-default ON path.
Test suite covers: default-on happy path, dump-throws-aborts-run,
opt-out + recent dump + proceeds, opt-out + no-dump + fail-loud,
opt-out + 0-byte dump + fail-loud. Mocks
databaseBackupService.backup so the tests don't depend on pg_dump
or sqlite3 CLI binaries being installed.
Stage A of three-stage backup hardening plan. Stage B (config-driven
walker) and Stage C (audit + diagnostic UI) follow in separate
commits.
Closes#580.
Slovenian community contribution from @blazmaric (filed as an issue
with attached files rather than as a PR — files inlined here unchanged
except for the migration number).
## Changes
- **`frontend/src/i18n/locales/sl.json`** — full Slovenian UI
translations. Covers every top-level key present in `en.json` as
of pre-CRM beta. The new CRM-module keys (`bills`,
`businessProfile`, `calendar`, `contracts`, `crm`, `crmDev`,
`crmSettings`, `dealLineage`, `eventReminderOverride`,
`hoursLogging`) are not yet translated and will fall back to
English — same posture as FR / NL / PT / RU / ES currently have
for the CRM module (see PR #555 description).
- **`frontend/src/components/common/LanguageSelector.tsx`** — adds
`SLFlag` SVG component + registers `{ code: 'sl', name:
'Slovenščina', Flag: SLFlag }` in `SUPPORTED_LANGUAGES`. Frontend
i18n auto-discovers locale files via `import.meta.glob` so no
separate config registration is needed.
- **`backend/migrations/core/108_seed_sl_email_template_translations.js`** —
contribution-author's `107_*` filename renumbered to `108_` to
avoid collision with `107_crm_consolidated.js` that landed on beta
in the meantime. Idempotent insert via (template_id, language)
uniqueness check — re-runnable, never overwrites admin edits.
Covers 17 templates: admin invitation / password reset, archive
complete, backup completed / failed, customer gallery assigned,
customer invitation / password reset, database backup completed /
failed, expiration warning, gallery created / expired, restore
completed / failed, version update available / test.
- **`backend/src/services/emailProcessor.js`** — adds `.si → sl` to
the email-domain → language inference map, matching the pattern
for every other supported locale. A customer with `@example.si`
now gets Slovenian emails automatically without needing to set
their preferred_language explicitly.
## Out of scope (consistent with existing locales)
- CRM email templates (quote_sent, invoice_sent, contract_sent, etc.,
seeded at boot by `crmEmailTemplates.ensureCrmEmailTemplatesSeeded`)
will fall back to English for Slovenian customers — those seeders
only emit EN + DE rows today across every locale.
- CRM UI strings under the missing top-level keys listed above will
fall back to English.
Both gaps mirror the existing FR / NL / PT / RU / ES situation.
Resolves a conflict with the CRM merge (#555) that landed on beta
between when this branch was cut and now.
Two conflict regions in backend/src/routes/adminCustomers.js:
1. **Require block** — both branches added new requires after
customerAccountsService. Kept both: this branch's
emailNormalization import AND beta's customerHoursService +
invoiceService imports (the CRM merge added the hours-billing +
invoice-creation paths to this router).
2. **Edit-customer validators** — both branches changed the same set
of body() validators in the PUT /:id handler. This branch added
the IDENTITY_PRESERVING_NORMALIZE_EMAIL options arg to
normalizeEmail; beta changed every body() to optional({ nullable:
true }) so passive-customer records that store nulls for missing
profile fields don't reject on save. Kept both: the nullable
pattern from beta + the email-normalization options from this
branch. Preserved beta's explanatory comment about the nullable
choice.
Also patched one NEW normalizeEmail site the CRM merge introduced:
- backend/src/routes/adminCustomers.js:231 — POST /admin/customers
now exists (CRM-era customer-create endpoint). Same options arg
applied.
backend/src/routes/adminBusinessProfile.js has an isEmail() WITHOUT
normalizeEmail() on the issuer email — intentional (no normalization
means no risk of the Gmail dot-strip bug for that field), no change
needed.
All 18 normalizeEmail sites now pass IDENTITY_PRESERVING_NORMALIZE_EMAIL.
7/7 regression tests still pass. Lint clean on the merged file.
When PR #555 (CRM module) added pdfkit/swissqrbill/pdf-lib/qrcode to
backend/package.json, every dev with an already-built dev image hit
a MODULE_NOT_FOUND restart loop on the next pull. Root cause: the dev
compose bakes node_modules into the image while live-mounting src/
from disk — a dep added on disk isn't visible to the running container
until the image is rebuilt.
The symptom doesn't point at the cause, so this adds a short rebuild
note to the Local Development section of CONTRIBUTING.md. A
self-healing entrypoint (compare node_modules/.package-lock.json
vs /app/package-lock.json on boot, npm ci if they differ) would fix
this at the runtime layer too; tracked as a follow-up.
The fix shipped in 3ab3756 added /backup to the boot-time chown list.
That broke installs that don't bind-mount ./backup:/backup — the
single greedy `chown -R /a /b /c /backup` returned non-zero on any
individual failure, exiting the script and putting the backend into
a restart loop.
Reverting to the upstream-stable version. The original EACCES at
backup time is better fixed by admins pointing the backup destination
at a writable path via the admin UI (e.g. /app/storage/backups,
which the script already chowns) rather than baking a /backup
assumption into every install's boot path.
The docker-compose `./backup:/backup` mount was the only bind mount
not included in wait-for-db.sh's startup chown step. On a fresh
install (or any time the mount point is recreated), it stays
owned by root, and the nodejs (UID 1001) process running the
backup service gets EACCES when trying to mkdir under /backup.
Added /backup to both the chown list (root branch) and the
writable-check list (compose `user:` override branch), each guarded
by `[ -d /backup ]` so installs that don't use the bind mount —
native deployments, k8s with a different backup destination, etc. —
still boot cleanly.
Existing installs hit by this need a one-time host-side
sudo chown -R 1001:1001 <host-mount-for-/backup>
because the on-disk ownership won't fix itself; the script only
chowns at startup, and the directory was already created with
the wrong ownership by Docker's mount-point auto-creation. From
this commit onward, fresh installs are correct from the first
boot.
PR #555 shipped the CRM module on beta. The README's "Beta Features
(Use at your own risk)" table is the right place to signal that the
feature exists, is opt-in, and carries non-trivial legal / financial
caveats — readers landing on the README should not first discover the
CRM by enabling its feature flags and bumping into the seeded
example contract bodies without warning.
Adds one row to the Beta Features table linking to
docs.picpeak.app/features/crm where the full disclaimers,
sub-feature pages, and admin-settings reference live.
CRM is intentionally NOT added to the top-of-README "Key Features"
list — those are stable, production-ready features. Mixing the beta
CRM in there would undermine the clear stable/beta distinction.
Line 205 of databaseBackup.test.js reassigned `fs.unlink` directly
(`fs.unlink = jest.fn(...)`), which permanently mutated the global
fs.promises module. Every test running after this in the same jest
worker process inherited the no-op stub, including
integration/storageBackend.test.js — whose LocalFsStorage.delete()
silently became a no-op, making the subsequent exists() assertion
flip from false to true.
Confirmed by adding a diagnostic patch to LocalFsStorage.delete:
post-await fsp.unlink, fs.existsSync(abs) returned true. unlink had
resolved without throwing but the file was still there → the unlink
was a mock.
Fix: jest.spyOn(fs, 'unlink').mockResolvedValue(undefined) + a
matching mockRestore() at the end of the test. Behaviour is
identical inside this test; the original fs.unlink is restored
when the test finishes, so subsequent tests get real fs.unlink
again.
Pre-existing issue — has been latent on upstream/beta forever.
Only surfaces consistently when CI load shifts jest's worker
allocation such that databaseBackup and storageBackend land in
the same worker process. This PR's extra integration test files
made that allocation deterministic locally and frequent enough on
CI to fail reliably.
The 5-minute session-sweep interval at sessionTimeout.js:17 fired at
module-load time without .unref(), so every jest worker that
transitively required this module (server.js → middleware → most
of the route layer) kept the event loop alive forever. The worker
then got force-killed on shutdown, surfacing as the longstanding
"worker failed to exit gracefully" warning at the end of every CI
run on upstream/beta.
Under enough I/O / memory pressure on a CI runner, the force-kill
could land MID-test rather than after the suite finished, taking
out whatever else was running on that worker — most visibly
integration/storageBackend.test.js on PR #555's runs.
.unref() makes the timer not keep the loop alive on its own.
Production behaviour is unchanged: the timer still fires every
5 min as long as anything else is holding the loop open (the HTTP
server, always).
CI's SQLite returned `[N]` (plain int) from `.insert().returning('id')`
while local SQLite returned `[{ id: N }]` (object form). The brittle
`const [{ id }] = ...` destructure crashed on the int shape. Switched
to the unwrap pattern used by the existing crmDb test harness so the
suite runs on both PG and every SQLite/knex combo the project supports.
Frontend half of the diagnostic shipped in 4812fcd. Adds:
- BackupIntegrityCard component — runs the check on demand, surfaces
the five summary counters (total / verifiedOk / existsButNoHash /
missing / hashMismatches), and expands collapsible result tables
for missing files + hash mismatches. existsButNoHash is exposed as
a separate amber-toned bucket so admins can distinguish hash-
verified evidence from existence-only at a glance — the latter is
explicitly weaker in a legal dispute and the UI says so.
- "Integrity" tab on BackupManagement, alongside the existing
Dashboard / Configuration / History / Restore tabs. Card is
portable — when the System Health page (backlog item) lands it
can lift the component without changes.
- Post-restore CTA on the RestoreWizard success card (D2 follow-
through): "Verify document integrity now" button that switches
the parent tab to Integrity. The audit trail captured at sign /
issue time is worth nothing if the documents it refers to are
missing from the restored copy — verifier surfaces that drift
in one click before the admin trusts the restored state.
i18n strings added in EN + DE (per user_languages — only those two
are native; other locales fall back to the English defaults and
should be flagged for native-speaker review per
feedback_translation_flagging if anyone picks them up).
Diagnostic for the bug fixed in a9280ea — confirms every *_path
column on quotes / contracts / invoices points at a file that
actually exists on disk and (where a *_sha256 column is set) the
file's bytes still hash to the expected value. Read-only;
on-demand only; no scheduler.
Per the design decisions locked in this PR's design call:
D1 — on-demand only for v1; scheduling deferred until we have
runtime data on large installs
D2 — not auto-triggered after restore; surface a "verify
integrity now" CTA on the restore-completed screen instead
D3 — wet-upload contracts hash-verified same as system-rendered
(signed_pdf_sha256 is computed at upload time, no special
case needed in the verifier)
Coverage (single source of truth in backupIntegrityService.CHECKS):
quotes.pdf_path existence
contracts.pdf_path + pdf_sha256 existence + hash
contracts.signed_pdf_path + signed_pdf_sha256 existence + hash
contracts.signed_customer_signature_path existence (PNG/JPG, no hash)
contracts.signed_admin_signature_path existence (PNG/JPG, no hash)
invoices.pdf_path existence
invoices.imported_pdf_path existence (admin-uploaded scans)
Report shape buckets each row into verifiedOk / missing /
hashMismatches / existsButNoHash so callers can distinguish hash-
verified from existence-only — the latter is weaker evidence in
a legal dispute and the UI should reflect that.
Route GET /api/admin/system-health/backup-integrity accepts an
optional ?scope= CSV filter (quote | contract | contract-signature
| invoice). Unknown scope tokens are rejected with a 400 +
BACKUP_INTEGRITY_UNKNOWN_SCOPE code rather than silently scanning
everything.
Frontend half (BackupIntegrityCard on a System Health page) is
deferred until backlog #11 (System Health page) is scaffolded.
The endpoint is independently useful via curl in the meantime.
backupService.getFilesToBackupInternal() enumerated a fixed list of
storage subdirectories (events/active, events/archived, thumbnails,
previews, heroes, uploads) and silently omitted the entire
business-docs/ tree. Every CRM PDF artefact and signature image fell
outside the in-app scheduled backup — restoring the DB without the
PDFs would have left every *_path column on quotes/contracts/invoices
as a broken FK and lost forensic evidence (the customer signature
PNG/JPG drawn on the public signing page is referenced by
contracts.signed_customer_signature_path; the rendered contract PDF
is referenced by signed_pdf_path with a stored signed_pdf_sha256
that would have nothing to verify against; wet-uploaded contracts
and admin-imported historical invoices are irrecoverable by design
since no renderer can reproduce them).
Single new scanDirectory call after the existing uploads scan,
covering:
- business-docs/quote/<year>/*.pdf
- business-docs/contract/<year>/*.pdf
- business-docs/contract/signatures/<contract_id>/*.{png,jpg}
- business-docs/invoice/<year>/*.pdf
- business-docs/invoice-imports/<year>/*.pdf
- and incidentally business-docs/dev-test/ (managed by adminDev.js,
bounded to 7 newest files, harmless to back up)
Verified that no migration is needed: hasFileChanged returns
!existing || checksum mismatch, so the first backup after this lands
flags every business-docs/** file as new and copies it. Restore path
in restoreService.performFilesRestore uses fs.mkdir({ recursive:
true }) on path.dirname(targetPath), so business-docs subdirectories
are recreated automatically from manifest entries — no restore-side
code change required.
Integration test pins the contract so a future refactor cannot
silently drop business-docs again.
The shell-script backup at scripts/backup.sh already covered all of
this via blanket `tar -czf storage`; only the in-app service was
affected.
Closes#574.
Reporter (@blazmaric) identified the root cause cleanly:
express-validator's `.normalizeEmail()` applies provider-specific
canonicalization by default — Gmail dot-stripping, +tag stripping,
googlemail → gmail folding, etc. That's wrong for identity: PicPeak
uses email as a login identifier, so `[email protected]` getting
silently stored as `[email protected]` means the user can't log in
with the address they were invited with.
The bug existed at 17 call sites across the codebase (auth, admin user
create/update, customer create/update, event create/update on three
different routes, customer login, feedback submission). All of them
are identity-bearing — none had a legitimate reason to strip dots
for deduplication.
Fix: introduce one shared options object in `utils/emailNormalization`
disabling every provider-specific normalization
(gmail_remove_dots, gmail_remove_subaddress,
gmail_convert_googlemaildotcom, outlookdotcom_remove_subaddress,
yahoo_remove_subaddress, icloud_remove_subaddress). The only default
left enabled is `all_lowercase`, which is safe — local-parts are
case-insensitive in practice on every major provider, and lowercasing
keeps login lookup consistent.
Every call site updated to pass the shared options. 7 unit tests pin
the preserved-dots, preserved-subaddress, preserved-googlemail-domain,
and still-lowercase behaviours so a future refactor can't silently
regress.
## Migration note
Existing accounts whose emails were already stripped before this fix
remain with the stripped form in the DB. The fix takes effect for new
invitations going forward. If an admin re-invites an existing user
with the un-stripped address, that would create a duplicate account —
out of scope here; if it becomes a real problem we can add a
backward-compat login fallback (try lookup with dot-stripped form too)
as a separate change.
Closes#567.
The sidebar already had a "vX.Y.Z available" indicator (#566 made it a
link to that release's page) but there was no way to read the actual
changelog inline or to grab a copy-paste upgrade command. This adds
the modal the issue spec'd, layered on top of the existing
updateCheckService / environmentService backend infrastructure that
already shipped.
## Backend
- `updateCheckService.fetchAvailableVersions` now returns full release
objects (tag, name, body, publishedAt, htmlUrl) instead of just
version strings — body data is what the changelog modal renders.
`checkForUpdates` extracts the version strings for its existing
consumers; no API change visible to callers.
- New `getReleasesSince(currentVersion, channel)` returns the list of
releases strictly newer than current, filtered to the user's
channel. Reuses the same 1-hour cache as `checkForUpdates` so the
modal opening doesn't trigger an extra GitHub round-trip.
- New `GET /admin/system/updates/changelog` route in `adminSystem.js`,
same auth + UPDATE_CHECK_ENABLED gating as the existing
/updates and /updates/instructions endpoints.
- 4 unit tests (axios mocked) pin: strictly-newer filtering,
channel-scoped, empty array on GitHub fetch failure, empty array
when already on latest.
## Frontend
- New `UpdateAvailableModal.tsx` — opens from the sidebar chip. Two
sections:
1. **How to upgrade** — fetches /updates/instructions for the
environment-detected copy-paste command (Docker compose / git /
standalone). Copy-to-clipboard button per step.
2. **Release notes** — fetches /updates/changelog for every
version between current and latest in the user's channel.
Latest is auto-expanded; older releases are collapsed by
default (click to expand). Each release also has a "View on
GitHub" link to the canonical release page.
- Renders release body markdown through the existing safe
MarkdownContent component (marked + DOMPurify allowlist).
- New `updateDismissal.ts` helper — single localStorage key holds the
last-dismissed version. Chip stays hidden until a STRICTLY newer
version appears, using the same compare semantics as the backend
(stable > beta, higher beta > lower beta, semantic numeric on
major.minor.patch). 9 unit tests pin the rules.
- `VersionInfo.tsx` — chip is now a button that opens the modal
instead of an external link (the #566 link-to-release behaviour is
preserved on the modal's per-release "View on GitHub" affordance).
Dismissal triggers an immediate re-render so the chip disappears
without waiting for the next route change.
No new dependencies — uses `marked` + `DOMPurify` that were already
present in the bundle for the contract block renderer.
Luca