Stable twin of the same fix on main.
The dropdown rendered as `value="0"` and adminPhotos.js:1001 skips '0', so no
category condition was applied and the whole event came back. The branch that
does the work sits four lines below, keyed on the literal 'uncategorized' that
nothing was sending.
Silent by nature — a full list reads as 'nothing to narrow' rather than 'the
filter did not run' — which is why it survived this long.
The reporter in #1209 is on 3.46.4, so this is the branch that reaches them.
Tests both ends of the contract, since the bug was the pairing rather than
either half.
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Stable twin of #1204.
The capture-date backfill committed its result keyed on the row id alone. It
snapshots every candidate up front, then walks them one at a time reading
originals off S3 or a NAS mount — a pass that can run for many minutes.
replacePhoto, reachable from the replace_by_name upload path, swaps a NEW file
under an existing row and rewrites path/filename. A replacement landing inside
that window carries no date of its own, so captured_at was still NULL, the
whereNull guard passed, and the previous file's EXIF date was written onto the
new photo. Silent: nothing errored, the run reported it as a success, and the
gallery just sorted that photo to the wrong place.
Fenced on path and filename as well as the id, so a replaced row matches zero
rows and is skipped. The candidate query already selects both columns, so no
query change. Knex renders a null value in the object form as `is null` on both
the pg and sqlite3 clients, so a row with a NULL path still matches itself.
Those skipped candidates are now counted rather than dropped. replacePhoto is
not the only writer of path/filename — eventRenameService rewrites both on an
event rename, which is not a content change — and another writer filling
captured_at first lands in the same place. Without a counter they fell out of
the run's arithmetic entirely: success + noExif + failed no longer added up to
the count the operator was shown when they started the job.
The card shows the count only when it is non-zero, and states what is known —
changed by something else, not updated — rather than promising a retry: for the
already-dated case there is nothing to retry, and the Missing Capture Date
figure above is what says whether work is left. Locale coverage: en, de, fr, sl,
with the defaultValue carrying the rest.
Regression test: a replacement landing mid-run leaves captured_at NULL and is
not counted as updated. Verified to fail against the unfenced code on this
branch.
Stable twin of #1184. Both photo sweeps tracked whether they were running in a
module-level variable, which is invisible to every other replica: a status poll
routed to an idle replica reports isRunning false while another is mid-run, and
the next POST starts a second pass over the whole library.
Migration 179 adds one row per job, claimed with a conditional UPDATE whose
affected-row count is the answer. The lease is fenced on a per-claim token so a
runner superseded by a stale takeover cannot renew a claim it has lost or
release one it no longer owns; renewal runs on a timer spanning the claim
through release, since one hung NAS read can outlast the stale window inside a
single iteration. maintenance_jobs is excluded from .picpeak archives.
Gated on settings.edit / settings.view rather than main's system.manage, which
does not exist on this branch — they are what settings.edit was later split
into, so both branches let the same people through.
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
* fix(external-media): record captured_at on import and add a backfill (#1172)
Stable twin of #1179.
External media never went through photoProcessor, so captured_at stayed NULL
for every externally imported photo. The gallery's "Date Taken" sort then
degraded into import order through its own COALESCE fallback — a library
imported in two batches showed the first days of a trip after the last ones.
Ported whole:
- adminExternalMedia.js reads the capture date at import, off the file it has
already opened for the dimensions. Best-effort, like the dimensions.
- A backfill endpoint for photos imported before this, so existing installs
can fix historical rows rather than only new imports. Managed originals go
through resolvePhotoStorageKey + withLocalCopy so S3 installs work; archived
events are excluded because archiving deletes their originals; the run flag
is claimed before the candidate query so two POSTs cannot both start.
- gallery.js carries photos.id as a tiebreaker on all three sorts. A bulk
import writes hundreds of rows inside the same second, so uploaded_at ties
are the normal case and the grid reshuffled between page loads.
One deliberate difference from main: the backfill is gated on settings.edit /
settings.view rather than system.manage / system.view, which do not exist on
this branch. They are what settings.edit was later split into, and main's
migration 175 projects every settings.edit holder forward onto system.manage,
so both branches let exactly the same people through.
* fix(external-media): gate the status card on the permission the button needs (#1172)
The built-in admin role holds settings.view but not settings.edit
(056_add_role_permissions_table.js:63), and StatusTab renders its card and
enabled button purely on a successful status payload. Gating the status
endpoint on settings.view therefore showed every admin a Backfill button whose
every click 403s with no error surfaced.
* fix(gallery): make the Date Taken sort correct on SQLite (#1172)
Same defect as the main twin: photos.captured_at holds three storage classes on
SQLite — an epoch-millisecond integer from managed uploads (photoProcessor.js:441
hands knex a Date), ISO text from external imports and the backfill, and null
falling through to uploaded_at's 'YYYY-MM-DD HH:MM:SS' text. SQLite sorts
INTEGER before TEXT unconditionally, so a 2027 capture came back before a 2020
one, and within the text values the 'T' separator outranked the space.
Normalised in the ORDER BY; Postgres keeps the plain COALESCE, its column being
a real timestamp. Regression tests drive the real gallery route on real SQLite.
* fix(gallery): normalise epoch-integer uploaded_at too, and stop polling a 403 (#1172)
Both follow-ups from the main twin's review, ported.
uploaded_at is not always text on SQLite: a .picpeak restore can carry epoch
milliseconds in from an install that stored them that way, and the fallback
branch read it with substr(), comparing '1830297600000' against
'2020-01-01 00:00:00' as text. Both columns now get the integer/real branch.
The status card also polled every ten seconds regardless of permission. On this
branch that hits every built-in admin — they hold settings.view but not
settings.edit — so each would have had a 403 and a logged denial every ten
seconds for a panel they were never shown.
* style: quote convention in the capture-sort test (#1172)
* fix(capture-dates): skip watcher-imported videos, and make the status counts consistent (#1172)
All three follow-ups from the main twin, ported: the three-marker video filter
(fileWatcher sets type/mime but not media_type, so those rows sat in the
backlog forever), the single-aggregate status counts (two queries could report
a negative backlog mid-import), and the card's render gated on settings.edit as
well as the cached payload.
---------
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
* fix(external-media): store external paths from the media root (#1163)
Stable twin of #1168. Stacked on the #1162 twin, which supplies
deleteDuplicatePhotos.
Importing a second folder into an event silently invalidated every photo
already in it. external_relpath was stored relative to events.external_path,
and every import overwrites that column, so the older rows were rebased onto
the new folder. Nothing errored and the grid still rendered — thumbnails are
written to local storage during the import while the base path is still
correct — so only the things that need the original broke. The reporter had
7547 of 8004 rows pointing into the void.
- external_relpath is now relative to EXTERNAL_MEDIA_ROOT, so a row is
self-describing and nothing an admin does to the event can move it.
- migration 177 folds each event's base into its rows. Where the current
resolution is missing it walks up for an ancestor holding a file of the same
name AND the size the import recorded — existence alone would let a deleted
file adopt an unrelated namesake and serve the wrong original. Rows it
cannot place keep resolving where they resolve today, and the probe is
skipped entirely when the mount is unreachable.
- probing is read-only and runs first; the rewrites and the marker commit
together, so an interrupted fold cannot be folded twice.
- rewrites are staged through a per-row parking value, because a final path
can equal another row's current one; and migration 177 re-throws without the
driver's error code, which run-migrations-safe would otherwise read as
"schema already exists".
- the fold also runs after a .picpeak restore, since knex_migrations is
excluded from the archive, and a failure there is reported rather than
presented as a clean restore.
- drops the duplicate-leaf-segment guess in photoResolver, which papered over
this same double-prefixing.
Divergence from the main twin: no face-scan requeue reordering. Face
recognition is main-only, so the hazard of queueing rows against unconverted
paths does not exist on this branch — in picpeakImportService or in
restoreService.
Verified on this branch: 23 new tests pass, and the four suites carrying
base-relative fixtures were updated. Full suite leaves the same 5 pre-existing
failures as origin/stable, unchanged.
* fix(external-media): the fold's staging value must be storable on Postgres (#1163)
External review found this on this branch first; it was on both.
The two-pass rewrite parks each row on a temporary value, and that value was
written with a leading NUL. SQLite stores NUL in TEXT without complaint;
Postgres rejects it outright — "invalid byte sequence for encoding UTF8: 0x00"
— so migration 177 rolled back on exactly the installs that need the two-pass
repair, and only on the engine most of them run. Restores hit the same wall
and reported the conversion as failed.
The prefix is ordinary text now. Adds a gated Postgres test alongside the
existing picpeakRestorePg one, because a SQLite-only suite structurally cannot
catch this class: restoring the NUL makes exactly the two-pass repair case
fail with that error, and nothing else.
---------
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
* fix(external-media): one row per external file per event (#1162)
Stable twin of #1167.
Two overlapping import-external runs against the same event inserted every
file twice. The route checked for an existing external_relpath and then
inserted, with an fs.stat and a sharp().metadata() read sitting in between — a
window wide enough for both runs to see "not there". A reporter's event held
8004 rows for 6012 distinct paths. Nothing at the storage layer stopped it:
migration 041 created only a NON-unique (event_id, source_origin) index.
- migration 176 removes the existing duplicates and adds a partial unique
index on (event_id, external_relpath), verified against the catalog
afterwards — a failed CREATE INDEX raises 23505 on Postgres, which
run-migrations-safe treats as "schema already exists" and would record as
applied on an install that never got the index.
- dependent rows are removed explicitly rather than by cascade: PicPeak never
sets `PRAGMA foreign_keys = ON`, so on SQLite the declared CASCADE is inert
and a bare delete strands feedback and access-log rows. Guest feedback moves
to the survivor instead of being discarded, keyed on guest identity the way
feedbackService defines it, and the survivor's denormalized counters are
recomputed.
- the route treats a unique violation as a skip, so a writer this process
cannot see converges instead of duplicating, and a second import while one
is running gets a 409.
- a .picpeak taken before migration 176 carries exactly these duplicates, and
suspending FK enforcement does not suspend a unique index — so the restore
drops the index for the load and rebuilds it after running the same dedupe.
Divergences from the main twin, both because the feature is absent here:
faces (no faceProcessor, so no purgePhotoFaces reconciliation — the rows are
still deleted so nothing dangles), admin marks, transfer membership, and
photos.view_count/download_count. The service guards each on hasTable /
hasColumn, so those branches simply do not fire.
Verified on this branch: 36 new tests pass; full suite leaves the same 5
pre-existing failures as origin/stable, unchanged.
* fix(external-media): invalidate the download zip when duplicates are removed (#1162)
External review. Same fix as the main twin.
The pre-built "download everything" archive still contained the duplicate rows
the dedupe had just deleted, so guests kept receiving them. Every ordinary
photo-deletion path calls downloadZipService.invalidate for exactly this
reason.
The columns are cleared rather than the service being called: that service
carries debounce timers and a regeneration queue, which a migration should not
start. getZipInfo already treats a cleared record as a cache miss and rebuilds
on the next request. The stale object is left in storage, as elsewhere.
---------
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Stable twin of #1153.
Everything in the system treats a hidden row as absent, but the per-viewer is_liked heart read the row without looking at is_hidden — so a like the photographer had hidden still showed as liked on a photo whose like_count was zero.
Making that agree exposes the second half: the duplicate check behind like/favorite toggling did not skip hidden rows either, so the now-empty heart, when clicked, found the hidden row and toggled it OFF. Skipping hidden rows there makes the click create a fresh, visible row.
Also carried from review: the per-guest caps, /my-feedback and getEventFeedbackSummary no longer count hidden rows, and unhiding collapses the guest's replacement — skipped when there is no stable identity, since that fallback was 'guest_identifier IS NULL', i.e. other visitors' rows.
Not carried: the my_color_label badge (colour labels are #1044) and the clearScope / singleValueScope visibility fix, neither of which exists on this branch.
Merged with admin privileges: the author cannot self-approve.
Stable twin of #1147, filter half only.
Every filter token on /photos is an OR of two halves: what THIS viewer marked, and what ANYONE marked. The fields built from the second half are gated on show_feedback_to_guests; the filter was not, so with the setting off ?filter=liked still returned exactly the photos other people liked — the membership instead of the count, one token at a time.
The half it left standing was also the wrong half: it read guest_identifier from the guest_id query parameter, which never matched anything, and accepting a caller-supplied identifier was a way back through the gate. Resolved from the request now, hidden rows excluded to match what the viewer can see.
Not carried: the color: token and the photo_admin_marks concurrent-write fix — colour labels and admin marks are not on this branch.
Merged with admin privileges: the author cannot self-approve.
Stable twin of #1151.
The script here carried the identical defect: it computed `storage/events/active/<photo.path>` and fs.access'd it, which does not exist for external or reference rows. #1129 already landed on this branch, so the route was fixed and the script was the remaining half.
Resolution goes through ensureThumbnail, which stable already exports with the external branch intact. Also carried: videos skipped on every marker, skip-vs-generate asked from isThumbnailValid, and a nonzero exit when a photo could not be built.
Not carried: the responsive-tier backfill — THUMBNAIL_WIDTHS and ensureThumbnailAtWidth are #1095/#1109 and do not exist on this branch.
Merged with admin privileges: the author cannot self-approve.
POST /admin/thumbnails/regenerate resolved every source as
storage/events/active/<photo.path> and fs.access'd it. External and reference
rows are not there — their originals live under events.external_path — so every
one failed the check and was counted as an error, while the UI reported success
because the response is sent before the background loop starts.
It now goes through ensureThumbnail, which resolves both source kinds and writes
thumbnail_path back itself. Nulling thumbnail_path stops it short-circuiting on
isThumbnailValid, which matters because the old thumbnail is normally still
readable at exactly the moment someone presses regenerate.
And the half that destroys data: generateThumbnail deleted the target BEFORE
sharp had opened the source, and again in its catch. A source that could not be
read — a NAS mount that blipped — left the previous rendition gone and the
database pointing at it. Across a bulk regenerate that is the whole gallery.
Neither delete was needed: put stages to a temp file and renames atomically, and
put is the last statement in the try so no partial object can exist.
Also: videos filtered out, and the superseded rendition removed only when the
storage key actually moved, compared through the same canonicalisation the
backends apply.
Stable twin of #1134.
Stable backport of #1043 (main: 8809564a).
sqlite → pg restore is allowed for anyone holding backup.restore, from the
upload UI and the CLI alike, gated by the manifest-direction rule in
validateManifest. pg → sqlite stays refused, with an error naming the
supported direction. allowEngineSwitch is removed rather than kept alongside:
one gate, no way to drive the refused direction.
Two resolutions were needed against stable rather than a clean cherry-pick,
both from known main/stable divergences:
- replaceAllTables has no roleSnapshot parameter on this branch, so the call
keeps stable's 4-arg signature while taking the derived { crossEngine }.
- resyncSequences was guarded by `if (allowEngineSwitch)`, which this change
removes — leaving an undefined reference. It now runs unconditionally,
matching main. That also closes a stable-only gap: a same-engine pg → pg
restore previously left identity sequences stale, so the next natural
insert collided on the primary key.
Also exports resyncSequences (the function already existed here, main already
exports it) so the cross-engine suite can drive the post-restore fixup.
Verified on this branch: all four picpeak suites green on SQLite, and 20/20
against a real Postgres 15 with the PICPEAK_PG_TEST_URL-gated cases executing.
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Backport of #1018 to the curated channel; the reporter on #1015 is running v3.45.14.
The slideshow resolved its image as preview_url || hero_url || url. preview_url is only emitted when lightbox_preview_enabled is on (default false), so a default install fell through to hero_url — the 1920x1080 fit:'cover' centre crop built for gallery header banners. object-fit: contain then letterboxed an already-cropped 16:9 frame.
Emits slideshow_url (same aspect-preserved preview tier) unconditionally for image photos; the show prefers it and never falls back to hero_url. preview_url stays gated so the lightbox opt-in is unchanged.
Stable backport combining #860 (never reached stable) and #900:
- jest.config.js gains testTimeout: 120000 — stable still ran on Jest's
5s default for anything unpinned, while its migration chain (134 core
migrations via backports) is nearly as long as beta's.
- All 19 suite-level jest.setTimeout(30000/60000) pins raised to 120s;
local pins override the config default (#860's rationale).
- All 15 hook-ARGUMENT timeout pins on migration-booting beforeAll
hooks raised to 120s (#900's rationale — the 3.97.0-beta.0 release PR
failed on exactly this class on the beta side).
Untouched: the three suites whose pinned hooks don't run migrations
(webhookDelivery, imageProcessor.storage, storageBackend) and
publicQuotes' 30s pin on the rate-limit lockout test.
No test logic changed.
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
* fix(security): close the 5 open Trivy alerts — dep bumps + drop npm from the runtime image
Backend deps:
- postcss 8.5.10 -> 8.5.18 (CVE-2026-45623, GHSA-r28c-9q8g-f849; the pin
exists to force sanitize-html's transitive copy onto a fixed version)
- tar pin/override >=7.5.16 -> >=7.5.21, resolves 7.5.22
(GHSA-r292-9mhp-454m)
Runtime image:
- Remove the npm CLI from the final stage instead of upgrading it: npm's
bundled node_modules ship tar 7.5.19 and brace-expansion 5.0.7 (no npm
release bundles the fixed versions — checked 11.18.0 and 12.0.1), and
npm never runs in production. wait-for-db.sh now invokes the migration
runners via node directly. This ends the recurring npm-bundled-CVE
alert class; the previous 'npm install -g npm@11' line was itself a
patch for the last batch. (stable)
* fix(restore): run post-restore migrations via node — the image ships no npm
restoreService still shelled out to 'npm run migrate:safe' after a
restore; with npm removed from the runtime image that would ENOENT into
the non-fatal catch, silently leaving a restored older backup on a
schema behind the running code until the next container restart. Invoke
migrations/run-migrations-safe.js through node directly, matching
wait-for-db.sh. The PR #596 source-contract test now pins the new
invocation. (stable)
* fix(backup): make backup settings actually apply (#871) (stable)
- Wire the What-to-Backup toggles into the walker: honor
backup_include_thumbnails / backup_include_photos (opt-out,
default ON) and accept the UI's backup_include_archives spelling
for the archived gate (the engine expected _archived, so the
Archives checkbox silently never worked).
- Fix the 167.6 TB dashboard size: file_size_bytes is a bigint that
node-postgres returns as a string, and the S3 path concatenated it
onto the byte counter; coerce to Number at the source.
- Compute the real next scheduled run (cron-parser) and return it as
nextBackup; the UI read a field the API never sent and rendered a
hardcoded 'Not scheduled'. A named schedule label now beats the
stray default cron the UI always sent, which silently turned
weekly schedules into daily 03:00 runs.
- Never back up filesystem noise (.nfs* silly-renames, .DS_Store,
Thumbs.db) and honor backup_exclude_patterns in the walker
(previously rsync-only).
- Remove the compression/encryption toggles from the configuration
UI: no backend implementation exists, and collecting an encryption
passphrase while uploading plaintext is a false promise.
* fix(backup): close the review gaps in the settings wiring (stable)
- The UI's backup_include_archives now beats the migration-seeded
backup_include_archived: every install has the singular key seeded
true, so the alias-only-when-absent lookup made unchecking Archives
a no-op.
- rsync destinations now receive the de-selected What-to-Backup paths
and the noise filters as anchored --exclude args; previously rsync
synced the whole storage root and the walker's selection only shaped
the manifest, which then misreported what was actually transferred.
- Escape regex metacharacters in the walker's glob matcher: '.nfs*'
compiled to /^.nfs.*$/ whose leading dot matched any character, so
files like anfs-photo.jpg were silently dropped from backups.
- The Backup Coverage report now uses the same gate as the walker
(new 'skipped-by-setting' status) instead of re-implementing it
without the opt-out toggles and the archives alias.
* fix(backup): make the coverage diagnostics agree with the walker
- The coverage table shows the alias-aware flag value the gate actually
used, instead of the seeded backup_include_archived shadowed by the
UI's plural key (true next to a 'Gated off' badge).
- skipped-by-setting paths are now counted in the coverage summary
(backend, TS contract, summary card, EN/DE locales) so the totals
reconcile again when Photos or Thumbnails is unchecked.
- The form's thumbnail default now matches the backend's never-saved
fallback (include): the checkbox no longer shows 'off' while
thumbnails are being backed up, and saving an unrelated setting no
longer flips the backup scope. (stable)
* fix(backup): keep custom crons, exclude disabled rows from rsync, normalize flag display
- Saving a named schedule no longer wipes the stored custom cron: the
backend already prefers the label, so the cron field stays inert for
named schedules and is preserved for switching back to Custom. A
custom schedule now validates the 5-field expression before saving
(the backend silently fell back to daily 02:00 on a blank value).
- resolveExcludedBackupPaths now also returns rows disabled via
include_in_default, so rsync excludes them; the enabled-only loader
hid them and rsync transferred their contents anyway.
- The coverage table normalizes flag values like the walker does —
Boolean('false') displayed true beside a gated-off badge. (stable)
Enabling the invoice-dunning built-in suppressed the legacy reminder ladder
but only created runs for invoices sent AFTER enabling — already-sent unpaid
invoices got dunned by neither. Now:
- Turning dunning ON enrolls every open sent/overdue unpaid invoice via
emitWorkflowEvent('invoice.sent') (engine.backfillDunningRuns(), wired into
the workflow enable toggle). Idempotent via the per-(flow,entity) dedup.
- The grace wait is anchored to the invoice's due date: computeWakeAt now
treats { untilVar, delayDays } as "var + offset" (was var-only OR now+offset),
and the built-in's waitGrace becomes { untilVar: 'dueDate', delayDays:
firstDays } (seed v6 -> v7). An already-overdue invoice duns on its real
timeline instead of restarting a fresh grace clock.
Note: the due-date-anchored graph applies to freshly seeded built-ins; an
already-admin-enabled dunning workflow still enrolls via backfill but keeps its
current grace timing until re-seeded.
26 tests as a safety net ahead of decomposition — invoice create/list/
status transitions, adminEvents CRUD via Supertest+SQLite, backup config
parsing and manifest validation.
From the-luap's review:
- Import no longer trusts manifest.tables blindly. It now intersects the
manifest's table list with the real data tables of THIS database
(listDataTables(), which already excludes knex_migrations/_lock) and
drops anything else. A crafted/corrupted .picpeak listing knex_migrations
or a non-existent table can no longer wipe it; skipped tables are logged.
- The Postgres session_replication_role='replica' SET (needs superuser) is
now wrapped: on a managed-PG non-superuser it fails BEFORE any rows are
deleted (transaction rolls back) and surfaces a clear, actionable 400
instead of a cryptic permission error.
- Export: on an archiver error, the temp out dir (a partial plaintext-secret
archive) is now removed instead of orphaned.
Tests (+4, now 26): engine-mismatch rejection, forward-only newer-refused,
non-picpeak rejection, and files/ restored + filesRestored asserted.
Receiving half of the roundtrip. picpeakImportService.importFromPicpeak():
- Validates the manifest: rejects non-picpeak files, a newer format, an
engine mismatch (pg↔pg / sqlite↔sqlite only), and a backup from a NEWER
schema than this instance (forward-only). knex_migrations absence is
tolerated (test harnesses).
- Snapshots the current logged-in admin, then wipes + reloads every table
from the backup NDJSON in one transaction with FK enforcement suspended
(pg: session_replication_role=replica reset before commit; sqlite:
defer_foreign_keys). knex_migrations is never touched, so the target's
schema/migration state is preserved.
- Re-injects the current account so the operator is never locked out; a
backup admin colliding on email is overwritten with the current creds.
- Restores files/ into storage and detects external-media references so the
caller can prompt to reconfigure the mount.
Roundtrip integration test proves: backup data restored, current account
survives a full override (different email → added), and the email-collision
case keeps the operator's password.
First half of the GUI-only backup roundtrip. Adds a self-describing
".picpeak" archive that can be downloaded from one instance and (later)
re-uploaded to another via the web UI only.
- picpeakExportService.createPicpeak(): dumps every table as NDJSON
(tables introspected at runtime — no hardcoded list, won't rot), plus
a manifest (format version, app version, DB engine, latest migration,
per-table row counts + checksums, includePhotos, contains_secrets),
plus files/ (business-docs + uploads always; original gallery photos
only when includePhotos). NDJSON is engine-neutral so the target
rebuilds schema via migrations then loads rows — enabling pg↔pg /
sqlite↔sqlite and forward-only auto-migrate.
- GET /admin/backup/picpeak/export?includePhotos= streams the file and
sets X-Picpeak-Contains-Secrets (the file holds plaintext SMTP pass,
admin hashes, API keys — the UI must warn).
- Purely additive: no existing backup/restore path is touched.
Integration test proves the archive shape, knex-table exclusion, and
row-count/NDJSON consistency (85 tables on the seed schema).
Previously "Continue" on the token step only checked the field was
non-empty; a wrong token wasn't caught until the final submit, after the
user had filled in email + password. Add a non-burning verify:
- backend: POST /setup/verify-token constant-time compares the token
without consuming it (createInitialAdmin still claims it atomically on
submit), gated on no-admin-exists and rate-limited like /setup/admin.
- frontend: step-1 "Continue" calls verifyToken and only advances on a
valid token; a wrong token shows the invalidToken error on the field,
429 -> too-many-attempts, 409 -> redirect to login.
Adds integration tests for accept-without-burn / reject / closed-once-set.
Blockers:
- SetupPage now mirrors the server password rule (>=8 with upper/lower/digit) so
a green client isn't bounced by the server; server errors carry a `field`
(routes/setup.js) that the client maps to a translated key instead of
rendering raw English. New i18n: setup.invalidToken, setup.passwordRequirements.
- picpeak-setup.sh: the ADMIN_CREDENTIALS.txt block no longer dead-ends on the
wizard path — when no legacy admin was seeded it prints the one-time setup
token (from data/SETUP_TOKEN / docker compose logs) and points at /setup.
Concern:
- createInitialAdmin creates the admin + burns the token in ONE transaction,
atomically claiming the token (null-if-present, expect 1 row) so a
double-submit can't create two super_admins. Cross-DB (whereNotNull, trx-only
writes). Added a concurrency test.
Nits:
- SetupPage redirects to /login when /setup/status errors (no form flash on a
configured instance).
- Dropped the unused DATABASE_URL from docker-compose.yml.
- Documented why secrets are chmod 644 (three different reader users).
Issue 3 from #699 (@alexvaltchev's report): expose a custom-named short
URL per event that bots scrape for OG previews and browsers redirect to
the underlying gallery. WhatsApp / iMessage / Facebook cache the OG
metadata by the URL they crawl, so the SHORT URL becomes the cache key
— admins can rotate or split-test underlying gallery URLs without
re-pushing a fresh link to clients.
Additive feature; no existing route, table, or column is modified.
## Backend
- `gallery_short_urls` table (migration 150): id, short_slug UNIQUE,
event_id FK CASCADE, target_path TEXT, created_by/at, hit_count,
last_hit_at, deleted_at/by. hasTable-guarded so the migration is
idempotent on re-run.
- `src/services/galleryShortUrlService.js` — validator + CRUD +
resolver. Slug rules: `/^[a-z0-9](?:[a-z0-9-]{0,62}[a-z0-9])?$/`,
reserved blocklist (admin, api, auth, gallery, og, s, login, ...).
target_path snapshots at create-time from the event + global
short-URL toggle, so a later flip of the toggle does NOT silently
change where existing short URLs resolve.
- `src/routes/adminShortUrls.js` — `GET/POST
/api/admin/events/:eventId/short-urls`, `DELETE
/api/admin/short-urls/:id`. Structured errors: 400 INVALID_SLUG,
409 SLUG_TAKEN (with `suggested`), 404 EVENT_NOT_FOUND. Gated by
events.view / events.edit + requireEventOwnership.
- `server.js` /s/:shortSlug public route. Bot UA → server-render the
same OG metadata the existing /og/gallery/<slug> handler produces,
then override og:url to point at /s/<shortSlug> itself (cache-key
invariant — social platforms key by the URL they scrape).
Browser UA → 302 to target_path. Soft-deleted slug → 410 Gone
(intentional-delete signal, distinct from 404 unknown slug).
Hit accounting is fire-and-forget.
## Frontend
- `services/shortUrls.service.ts` — list/create/remove.
- `components/admin/ShortUrlsCard.tsx` — per-event card on the
EventDetailsPage. Form for custom or auto-generated slug, list with
copy-to-clipboard + soft-delete. SLUG_TAKEN error surfaces the
service's `suggested` slug with a "use suggested" button.
- i18n: events.shortUrls.* added to EN + DE.
## Tests
78 new tests, all passing:
- `__tests__/utils/galleryShortUrlValidation.test.js` (48) — pure-
function tests for validateSlug: accepts/rejects, reserved-slug
blocklist, path-traversal + URL-injection vectors.
- `__tests__/integration/galleryShortUrls.test.js` (19) — service
layer against a real SQLite DB. Covers custom + auto-generated
slugs, collision + SLUG_TAKEN + suggested, target_path
snapshotting (backward-compat invariant), soft-delete + slug
rotation, hit counting.
- `__tests__/integration/galleryShortUrlRoute.test.js` (11) —
HTTP-level: 302 redirect for browser UA, 200 + OG HTML for bot UA,
og:url canonical points at /s/<slug>, 410 for soft-deleted +
orphaned events, 404 unknown + malformed.
Regression sweep: 47 existing migration-chain integration tests still
pass; migration 150 is additive only.
## Backward compatibility
- Existing `/gallery/<slug>`, `/gallery/<32-hex-share-token>`,
`/gallery/<slug>/show/<token>`, `/og/gallery/<slug>`,
`/og/gallery/<slug>/cover` routes are untouched.
- The `/s/` namespace is new; no existing route lives there.
- Migration 150 only ADDs the new table — no ALTERs on existing
schema, no destructive changes.
- target_path is snapshotted at create-time so flipping the global
"Use short gallery URLs" setting after a short URL exists does NOT
change where that short URL resolves.
The customer's accept/decline can be toggled for crm_quotes_accept_window_minutes
(default 15) before it locks, and the public page promises exactly that. But the
booking workflow fired on the FIRST accept click and immediately converted the
quote (status -> 'converted'), so a decline within the window was rejected
('Quote cannot be responded to in status converted') — the grace period was dead
on arrival.
recordResponse / adminAcceptQuote now DEFER the workflow emit while the toggle
window is open; the new scheduler sweep finalizeQuoteResponses fires the FINAL
status once response_locked_at passes (idempotent via the new
quotes.workflow_response_emitted_at column, migration 149). A response recorded
with the window already closed (0-min window, or admin decline which locks
immediately) still emits inline. So toggling accept->decline->accept inside the
window converts at most once, for the final state, after the customer's grace
period — and a plain decline never converts.
Trade-off: with the hourly CRM scheduler, the booking flow now starts up to ~1h
after the window locks instead of instantly. Acceptable — the flow gates on admin
review anyway, and the alternative (graph-level wait) wouldn't reach already-
enabled built-ins (admin_toggled_at blocks re-seed).
Adds a finalize sweep test (deferred while open, fires + stamps once locked,
idempotent).
A quote with no explicit payment timing falls back to a single after_delivery
installment. spawnInstallmentInvoices marked those 'pending_delivery' even in
hold mode, so the booking flow's send_document -> sendInvoice threw 'Cannot send
invoice with status pending_delivery', the run failed, and no invoice email went
out (the symptom: approve the quote->invoice flow, receive nothing).
In hold mode the flow's review gate + explicit send_document IS the delivery
release, so a held invoice is always 'scheduled' (editable + sendable) regardless
of trigger; scheduled_send_at stays null so the scheduler never auto-sends it.
Non-hold after_delivery invoices keep 'pending_delivery' as before.
Adds a regression test (default after_delivery term -> draft -> scheduled+null).
These were the last guard-stubbed actions — offered in the builder palette but
refused on enable. Now all three are real, backed by existing converters:
- prepare_gallery: alias of prepare_event (a gallery IS an event in picpeak).
- reserve_date: convertToEvent({ skipInvoices: true }) — a pure draft date hold
with no money documents (new skipInvoices option on convertToEvent).
- prepare_quote: createQuote (customer entity) or duplicateQuote (quote entity),
producing a status='draft' quote; idempotent via ctx.vars.preparedQuoteId.
With no stubs left, the enable-guard switches from a hardcoded DOCUMENT_ACTIONS
list to a registry lookup: an action node whose config.action has no registered
handler is unimplementable. This can't drift from what the engine can run and
also catches typo'd/future actions. (Fixes the enable-route node mapping to
carry node.type so the action-node filter matches.)
Extends the single-connection SQLite in-trx deadlock fixes to the quote-create
path (prepare_quote runs unattended): nextQuoteNumber reads getAppSetting
through trx, createQuote logs via trx and hoists its hasColumnCached schema-drift
checks before the transaction.
Adds tests for reserve_date (no invoices), prepare_quote (draft, no deadlock),
and registry coverage; retargets the enable-guard refusal test at a genuinely
unregistered action. Full backend suite: 985 passed, 1 skipped.
The booking_full / booking_simple flows go prepare_event -> prepare_invoice,
but prepare_event was still a guard-stub, so enabling either flow returned
409 'uses actions that aren't implemented: prepare_event'.
prepare_event now calls convertToEvent({ hold: true }): convertToEvent already
creates the event as is_draft=true AND schedules its invoices, so this creates
those invoices on HOLD (scheduled_send_at NULL) and stashes their ids in
ctx.vars.preparedInvoiceIds. The downstream prepare_invoice already short-
circuits on a populated preparedInvoiceIds, so it ADOPTS the event's held
invoices instead of calling convertToInvoiceOnly again (which would both
double-create and throw ALREADY_CONVERTED_TO_EVENT). The review gate, the
wait-until-event-date, and send_document then issue those same invoices.
send_document(event)=publish is intentionally left a graceful skip — the
gallery is published manually after photos are uploaded, not auto-published
on an empty draft.
convertToEvent gains the same single-connection SQLite deadlock fixes as
convertToInvoiceOnly (getAppSetting reads through trx; logActivity moved after
commit) since prepare_event runs unattended, returns invoiceIds (incl. the
idempotent already-converted re-entry, which recovers them by event_id), and
removes prepare_event from the enable-guard list.
Adds a convertToEvent hold-mode test (draft event + held invoices + quote
linkage) and updates the enable-guard test to a still-stub action
(prepare_gallery). Full backend suite: 982 passed, 1 skipped.
Implements the draft-seam booking cutover so the booking_invoice_only flow
becomes enableable. The booking flows trigger on quote.accepted, so the run
entity is the quote:
- prepare_invoice: convertToInvoiceOnly({draft:true}) creates the invoice(s)
on HOLD (scheduled_send_at NULL, status stays 'scheduled') so the scheduler
never auto-sends before the review gate; crash-recovery recovers drafts by
the quote's deal_uuid. Stores ids in ctx.vars.preparedInvoiceIds.
- prepare_contract: createFromQuote (idempotent via converted_contract_id).
- send_document: dispatches the prepared draft (invoice -> sendInvoice each id,
contract -> sendContract).
- resolveActor: quote creator -> workflow creator -> first admin.
- prepare_contract/prepare_invoice/send_document removed from the enable-guard
list; prepare_event/prepare_quote/prepare_gallery/reserve_date still guarded,
so booking_full/booking_simple stay blocked until the event-path increment.
Fixes a latent single-connection SQLite deadlock these unattended paths would
hit: getAppSetting/logActivity/adminActor read or write the global db, which
deadlocks when issued inside an open knex transaction. Thread the active trx
through getAppSetting, logActivity, nextInvoiceNumber, nextContractNumber, the
spawnInstallmentInvoices audit log, and hoist adminActor before createFromQuote's
transaction. convertToInvoiceOnly now logs after commit and returns invoiceIds.
Adds bookingCutover integration test (hold-mode null send-at, normal scheduled
contrast, contract path no-deadlock) and a route test that the now-implemented
booking invoice actions can be enabled.
When an event has no inline customer_email/host_email but has customer
account(s) assigned (event_customer_assignments), the pre-event reminder now
sends to those registered customers instead of skipping with no_recipient.
Recipients sent to an assigned account are queued WITHOUT eventId so the
language resolver uses the customer's preferred_language (vs the event's
language for inline-email sends). Applies to both the flow path
(sendReminderForEvent) and the legacy pass. The gallery-ready mail deliberately
does NOT fall back to accounts — only the reminder does. Test covers the
no-inline-email + assigned-customer case.
composePayload pre-formatted event_date to DD.MM.YYYY, but emailProcessor runs
date variables through formatDate(value, language) — new Date("25.06.2026")
can't parse → the email rendered "Invalid Date". Pass the raw event_date and let
the processor localise it, matching the expiry mailer's contract. Pre-existing
in the migration-143 composePayload (dormant while the legacy pass was gated
off); surfaced once the pre_event_email flow ran.
Replaces the one-shot guarded POST with the maintainer's intended end-state: the
webhook node now references a CONFIGURED webhook subscription (Settings →
Webhooks) and enqueues a real webhook_deliveries row via
webhookService.enqueueForWebhook. Delivery then rides the existing worker
pipeline, inheriting — not reimplementing — per-delivery SSRF re-validation
(validateExternalUrl / GHSA-wmjx-pc37-272r), HMAC signing with the
subscription's secret, retry/backoff, and the deliveries audit log.
- webhookService.enqueueForWebhook(webhookId, eventType, data): enqueue for one
active subscription, bypassing fire()'s event-type matching. No schema change.
- webhook action: config.webhookId; unset/missing/inactive → observable skip;
dry-run does not enqueue. event_type = workflow.<trigger>.
- Editor: webhook node config is now a subscription dropdown (was a raw URL),
fed by the admin webhooks list, with a hint pointing to Settings → Webhooks.
- EN/DE strings; test asserts enqueue + dry-run no-op + inactive skip.
Second-review loose end: the `webhook` node type passed validation but had no
registered handler → engine dispatched to registry.getAction('webhook') →
undefined → every run silently skipped. An enabled webhook flow no-op'd.
Register a real `webhook` action (covers both the webhook node type and the
"Call a webhook" action). It POSTs the run context to config.url, guarded by
validateExternalUrl — the same NAT64/private-range SSRF protection the webhook
delivery worker uses (GHSA-wmjx-pc37-272r) — with no redirects and a timeout,
unless WEBHOOK_ALLOW_PRIVATE_URLS=true (local-dev opt-out). Missing URL /
rejected URL / network error record an observable skipped step, not a crash.
So the action is now implemented → it passes the enable guard legitimately.
Test covers dry-run, missing-url, and metadata-IP (169.254.169.254) rejection.
Renaming an event type's slug_prefix is editable in the UI but previously
orphaned everything keyed on the old slug: existing events/quotes (their
event_type) detached, and the authored per-type pre-event reminder template
(event_reminder_<slug>) was left behind → reminders fell back to default.
updateEventType now cascades atomically when the slug changes: re-points
events.event_type + quotes.event_type old→new and renames the
event_reminder_<old> template to <new> (guarded so it never clobbers an existing
target). So a photographer can rename a type to e.g. "concert" and the edited
subject/body follow. Column check resolved before the transaction (avoids the
SQLite global-read-in-trx deadlock). Tests cover the cascade + no-clobber.
The reminder template family (prefix) is now chosen on the notify_pre_event
block via config.templateGroup (default 'event_reminder'); within that group the
exact template is still auto-resolved per event type:
<group>_<eventType> if authored → else <group>_default
So an admin can point a flow at a different reminder family, while wedding/
birthday/… routing and the catch-all fallback stay automatic. resolveTemplateKey
now takes (eventType, group) and tolerates a trailing "_" on the group.
Editor: notify_pre_event (+ the gallery notify actions) added to the action
dropdown, with a "Reminder template group" field and hint. Seed sets
templateGroup='event_reminder' on the built-in (v4). EN/DE strings. Tests cover
the per-type / group-default resolution.
The reminder query joined customer_accounts on events.customer_account_id — a
column the events table doesn't have (events store the recipient inline as
customer_email/host_email, like the gallery emails). So the query threw, the run
failed, and no pre-event email went out for an event that has an email but no
CRM customer account. Latent in the legacy pass (gated off by default); surfaced
the moment the pre_event_email flow ran notify_pre_event.
Both runEventReminderPass and sendReminderForEvent now read the recipient from
the event's own columns (customer_email || host_email, name from
customer_name || host_name) via SELECT events.* — no join, safe on installs
predating the customer_email column. Regression test covers an event with a
direct email and no customer account.
- gate decision with no matching edge → run fails (not silent done)
- enabled-based mutex: legacy reminder pass stands down only when the flow is on
- built-ins now seeded disabled (v6/v2/v3); re-seed flips never-touched defaults
but preserves an admin_toggled_at-owned flow
- route: rejects unknown node type; refuses enabling a flow with unimplemented actions
A quote can now choose which flow runs on acceptance instead of every enabled
quote.accepted flow firing. Migration 147 adds quotes.booking_workflow_id; the
editor shows a "Booking workflow (on acceptance)" dropdown listing the
quote.accepted flows (workflow-engine flag only); emitQuoteEvent passes it as
the new emitWorkflowEvent targetWorkflowId so ONLY the picked flow runs (still
gated on enabled + trigger match → a disabled/None selection runs nothing).
Adds the booking_invoice_only built-in (quote.accepted → prepare invoice →
review gate → send; no event/gallery, no wait), the variant requested for
shoots billed without an online gallery. Disabled stub like the other booking
flows until the prepare_*/send_document cutover.
Tests: targetWorkflowId runs only the selected flow; invoice-only built-in has
no wait/prepare_event.
- Booking built-ins reordered: prepare the invoice EARLY (admin adjusts line
items), admin approves at the review gate whenever, then the wait holds
dispatch until the event date and it sends itself. prepInvoice → reviewInvoice
→ waitEvent → sendInvoice (both booking_full and booking_simple; v3).
- Flow editor now reads/edits/saves trigger_config; the pre-event "days before
event" lead time is editable in the canvas toolbar (was only in settings,
which the cutover removed — closing that gap).
- Dashboard: pending-approvals card under "Events Expiring Soon" (workflows flag
+ non-empty only), with inline Confirm/Deny.
Confirms the design: a gate's confirm edge can feed a wait, so an admin OK
before the event parks the run at the wait and the scheduler dispatches on the
date. New test covers confirm-early-then-wait-dispatches.
Seed gallery_expiring / gallery_expired built-ins and make the live automations
flow-owned, with zero feature loss:
- New delegating actions (notify_gallery_expiring / notify_gallery_expired /
notify_pre_event) call the EXISTING send functions, so the engine path is
byte-identical to the legacy hourly checker/pass (same templates, recipients,
variables, dedup, per-event overrides, sent_at idempotency).
- Cutover built-ins (invoice_dunning, gallery_expiring, gallery_expired,
pre_event_email) now ship ENABLED; booking flows stay disabled (stubs).
- The legacy paths stand down via existence-based isBuiltinFlowPresent guards:
once a built-in is seeded (flag on) the engine is the single switch — flow
enabled = it sends, flow disabled = off — so no double-send and reminders/
expiry emails can still be fully turned off.
- emitDueEventReminders now honours the per-event reminder controls
(disabled / offset / sent_at) so pre-event timing is faithful; fixed a
Number(null)===0 offset bug.
Settings UI cutover is gated on the `workflows` flag (default off): when the
engine is live, the dunning reminder schedule (CRM settings) and the pre-event
global toggle (Reminder emails) are replaced with a "now in Workflows" callout;
when it's off, the legacy controls stay so flag-off installs lose nothing. The
late-fee math and installment-trigger defaults stay (fee math / scheduler-owned).
Split/installment invoices intentionally remain scheduler-driven (no flow).
Booking built-ins now gate every outbound document on an explicit admin OK:
prepare_* drafts the doc, the admin adjusts line items/terms, confirms the
"Review … before sending" gate, and only then does send_document fire. Added to
booking_full (contract + invoice) and booking_simple (invoice); seed versions
bumped so the disabled built-ins self-heal.
Migrated the remaining time- and event-driven triggers into the engine, all
additive / best-effort / fail-closed (no behaviour change when the flag is off):
- gallery.published (event creation)
- gallery.expiring + gallery.expired (expiration checker, alongside the email)
- quote.sent (was queued but never emitted — gap closed)
- contract.sent + contract.signed (sent, fully-signed via counter-sign or wet upload)
- customer.created (direct add + invitation accept)
- invoice.overdue (status→overdue flip, deduped per invoice)
Editor trigger list extended to match. Tests assert the review gates wire
confirm→send on both booking flows.
The workflow/dunning suites boot the full core-migration set in beforeAll via
bootCrmDb. In isolation that's ~1.3s, but under full-suite parallel load on a
small CI runner it can exceed Jest's 5s default, timing out beforeAll and
failing every test in the file (the CI flake). Match the existing pattern used
by the other migration-heavy CRM suites (discountLineItems, incomingInvoiceRebill)
and set jest.setTimeout(30000) on workflowEngine, workflowRoutes and
invoiceDunning.
Three more editable built-in flows, seeded disabled like the dunning ladder:
- booking_full: quote.accepted → prepare/send contract → admin "signed?" gate
→ create event → wait to event date → prepare/send invoice
- booking_simple: the no-contract path (quote.accepted → event → invoice)
- pre_event_email: customer reminder + admin heads-up, fired daysBefore the
event date
The booking document actions stay stubs (observable skipped steps) until the
booking cutover. pre_event_email uses the already-wired send_email action, so
it is functional once enabled — backed by a new scheduler emitter
(emitDueEventReminders) that fires event.date_approaching for events entering a
flow's lead window, deduped per event. Refactors the boot seeder to a built-in
registry so each flow self-heals on its own SEED_VERSION.
Engine testRun() walks the whole graph immediately: waits pass through,
gates auto-confirm, side-effecting actions short-circuit to {dryRun, would}
so no real emails go out. POST /admin/workflows/:id/test-run returns the
run status + per-node step log. Admin list gets a flask button that opens
a result modal with an optional entity id (e.g. invoice) for conditions.
Closes the crash-safety gap: a run left in running/pending by a crash had
nothing to resume it (the scheduler only wakes 'waiting'). Adds a heartbeat
(workflow_runs.updated_at, stamped on every node advance + start/resume) and a
recoverStaleRuns() sweep that re-enters runs whose heartbeat has gone stale
(>10 min) from their persisted node. Runs on the scheduler tick AND the boot
tick, so a restart catches anything stranded during downtime.
Re-entry is at-least-once (the current node may re-execute) — loop counters +
the late-fee math are idempotent, so the only residual risk is a duplicate
reminder email. An attempts counter (migration 145, cap 5) marks a run failed
instead of recovering a node that reliably crashes the process (crash-loop
backstop). Flag-gated. Tests: orphan-resume + crash-loop cap.
Covers the tax-sensitive bits the dunning rework added (previously untested):
flat vs percent fee, the VAT toggle applying the org rate AND no-op'ing when
the org has no VAT rate, per-reminder accumulation (2nd=1x / 3rd=2x), the
invoice total staying immutable while the fee is tracked, and the 3-reminder
cap. Exports the fee resolvers + applyReminder for testing; PDF render stubbed
(flaky in CI, verified manually). 6/6 pass.
New escalate_to_collections action: when the 3-reminder loop ends still
unpaid, consolidate ONE email to the admin — customer data, outstanding
(invoice + late fees − paid), and the invoice PDF attached — ready to forward
to an Inkasso agency / for Betreibung. Internal mail, sent immediately; does
NOT touch the invoice. New invoice_collections_handoff email template (en +
native de, seeded by the boot self-heal). Wired into the built-in dunning
flow: loop 'exit' → collections → end (seed v4, re-seeds the disabled
built-in). Selectable + labelled in the canvas editor. Tests 9/9, tsc 0,
build green.
- Late fee can now be a FLAT amount OR a PERCENTAGE of the invoice gross
(crm_invoices_late_fee_type/_percent, migration 143; defaults preserve the
current flat behaviour).
- Fee is charged from the 2nd reminder onward and accumulates per fee-bearing
reminder (2nd = 1×, 3rd = 2×), computed from the level so re-applying a level
never stacks. New resolvePerReminderFeeMinor() shared by applyReminder + the
payment-check fee preview.
- Reminder ladder extended to 3 levels (caps raised in sendReminder +
recordPaymentCheckAction); the built-in dunning flow now loops 3× (seed v3,
re-seeds the disabled built-in on boot).
- Settings UI: flat/percent toggle + percent field, and a prominent AGB
callout — a late fee is only enforceable if the concrete amount is stated in
the terms (Mara's wording), 'verify with your Treuhänder'. en + native de.
The fee math is examples-only / Treuhänder-verify; issued invoices stay
immutable (the fee is tracked in late_fee_amount_minor, not folded into the
original total). Tests 17/17, tsc 0, build green.
Paul Nothaft